a a @sdZddlZddlZddlmZmZmZmZm Z m Z ddlm Z m Z m Z ddlZddlZddlmZddlZddlmZddlmZmZgd ZGd d d eZd d ZddZddZddZe d!d!dZ"ddZ#dHddZ$e#e$dId!d"Z%Gd#d$d$Z&Gd%d&d&Z'Gd'd(d(Z(d)d*Z)Gd+d,d,e'Z*Gd-d.d.Z+d/!e"d0<Gd1d2d2e*Z,Gd3d4d4e,Z-Gd5d6d6e*Z.Gd7d8d8e*Z/Gd9d:d:e*Z0Gd;d<dd>e'Z2d?d@Z3e3dAe,Z4e3dBe-Z5e3dCe.Z6e3dDe0Z7e3dEe/Z8e3dFe1Z9e3dGe2Z:dS)Ja Nonlinear solvers ----------------- .. currentmodule:: scipy.optimize This is a collection of general-purpose nonlinear multidimensional solvers. These solvers find *x* for which *F(x) = 0*. Both *x* and *F* can be multidimensional. Routines ~~~~~~~~ Large-scale nonlinear solvers: .. autosummary:: newton_krylov anderson General nonlinear solvers: .. autosummary:: broyden1 broyden2 Simple iterations: .. autosummary:: excitingmixing linearmixing diagbroyden Examples ~~~~~~~~ **Small problem** >>> def F(x): ... return np.cos(x) + x[::-1] - [1, 2, 3, 4] >>> import scipy.optimize >>> x = scipy.optimize.broyden1(F, [1,1,1,1], f_tol=1e-14) >>> x array([ 4.04674914, 3.91158389, 2.71791677, 1.61756251]) >>> np.cos(x) + x[::-1] array([ 1., 2., 3., 4.]) **Large problem** Suppose that we needed to solve the following integrodifferential equation on the square :math:`[0,1]\times[0,1]`: .. math:: \nabla^2 P = 10 \left(\int_0^1\int_0^1\cosh(P)\,dx\,dy\right)^2 with :math:`P(x,1) = 1` and :math:`P=0` elsewhere on the boundary of the square. The solution can be found using the `newton_krylov` solver: .. plot:: import numpy as np from scipy.optimize import newton_krylov from numpy import cosh, zeros_like, mgrid, zeros # parameters nx, ny = 75, 75 hx, hy = 1./(nx-1), 1./(ny-1) P_left, P_right = 0, 0 P_top, P_bottom = 1, 0 def residual(P): d2x = zeros_like(P) d2y = zeros_like(P) d2x[1:-1] = (P[2:] - 2*P[1:-1] + P[:-2]) / hx/hx d2x[0] = (P[1] - 2*P[0] + P_left)/hx/hx d2x[-1] = (P_right - 2*P[-1] + P[-2])/hx/hx d2y[:,1:-1] = (P[:,2:] - 2*P[:,1:-1] + P[:,:-2])/hy/hy d2y[:,0] = (P[:,1] - 2*P[:,0] + P_bottom)/hy/hy d2y[:,-1] = (P_top - 2*P[:,-1] + P[:,-2])/hy/hy return d2x + d2y - 10*cosh(P).mean()**2 # solve guess = zeros((nx, ny), float) sol = newton_krylov(residual, guess, method='lgmres', verbose=1) print('Residual: %g' % abs(residual(sol)).max()) # visualize import matplotlib.pyplot as plt x, y = mgrid[0:1:(nx*1j), 0:1:(ny*1j)] plt.pcolormesh(x, y, sol, shading='gouraud') plt.colorbar() plt.show() N)normsolveinvqrsvd LinAlgError)asarraydotvdot)get_blas_funcs)getfullargspec_no_self)scalar_search_wolfe1scalar_search_armijo)broyden1broyden2anderson linearmixing diagbroydenexcitingmixing newton_krylovc@s eZdZdS) NoConvergenceN)__name__ __module__ __qualname__rre/Users/vegardjervell/Documents/master/model/venv/lib/python3.9/site-packages/scipy/optimize/nonlin.pyrsrcCst|SN)npabsolutemaxxrrrmaxnormsr#cCs*t|}t|jtjs&t|tjdS|S)z:Return `x` as an array, of either floats or complex floatsdtype)rrZ issubdtyper%Zinexactfloat_r!rrr _as_inexactsr'cCs(t|t|}t|d|j}||S)z;Return ndarray `x` as same array subclass and shape as `x0`__array_wrap__)rZreshapeshapegetattrr()r"x0wraprrr _array_likesr-cCs"t|sttjSt|Sr)risfiniteallarrayinfrvrrr _safe_norms r4z F : function(x) -> f Function whose root to find; should take and return an array-like object. xin : array_like Initial guess for the solution a iter : int, optional Number of iterations to make. If omitted (default), make as many as required to meet tolerances. verbose : bool, optional Print status to stdout on every iteration. maxiter : int, optional Maximum number of iterations to make. If more are needed to meet convergence, `NoConvergence` is raised. f_tol : float, optional Absolute tolerance (in max-norm) for the residual. If omitted, default is 6e-6. f_rtol : float, optional Relative tolerance for the residual. If omitted, not used. x_tol : float, optional Absolute minimum step size, as determined from the Jacobian approximation. If the step size is smaller than this, optimization is terminated as successful. If omitted, not used. x_rtol : float, optional Relative minimum step size. If omitted, not used. tol_norm : function(vector) -> scalar, optional Norm to use in convergence check. Default is the maximum norm. line_search : {None, 'armijo' (default), 'wolfe'}, optional Which type of a line search to use to determine the step size in the direction given by the Jacobian approximation. Defaults to 'armijo'. callback : function, optional Optional callback function. It is called on every iteration as ``callback(x, f)`` where `x` is the current solution and `f` the corresponding residual. Returns ------- sol : ndarray An array (of similar array type as `x0`) containing the final solution. Raises ------ NoConvergence When a solution was not found. )Z params_basicZ params_extracCs|jr|jt|_dSr)__doc__ _doc_parts)objrrr_set_docsr8krylovFarmijoTc sh| dur tn| } t|||| || d}tfdd}}t|tj}||}t|}t|}| | |||dur|dur|d}nd|j d}| durd} n | d urd} | d vrt d d }d }d}d}t |D]$}||||}|rq&t|||}|j||d }t|dkr8t d| rXt||||| \}}}}nd}||}||}t|}|| || r| ||||d|d}||d|krt||}nt|t|||d}|}|rtjd|| ||ftjq|r"tt|nd}| rZ|j|||dkddd|d}t||fSt|SdS)a Find a root of a function, in a way suitable for large-scale problems. Parameters ---------- %(params_basic)s jacobian : Jacobian A Jacobian approximation: `Jacobian` object or something that `asjacobian` can transform to one. Alternatively, a string specifying which of the builtin Jacobian approximations to use: krylov, broyden1, broyden2, anderson diagbroyden, linearmixing, excitingmixing %(params_extra)s full_output : bool If true, returns a dictionary `info` containing convergence information. raise_exception : bool If True, a `NoConvergence` exception is raise if no solution is found. See Also -------- asjacobian, Jacobian Notes ----- This algorithm implements the inexact Newton method, with backtracking or full line searches. Several Jacobian approximations are available, including Krylov and Quasi-Newton methods. References ---------- .. [KIM] C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations". Society for Industrial and Applied Mathematics. (1995) https://archive.siam.org/books/kelley/fr16/ N)f_tolf_rtolx_tolx_rtoliterrcstt|Sr)r'r-flatten)zFr+rrznonlin_solve..r dTr:F)Nr:wolfezInvalid line searchg?gH.?g?gMbP?)tolrz[Jacobian inversion yielded zero vector. This indicates a bug in the Jacobian approximation.?z%d: |F(x)| = %g; step %g z0A solution was found at the specified tolerance.z:The maximum number of iterations allowed has been reached.)r rJ)ZnitZfunstatussuccessmessage)r#TerminationConditionr'r@rZ full_liker1r asjacobiansetupcopysize ValueErrorrangecheckminr_nonlin_line_searchupdater sysstdoutwriteflushrr- iteration) rCr+jacobianr?verbosemaxiterr;r<r=r>Ztol_normZ line_searchcallbackZ full_outputZraise_exception conditionfuncr"dxFxFx_normgammaZeta_maxZ eta_tresholdetanrKrHsZ Fx_norm_newZeta_AinforrBr nonlin_solves-       rl:0yE>{Gz?c sdg|gt|dgttdfdd fdd}|dkrxt|dd |d \}} } n&|d krtdd |d \}} |durd }||dkr̈d}n}t|} ||| fS)NrrJTcsT|dkrdS|}|}t|d}|rP|d<|d<|d<|S)NrrJ)r4)rjstoreZxtr3p)rdrctmp_Fxtmp_phitmp_sr"rrphiys   z _nonlin_line_search..phics0t|d}||dd||S)Nr F)ro)abs)rjds)rtrdiffs_normrrderphisz#_nonlin_line_search..derphirGrn)Zxtolaminr:)rzrI)T)rrr) rcr"rerdZ search_typerwZsminryrjZphi1Zphi0rfr) rdrcrtrwrxrqrrrsr"rrWrs,      rWc@s.eZdZdZdddddefddZddZdS)rNz Termination condition for an iteration. It is terminated if - |F| < f_rtol*|F_0|, AND - |F| < f_tol AND - |dx| < x_rtol*|x|, AND - |dx| < x_tol NcCsx|durttjjd}|dur(tj}|dur6tj}|durDtj}||_||_||_||_||_ ||_ d|_ d|_ dS)NgUUUUUU?r) rfinfor&epsr1r=r>r;r<rr?f0_normr])selfr;r<r=r>r?rrrr__init__s zTerminationCondition.__init__cCs|jd7_||}||}||}|jdur<||_|dkrHdS|jdurbd|j|jkSt||jko||j|jko||jko||j|kS)Nr rrJ) r]rr}r?intr;r<r=r>)r~fr"rdZf_normZx_normdx_normrrrrUs         zTerminationCondition.check)rrrr5r#rrUrrrrrNs   rNc@s:eZdZdZddZddZdddZd d Zd d Zd S)Jacobiana Common interface for Jacobians or Jacobian approximations. The optional methods come useful when implementing trust region etc., algorithms that often require evaluating transposes of the Jacobian. Methods ------- solve Returns J^-1 * v update Updates Jacobian to point `x` (where the function has residual `Fx`) matvec : optional Returns J * v rmatvec : optional Returns A^H * v rsolve : optional Returns A^-H * v matmat : optional Returns A * V, where V is a dense matrix with dimensions (N,K). todense : optional Form the dense Jacobian matrix. Necessary for dense trust region algorithms, and useful for testing. Attributes ---------- shape Matrix dimensions (M, N) dtype Data type of the matrix. func : callable, optional Function the Jacobian corresponds to c sbgd}|D]4\}}||vr,td||durt|||qtdr^fdd_dS)N) rrXmatvecrmatvecrsolveZmatmattodenser)r%zUnknown keyword argument %srcsSr)rrr~rrrDrEz#Jacobian.__init__..)itemsrSsetattrhasattr __array__)r~kwnamesnamevaluerrrrs  zJacobian.__init__cCst|Sr)InverseJacobianrrrraspreconditionerszJacobian.aspreconditionerrcCstdSrNotImplementedErrorr~r3rHrrrrszJacobian.solvecCsdSrrr~r"rCrrrrXszJacobian.updatecCs:||_|j|jf|_|j|_|jjtjur6|||dSr)rcrRr)r% __class__rPrrXr~r"rCrcrrrrPs zJacobian.setupN)r) rrrr5rrrrXrPrrrrrs %  rc@s,eZdZddZeddZeddZdS)rcCs>||_|j|_|j|_t|dr(|j|_t|dr:|j|_dS)NrPr)r^rrrXrrPrr)r~r^rrrr$s  zInverseJacobian.__init__cCs|jjSr)r^r)rrrrr)-szInverseJacobian.shapecCs|jjSr)r^r%rrrrr%1szInverseJacobian.dtypeN)rrrrpropertyr)r%rrrrr#s   rc stjjjttrStr2ttr2Stt j rj dkrPt dt t jdjdkr|t dtfddfddfd dfd djjd Stjrjdjdkrt d tfd dfddfddfddjjd StdrztdrztdrzttdtdjtdtdtdjjdStrGfdddt}|SttrttttttttdStddS)zE Convert given object to one suitable for use as a Jacobian. rJzarray must have rank <= 2rr zarray must be squarecs t|Sr)r r2JrrrDFrEzasjacobian..cstj|Sr)r conjTr2rrrrDGrEcs t|Sr)rr2rrrrDHrEcstj|Sr)rrrr2rrrrDIrE)rrrrr%r)zmatrix must be squarecs|Srrr2rrrrDNrEcsj|Srrrr2rrrrDOrEcs |Srrr2rspsolverrrDPrEcsj|Srrr2rrrrDQrEr)r%rrrrrXrP)rrrrrXrPr%r)csLeZdZddZd fdd ZfddZdfdd Zfd d Zd S)zasjacobian..JaccSs ||_dSrr!rrrrrX_szasjacobian..Jac.updatercsB|j}t|tjr t||Stj|r6||StddSNzUnknown matrix type) r" isinstancerndarrayrscipysparse isspmatrixrSr~r3rHmrrrrbs      zasjacobian..Jac.solvecs@|j}t|tjr t||Stj|r4||StddSr) r"rrrr rrrrSr~r3rrrrrks     zasjacobian..Jac.matveccsN|j}t|tjr&t|j|Stj |rB|j|St ddSr) r"rrrrrrrrrrSrrrrrts    zasjacobian..Jac.rsolvecsL|j}t|tjr&t|j|Stj |r@|j|St ddSr) r"rrrr rrrrrrSrrrrr}s    zasjacobian..Jac.rmatvecN)r)r)rrrrXrrrrrrrrJac^s   r)rrrrrrr9z#Cannot convert object to a JacobianN) rrlinalgrrrinspectisclass issubclassrrndimrSZ atleast_2drr)r%rrr*rcallablestrdict BroydenFirst BroydenSecondAnderson DiagBroyden LinearMixingExcitingMixingKrylovJacobian TypeError)rrrrrrO6sf            $  ' rOc@s$eZdZddZddZddZdS)GenericBroydencCs`t||||||_||_t|dr\|jdur\t|}|rVdtt|d||_nd|_dS)Nalpha?r rI)rrPlast_flast_xrrrr )r~r+f0rcZnormf0rrrrPszGenericBroyden.setupcCstdSrrr~r"rrddfrdf_normrrr_updateszGenericBroyden._updatec Cs@||j}||j}|||||t|t|||_||_dSr)rrrr)r~r"rrrdrrrrXs   zGenericBroyden.updateN)rrrrPrrXrrrrrsrc@seZdZdZddZeddZeddZdd Zd d Z dd dZ dddZ ddZ ddZ ddZddZddZd ddZdS)! LowRankMatrixz A matrix represented as .. math:: \alpha I + \sum_{n=0}^{n=M} c_n d_n^\dagger However, if the rank of the matrix reaches the dimension of the vectors, full matrix representation will be used thereon. cCs(||_g|_g|_||_||_d|_dSr)rcsrvrir% collapsed)r~rrir%rrrrs zLowRankMatrix.__init__c Cs\tgd|dd|g\}}}||}t||D]"\}} || |} ||||j| }q4|S)N)axpyscaldotcr )r ziprR) r3rrrvrrrwcdarrr_matvecs  zLowRankMatrix._matveccCs t|dkr||Stddg|dd|g\}}|d}|tjt||jd}t|D]4\}} t|D]"\} } ||| f|| | 7<qlq\tjt||jd} t|D]\} } || || | <q| |} t|| } ||} t|| D]\} }|| | | j | } q| S)Evaluate w = M^-1 vrrrNr r$) lenr ridentityr% enumeratezerosrrrR)r3rrrvrrZc0AirjrqrZqcrrr_solves"   zLowRankMatrix._solvecCs.|jdurt|j|St||j|j|jS)zEvaluate w = M vN)rrr rrrrrvr~r3rrrrs zLowRankMatrix.matveccCs:|jdurt|jj|St|t|j|j|j S)zEvaluate w = M^H vN) rrr rrrrrrvrrrrrrs zLowRankMatrix.rmatvecrcCs,|jdurt|j|St||j|j|jS)rN)rrrrrrrvrrrrrs  zLowRankMatrix.solvecCs8|jdurt|jj|St|t|j|j|j S)zEvaluate w = M^-H vN) rrrrrrrrrvrrrrrrs zLowRankMatrix.rsolvecCsp|jdur<|j|dddf|dddf7_dS|j||j|t|j|jkrl|dSr)rrrappendrvrrRcollapse)r~rrrrrrs .  zLowRankMatrix.appendcCsl|jdur|jS|jtj|j|jd}t|j|jD]0\}}||dddf|dddf 7}q6|S)Nr$) rrrrrir%rrrvr)r~Gmrrrrrr s  *zLowRankMatrix.__array__cCs"t||_d|_d|_d|_dS)z0Collapse the low-rank matrix to a full-rank one.N)rr0rrrvrrrrrrs zLowRankMatrix.collapsecCsD|jdurdS|dksJt|j|kr@|jdd=|jdd=dS)zH Reduce the rank of the matrix by dropping all vectors. Nrrrrrvr~Zrankrrrrestart_reduces    zLowRankMatrix.restart_reducecCs>|jdurdS|dksJt|j|kr:|jd=|jd=qdS)zK Reduce the rank of the matrix by dropping oldest vectors. Nrrrrrr simple_reduce's   zLowRankMatrix.simple_reduceNc Cs8|jdurdS|}|dur |}n|d}|jrBt|t|jd}tdt||d}t|j}||krldSt|jj}t|jj}t |dd\}}t ||j }t |ddd \} } } t |t | }t || j }t|D]8} |dd| f|j| <|dd| f|j| <q|j|d=|j|d=dS) a Reduce the rank of the matrix by retaining some SVD components. This corresponds to the "Broyden Rank Reduction Inverse" algorithm described in [1]_. Note that the SVD decomposition can be done by solving only a problem whose size is the effective rank of this matrix, which is viable even for large problems. Parameters ---------- max_rank : int Maximum rank of this matrix after reduction. to_retain : int, optional Number of SVD components to retain when reduction is done (ie. rank > max_rank). Default is ``max_rank - 2``. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf NrJrr Zeconomic)modeFT)Z full_matricesZ compute_uv)rrrVrr rr0rrvrr rrrrTrQ) r~max_rankZ to_retainrprrCDRUSZWHkrrr svd_reduce2s0    zLowRankMatrix.svd_reduce)r)r)N)rrrr5r staticmethodrrrrrrrrrrrrrrrrrs          ra alpha : float, optional Initial guess for the Jacobian is ``(-1/alpha)``. reduction_method : str or tuple, optional Method used in ensuring that the rank of the Broyden matrix stays low. Can either be a string giving the name of the method, or a tuple of the form ``(method, param1, param2, ...)`` that gives the name of the method and values for additional parameters. Methods available: - ``restart``: drop all matrix columns. Has no extra parameters. - ``simple``: drop oldest matrix column. Has no extra parameters. - ``svd``: keep only the most significant SVD components. Takes an extra parameter, ``to_retain``, which determines the number of SVD components to retain when rank reduction is done. Default is ``max_rank - 2``. max_rank : int, optional Maximum rank for the Broyden matrix. Default is infinity (i.e., no rank reduction). Zbroyden_paramsc@sVeZdZdZdddZddZdd Zdd d Zd dZdddZ ddZ ddZ dS)ra Find a root of a function, using Broyden's first Jacobian approximation. This method is also known as \"Broyden's good method\". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='broyden1'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) dx^\dagger H / ( dx^\dagger H df) which corresponds to Broyden's first Jacobian update .. math:: J_+ = J + (df - J dx) dx^\dagger / dx^\dagger dx References ---------- .. [1] B.A. van der Rotten, PhD thesis, \"A limited memory Broyden method to solve high-dimensional systems of nonlinear equations\". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden1(fun, [0, 0]) >>> sol array([0.84116396, 0.15883641]) Nrestartcst|_d_|dur$tj}|_t|tr:dn|dd|d}|df|dkrvfdd_ n@|dkrfdd_ n&|d krfd d_ n t d |dS) Nrr rrcs jjSr)rrrZ reduce_paramsr~rrrDrEz'BroydenFirst.__init__..simplecs jjSr)rrrrrrrDrErcs jjSr)rrrrrrrDrEz"Unknown rank reduction method '%s') rrrrrr1rrr_reducerS)r~rZreduction_methodrrrrrs(   zBroydenFirst.__init__cCs.t||||t|j |jd|j|_dS)Nr)rrPrrr)r%rrrrrrPszBroydenFirst.setupcCs t|jSr)rrrrrrrszBroydenFirst.todensercCs:|j|}t|s.||j|j|j|j|Sr) rrrr.r/rPrrrc)r~rrHrrrrrs zBroydenFirst.solvecCs |j|Sr)rrr~rrrrrszBroydenFirst.matveccCs |j|Sr)rrr~rrHrrrrszBroydenFirst.rsolvecCs |j|Sr)rrrrrrrszBroydenFirst.rmatvecc CsD||j|}||j|}|t||} |j|| dSr)rrrrr r r~r"rrdrrrr3rrrrrrs  zBroydenFirst._update)NrN)r)r) rrrr5rrPrrrrrrrrrrrs5   rc@seZdZdZddZdS)raL Find a root of a function, using Broyden's second Jacobian approximation. This method is also known as "Broyden's bad method". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='broyden2'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) df^\dagger / ( df^\dagger df) corresponding to Broyden's second method. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden2(fun, [0, 0]) >>> sol array([0.84116365, 0.15883529]) c Cs:||}||j|}||d} |j|| dSNrJ)rrrrrrrrr0s  zBroydenSecond._updateN)rrrr5rrrrrrs2rc@s4eZdZdZdddZddd Zd d Zd d ZdS)ra Find a root of a function, using (extended) Anderson mixing. The Jacobian is formed by for a 'best' solution in the space spanned by last `M` vectors. As a result, only a MxM matrix inversions and MxN multiplications are required. [Ey]_ Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). M : float, optional Number of previous vectors to retain. Defaults to 5. w0 : float, optional Regularization parameter for numerical stability. Compared to unity, good values of the order of 0.01. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='anderson'`` in particular. References ---------- .. [Ey] V. Eyert, J. Comp. Phys., 124, 271 (1996). Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.anderson(fun, [0, 0]) >>> sol array([0.84116588, 0.15883789]) NrncCs2t|||_||_g|_g|_d|_||_dSr)rrrMrdrrgw0)r~rrrrrrrs zAnderson.__init__rc Cs|j |}t|j}|dkr"|Stj||jd}t|D]}t|j||||<q:zt |j |}Wn.t y|jdd=|jdd=|YS0t|D]*}||||j||j|j|7}q|SNrr$) rrrdremptyr%rTr rrrr) r~rrHrdridf_frrgrrrrrs         (zAnderson.solvec Cs,| |j}t|j}|dkr"|Stj||jd}t|D]}t|j||||<q:tj||f|jd}t|D]x}t|D]j}t|j||j||||f<||kr||j dkr||||ft|j||j||j d|j8<q|qpt ||} t|D]*} || | |j| |j| |j7}q|S)Nrr$rJ) rrrdrrr%rTr rrr) r~rrdrirrbrrrgrrrrrs"     :  (zAnderson.matvecc Cs|jdkrdS|j||j|t|j|jkrP|jd|jdq&t|j}tj||f|jd}t |D]R} t | |D]B} | | kr|j d} nd} d| t |j| |j| || | f<qqv|t |dj 7}||_dS)Nrr$rJr )rrdrrrpoprrr%rTrr Ztriurrr) r~r"rrdrrrrirrrwdrrrrs"       *zAnderson._update)Nrnr)r)rrrr5rrrrrrrrr=s F rc@sVeZdZdZdddZddZddd Zd d Zdd d ZddZ ddZ ddZ dS)ra- Find a root of a function, using diagonal Broyden Jacobian approximation. The Jacobian approximation is derived from previous iterations, by retaining only the diagonal of Broyden matrices. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='diagbroyden'`` in particular. Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.diagbroyden(fun, [0, 0]) >>> sol array([0.84116403, 0.15883384]) NcCst|||_dSrrrrr~rrrrrs zDiagBroyden.__init__cCs6t||||tj|jdfd|j|jd|_dS)Nrr r$)rrPrfullr)rr%rrrrrrPszDiagBroyden.setuprcCs | |jSrrrrrrr szDiagBroyden.solvecCs | |jSrrrrrrr szDiagBroyden.matveccCs| |jSrrrrrrrrszDiagBroyden.rsolvecCs| |jSrrrrrrrszDiagBroyden.rmatveccCst|j Sr)rdiagrrrrrrszDiagBroyden.todensecCs(|j||j|||d8_dSrrrrrrrszDiagBroyden._update)N)r)r rrrr5rrPrrrrrrrrrrrs(   rc@sNeZdZdZdddZdddZdd Zdd d Zd d ZddZ ddZ dS)ra Find a root of a function, using a scalar Jacobian approximation. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional The Jacobian approximation is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='linearmixing'`` in particular. NcCst|||_dSrrrrrrr4s zLinearMixing.__init__rcCs | |jSrrrrrrr8szLinearMixing.solvecCs | |jSrrrrrrr;szLinearMixing.matveccCs| t|jSrrrrrrrrr>szLinearMixing.rsolvecCs| t|jSrrrrrrrAszLinearMixing.rmatveccCstt|jdd|jS)Nr)rrrr)rrrrrrDszLinearMixing.todensecCsdSrrrrrrrGszLinearMixing._update)N)r)r) rrrr5rrrrrrrrrrrrs   rc@sVeZdZdZdddZddZdd d Zd d Zdd dZddZ ddZ ddZ dS)ra Find a root of a function, using a tuned diagonal Jacobian approximation. The Jacobian matrix is diagonal and is tuned on each iteration. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='excitingmixing'`` in particular. Parameters ---------- %(params_basic)s alpha : float, optional Initial Jacobian approximation is (-1/alpha). alphamax : float, optional The entries of the diagonal Jacobian are kept in the range ``[alpha, alphamax]``. %(params_extra)s NrIcCs t|||_||_d|_dSr)rrralphamaxbeta)r~rr rrrrfs zExcitingMixing.__init__cCs2t||||tj|jdf|j|jd|_dSr)rrPrrr)rr%r rrrrrPlszExcitingMixing.setuprcCs | |jSrr rrrrrpszExcitingMixing.solvecCs | |jSrr rrrrrsszExcitingMixing.matveccCs| |jSrr rrrrrrvszExcitingMixing.rsolvecCs| |jSrr rrrrryszExcitingMixing.rmatveccCstd|jS)Nr)rrr rrrrr|szExcitingMixing.todensecCsL||jdk}|j||j7<|j|j|<tj|jd|j|jddS)Nr)out)rr rrZclipr )r~r"rrdrrrincrrrrrszExcitingMixing._update)NrI)r)rrrrrrrKs   rc@sDeZdZdZdddZdd Zd d Zdd dZddZddZ dS)ra Find a root of a function, using Krylov approximation for inverse Jacobian. This method is suitable for solving large-scale problems. Parameters ---------- %(params_basic)s rdiff : float, optional Relative step size to use in numerical differentiation. method : {'lgmres', 'gmres', 'bicgstab', 'cgs', 'minres'} or function Krylov method to use to approximate the Jacobian. Can be a string, or a function implementing the same interface as the iterative solvers in `scipy.sparse.linalg`. The default is `scipy.sparse.linalg.lgmres`. inner_maxiter : int, optional Parameter to pass to the "inner" Krylov solver: maximum number of iterations. Iteration will stop after maxiter steps even if the specified tolerance has not been achieved. inner_M : LinearOperator or InverseJacobian Preconditioner for the inner Krylov iteration. Note that you can use also inverse Jacobians as (adaptive) preconditioners. For example, >>> from scipy.optimize.nonlin import BroydenFirst, KrylovJacobian >>> from scipy.optimize.nonlin import InverseJacobian >>> jac = BroydenFirst() >>> kjac = KrylovJacobian(inner_M=InverseJacobian(jac)) If the preconditioner has a method named 'update', it will be called as ``update(x, f)`` after each nonlinear step, with ``x`` giving the current point, and ``f`` the current function value. outer_k : int, optional Size of the subspace kept across LGMRES nonlinear iterations. See `scipy.sparse.linalg.lgmres` for details. inner_kwargs : kwargs Keyword parameters for the "inner" Krylov solver (defined with `method`). Parameter names must start with the `inner_` prefix which will be stripped before passing on the inner method. See, e.g., `scipy.sparse.linalg.gmres` for details. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method=='krylov'`` in particular. scipy.sparse.linalg.gmres scipy.sparse.linalg.lgmres Notes ----- This function implements a Newton-Krylov solver. The basic idea is to compute the inverse of the Jacobian with an iterative Krylov method. These methods require only evaluating the Jacobian-vector products, which are conveniently approximated by a finite difference: .. math:: J v \approx (f(x + \omega*v/|v|) - f(x)) / \omega Due to the use of iterative matrix inverses, these methods can deal with large nonlinear problems. SciPy's `scipy.sparse.linalg` module offers a selection of Krylov solvers to choose from. The default here is `lgmres`, which is a variant of restarted GMRES iteration that reuses some of the information obtained in the previous Newton steps to invert Jacobians in subsequent steps. For a review on Newton-Krylov methods, see for example [1]_, and for the LGMRES sparse inverse method, see [2]_. References ---------- .. [1] D.A. Knoll and D.E. Keyes, J. Comp. Phys. 193, 357 (2004). :doi:`10.1016/j.jcp.2003.08.010` .. [2] A.H. Baker and E.R. Jessup and T. Manteuffel, SIAM J. Matrix Anal. Appl. 26, 962 (2005). :doi:`10.1137/S0895479803422014` Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * x[1] - 1.0, ... 0.5 * (x[1] - x[0]) ** 2] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.newton_krylov(fun, [0, 0]) >>> sol array([0.66731771, 0.66536458]) Nlgmres c KsJ||_||_ttjjjtjjjtjjjtjjj tjjj d |||_ t||jd|_ |j tjjjur||j d<d|j d<|j ddn~|j tjjjur|j ddn^|j tjjjur||j d<d|j d<|j d g|j d d |j d d |j dd|D]4\}}|ds0td|||j |dd<qdS)N)bicgstabgmresrcgsminres)r`rZrestrtr r`Zatolrouter_kZouter_vZprepend_outer_vTZstore_outer_AvFZinner_zUnknown parameter %s)preconditionerrwrrrrrrrrrgetmethod method_kw setdefaultZgcrotmkr startswithrS) r~rwrZ inner_maxiterZinner_Mrrkeyrrrrrs:      zKrylovJacobian.__init__cCs<t|j}t|j}|jtd|td||_dS)Nr )rur+r rrwomega)r~ZmxZmfrrr_update_diff_stepsz KrylovJacobian._update_diff_stepcCslt|}|dkrd|S|j|}||j|||j|}tt|shtt|rhtd|S)Nrz$Function returned non-finite results) rrrcr+rrr/r.rS)r~r3nvZscrrrrrs  zKrylovJacobian.matvecrcCsLd|jvr(|j|j|fi|j\}}n |j|j|fd|i|j\}}|S)NrH)rrop)r~rhsrHZsolrkrrrr$s  zKrylovJacobian.solvecCs<||_||_||jdur8t|jdr8|j||dS)NrX)r+rr rrrX)r~r"rrrrrX+s   zKrylovJacobian.updatecCs|t||||||_||_tjj||_|j durJt |j j d|_ ||jdurxt|jdrx|j|||dS)NrrP)rrPr+rrrrZaslinearoperatorr"rwrr{r%r|r rr)r~r"rrcrrrrP5s   zKrylovJacobian.setup)NrrNr)r) rrrr5rr rrrXrPrrrrrs` *  rcCst|j}|\}}}}}}} tt|t| d|} ddd| D} | rXd| } ddd| D} | rx| d} |rtd|d} | t|| |j| d} i}| t t | |||}|j |_ t ||S) a Construct a solver wrapper with given name and Jacobian approx. It inspects the keyword arguments of ``jac.__init__``, and allows to use the same arguments in the wrapper function, in addition to the keyword arguments of `nonlin_solve` Nz, cSsg|]\}}d||fqS)z%s=%rr.0rr3rrr VrEz#_nonlin_wrapper..cSsg|]\}}d||fqS)z%s=%srr$rrrr&YrEzUnexpected signature %sa def %(name)s(F, xin, iter=None %(kw)s, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw): jac = %(jac)s(%(kwkw)s **kw) return nonlin_solve(F, xin, jac, iter, verbose, maxiter, f_tol, f_rtol, x_tol, x_rtol, tol_norm, line_search, callback) )rrjacZkwkw)_getfullargspecrlistrrjoinrSrrrXglobalsexecr5r8)rr' signatureargsvarargsvarkwdefaults kwonlyargs kwdefaults_kwargsZkw_strZkwkw_strwrappernsrcrrr_nonlin_wrapperJs,     r8rrrrrrr) r9NFNNNNNNr:NFT)r:rmrn);r5rYZnumpyrZ scipy.linalgrrrrrrrr r Zscipy.sparse.linalgrZ scipy.sparser rZscipy._lib._utilr r(Z linesearchrr__all__ Exceptionrr#r'r-r4rstripr6r8rlrWrNrrrOrrrrrrrrrr8rrrrrrrrrrrspm     4  -@D`E q@D.?A,