scijit.optimize.fmin_bfgs

scijit.optimize.fmin_bfgs(f, x0, fprime=None, args=(), gtol=1e-05, norm=inf, epsilon=1.4901161193847656e-08, maxiter=None, full_output=0, disp=1, retall=0, callback=None, xrtol=0, c1=0.0001, c2=0.9, hess_inv0=None)

Minimize a function with the quasi-Newton BFGS method.

Reaches the same core as minimize(method='BFGS'). Callable from Python and from inside @njit; both entries run the same compiled core.

Parameters:

fcallable

A plain @njit f(x, *args) -> float.

x0array_like

Initial guess. Any rank is flattened.

fprimecallable or None, optional

A plain @njit fprime(x, *args) -> 1-D float64 array. None (default) selects forward differences with step epsilon. A result whose length is not len(x0) raises ValueError.

argstuple or ndarray, optional

Extra parameters. A tuple is unpacked into separate arguments after x, so args=(a, b) calls f(x, a, b) and fprime(x, a, b). Its elements may be of any type a compiled call accepts, arrays and strings included. Default (), which calls f(x). An ndarray or a list arrives as ONE argument instead, f(x, args). See Notes.

gtolfloat, optional

Stop when vecnorm(g, norm) <= gtol. Default 1e-5.

normfloat, optional

Order of the gradient norm. inf (default) is max|g_i|.

epsilonfloat or ndarray, optional

Absolute step for the forward-difference gradient. An array gives a per-element step and must have length len(x0) or length 1.

maxiterint, float or None, optional

Iteration cap. None (default) means N*200. np.inf is accepted and means unbounded.

full_outputbool, optional

False (default) returns xopt. True returns (xopt, fopt, gopt, Bopt, func_calls, grad_calls, warnflag). Compile-time constant inside @njit.

dispint, optional

1 (default) prints a summary. The status line appears only on success and becomes an OptimizeWarning otherwise; the counters print either way. 0 is silent, and is what a prange sweep wants: see Notes.

retallbool, optional

Append allvecs to the return. Compile-time constant.

callbackcallable or None, optional

Called once per iteration as callback(xk), with a copy of the current iterate. A plain Python function may instead be written callback(intermediate_result) and is handed an object carrying x and fun. Raising StopIteration stops the run and returns the current best. From inside @njit the callback is an @njit callback(xk).

xrtolfloat, optional

Relative tolerance on x. Stop when the step is shorter than xrtol * (xrtol + |xk|_2). Default 0.

c1, c2float, optional

Armijo and curvature parameters. Defaults 1e-4 and 0.9. 0 < c1 < c2 < 1 is required.

hess_inv0ndarray or None, optional

Initial inverse-Hessian estimate, shape (N, N). None (default) is the identity. A non-symmetric or non-positive-definite matrix raises ValueError.

Returns:

xoptndarray

The minimizer.

foptfloat

f(xopt). With full_output.

goptndarray

Gradient at xopt. With full_output.

Boptndarray, shape (N, N)

The inverse-Hessian estimate at xopt. With full_output.

func_callsint

Objective evaluations, including those consumed by a finite-difference gradient. With full_output.

grad_callsint

Gradient evaluations. With full_output.

warnflagint

0 converged, 1 maxiter reached, 2 precision loss, 3 NaN encountered. With full_output.

allvecslist of ndarray

xk after each iteration, nit + 1 entries of shape (N,), the first being x0. With retall. From inside @njit this is a numba.typed.List.

Raises:

ValueError

If callback is neither None nor a callable nor an @njit function; if fprime returns a gradient whose length is not len(x0); or if hess_inv0 is not (N, N).

See also

scipy.optimize.fmin_bfgs

The scipy routine this mirrors.

scijit.optimize.fmin_cg

No stored matrix; O(N) memory.

scijit.optimize.fmin_l_bfgs_b

Bounds, and limited memory at large N.

scijit.optimize.minimize

Dispatches over the Fortran-backed methods.

Notes

Under retall, allvecs is a python list of 1-D arrays, as scipy’s is. From inside @njit it is a numba.typed.List of the same arrays.

args is unpacked into the argument lists of f and fprime, f(x, *args), which is scipy’s contract.

DELIBERATELY DIFFERENT: an ndarray or a list args reaches f and fprime as ONE argument, f(x, args). scipy unpacks those two element by element. The arity of a compiled call is fixed when it compiles, so a sequence whose length is known only at run time cannot be unpacked: f(x, *args) on an ndarray inside @njit is a TypingError. An objective written to scipy’s shape refuses the ndarray rather than reading it wrongly.

full_output and retall must be compile-time literals inside @njit: they select the return type.

callback takes a plain Python function or an @njit function, and both halt on StopIteration. The Python spelling reaches the compiled core through a numba.objmode block over a module-level slot: it takes the GIL once per iteration, and two concurrent solves that both install one overwrite each other, so a prange loop over it serializes and is not reentrant.

Two differences from scipy follow from the compiled path. A plain Python callable cannot be an argument of a compiled function, so a call written inside @njit passes the @njit spelling and a Python one is refused by numba. And numba matches no exception class, except StopIteration being a TypingError on 0.66, so any exception raised inside an @njit callback stops the run the way StopIteration does, where scipy propagates it. A Python callback keeps scipy’s behaviour: only StopIteration halts, and anything else reaches the caller.

disp reproduces scipy’s summary and scipy’s warning, at scipy’s default, so a bare call writes to stdout and warns on a limit path. The warning takes the GIL through a numba.objmode block and the printing through Python’s stdout, so with disp set a prange loop over this routine still gives the right answer but serializes. disp=0 is the parallel path.

A gradient whose length is not len(x0) raises. scipy 1.18 raises on the same input, ValueError: shapes (2,2) and (1,) not aligned, from the inverse-Hessian product. The one case it misses is a wrong-length gradient small enough for the gradient-norm test to halt at iteration 0, where it returns x at the starting point with warnflag 0.

Pure @njit, so it is safe to call from a numba.prange loop with disp=0 and with callback left at None or given as an @njit function.

https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_bfgs.html

Examples

>>> import numpy as np
>>> from numba import njit
>>> from scijit.optimize import fmin_bfgs
>>> @njit
... def rosen(x):
...     s = 0.0
...     for i in range(x.size - 1):
...         s += 100.0 * (x[i + 1] - x[i] ** 2) ** 2 + (1.0 - x[i]) ** 2
...     return s
>>> @njit
... def rosen_g(x):
...     g = np.zeros(x.size)
...     for i in range(x.size - 1):
...         g[i] += -400.0 * x[i] * (x[i + 1] - x[i] ** 2) - 2.0 * (1.0 - x[i])
...         g[i + 1] += 200.0 * (x[i + 1] - x[i] ** 2)
...     return g
>>> @njit
... def run():
...     return fmin_bfgs(rosen, np.array([0.0, 0.0]), rosen_g,
...                      disp=0)
>>> np.round(run(), 6)
array([0.999999, 0.999998])