scijit.optimize.fmin_l_bfgs_b

scijit.optimize.fmin_l_bfgs_b(func, x0, fprime=None, args=(), approx_grad=0, bounds=None, m=10, factr=10000000.0, pgtol=1e-05, epsilon=1e-08, maxfun=15000, maxiter=15000, callback=None, maxls=20, iprint=-1)

Minimize a function of many variables subject to simple bounds.

L-BFGS-B stores a limited number of past gradient differences instead of a full Hessian approximation, which is what makes it usable at large n.

Callable from Python and from inside @njit. Both entries run the same compiled driver. One argument SPELLING is python-only, named under Notes.

Parameters:

funccallable

A plain @njit function. Which protocol it must follow depends on the two arguments below: with fprime=None and approx_grad=0 it is f(x, *args) -> (value, gradient); with fprime given or approx_grad=1 it is f(x, *args) -> value.

x0array_like

Initial guess. Any rank is flattened, and it is clipped into bounds before the first evaluation.

fprimecallable or None, optional

jac(x, *args) -> gradient. None or a plain @njit function, decided at compile time.

argstuple or ndarray, optional

Extra values for func and fprime. A tuple is unpacked into their argument lists, f(x, *args); the entries may be of any types the callbacks accept. () (default) calls f(x).

approx_gradbool, optional

True computes the gradient by forward differences with absolute step epsilon. Compile-time constant.

boundssequence of (min, max) pairs, (n, 2) ndarray, or None, optional

None (default) is unconstrained. A sequence of pairs and an (n, 2) array both work, from Python and inside @njit. None inside a pair means “no bound in that direction”; -np.inf and np.inf are the equivalent spelling. Pairs may mix the two, and a list holding mixed pairs is written at the @njit call site rather than passed in as an argument; see Notes. The pairs are read by transposition, so the number of rows must be len(x0) and each row must hold exactly two entries.

mint, optional

Number of stored gradient corrections. Default 10.

factrfloat, optional

Convergence tolerance on f, in units of machine epsilon. Default 1e7. The absolute tolerance is factr * np.finfo(float).eps.

pgtolfloat, optional

Convergence tolerance on the projected gradient. Default 1e-5.

epsilonfloat or ndarray, optional

Absolute step for approx_grad. Default 1e-8. An array gives one step per coordinate; a size-1 array applies to every coordinate. Read only when approx_grad is set.

maxfun, maxiterint, optional

Evaluation and iteration budgets. Both default to 15000.

callbackcallable, optional

Called once per iteration, as callback(xk) or callback(intermediate_result). Two spellings are served: a plain Python callable, which halts the solve when it raises StopIteration, and a numba @njit callback(xk), which halts when it raises any exception. See Notes.

maxlsint, optional

Maximum line-search steps per iteration. Default 20. Must be positive.

iprintint, optional

Verbosity passed straight to the Fortran, whose output goes to the process stdout rather than through Python. Default -1, silent.

Returns:

xndarray, shape (n,)

The minimizer.

ffloat

func at x.

dLbfgsbInfo

Namedtuple with fields grad, task, funcalls, nit and warnflag, reached as attributes or by position. warnflag is 0 converged, 1 a limit was reached and 2 anything else; grad and task are meaningful whatever it is.

Raises:

ValueError

If callback is a @cfunc, a raw function address or a non-callable; if bounds has a length other than len(x0), or rows that are not pairs; if any lower bound exceeds its upper bound; if maxls is below 1; if epsilon does not broadcast against x0 while approx_grad is set; or if func or fprime returns a gradient whose length is not len(x0).

TypeError

If a row of bounds is a number rather than a pair.

numba.core.errors.TypingError

Inside @njit, where the refusal is decided by TYPE: a Python callback, a list of pairs that mixes None with a number, and an approx_grad or fprime whose value is not known when the call compiles.

See also

scipy.optimize.fmin_l_bfgs_b

The scipy routine this mirrors.

scijit.optimize.minimize

Dispatches here for method='L-BFGS-B'.

scijit.optimize.fmin_slsqp

Constrained minimization by SLSQP.

scijit.optimize.fmin_bfgs

Unbounded BFGS, a pure @njit port.

Notes

d is a namedtuple carrying scipy’s five field names in scipy’s order. d['warnflag'], d.warnflag and d[4] all reach the flag, and d.keys(), d.values(), d.items(), d.get(), 'task' in d and dict(d) all work. Unpacking yields the five VALUES, where unpacking scipy’s dict yields the five field names.

An ndarray or a list args reaches func and fprime as ONE argument, f(x, args), where scipy unpacks those two element by element. The arity of a compiled call is fixed when it compiles, so a sequence whose length is known only at run time cannot be unpacked.

At len(x0) == 0 the reported funcalls is 0 where scipy reports 1: scipy evaluates func once at x0 before the solver runs and this reports the calls it made, which is none. f comes back as a float where scipy returns the 0-d array it started with.

Whether fprime is None, and the value of approx_grad, are compile-time constants inside @njit: they select which callback protocol gets compiled, so neither can vary at run time.

A Python callback is reached from compiled code through a module-level slot and a numba.objmode block, so it takes the GIL and pays an interpreter round trip once per iteration. It is also not prange-safe, because the slot is module state that two concurrent solves share. Inside @njit only the @njit spelling is accepted, since a Python callable cannot cross into compiled code as an argument.

The two spellings differ in what halts the solve. The Python one halts on StopIteration, which is scipy’s contract; anything else it raises reaches the caller. The @njit one halts on ANY exception, because numba matches no exception class: except StopIteration does not compile.

A MIXED list of pairs such as [(None, 5.0), (0.0, 1.0)] is written at the @njit call site, not passed in as an argument. numba unifies the None and the float into an optional type for a literal, and refuses to unbox a heterogeneous list handed over as an argument: TypeError: can't unbox heterogeneous list. The same pairs as a tuple cross either way, and so do a list whose pairs are uniform, an (n, 2) array, and the -np.inf / np.inf spelling.

A gradient whose length is not len(x0) raises. scipy 1.18 raises at no length and no value. What it returns depends on the gradient it was handed: on a 2-variable problem with a length-1 gradient it reports warnflag 0 at the starting point for 0.0 and -1e-6, and warnflag 2 with the first coordinate moved for -1.0 and -2.0. The coordinates the short gradient does not cover are not written, measured coming back as -1.177e-310.

iprint has no scipy counterpart, and sits last in the signature so that every position before it is scipy’s. scipy 1.18 removed both iprint and disp from this function and from the minimize options dict.

Safe to call from a numba.prange loop with callback at None or an @njit function: the solver is reverse communication, so its state lives in caller-owned arrays. A Python callback is not, and serializes the loop.

https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_l_bfgs_b.html

Examples

>>> import numpy as np
>>> from numba import njit
>>> from scijit.optimize import fmin_l_bfgs_b
>>> @njit
... def fg(x):
...     f = (x[0] - 1.0) ** 2 + (x[1] - 2.5) ** 2
...     return f, np.array([2.0 * (x[0] - 1.0), 2.0 * (x[1] - 2.5)])
>>> @njit
... def run():
...     return fmin_l_bfgs_b(fg, np.array([0.0, 0.0]))
>>> x, f, d = run()
>>> x
array([1. , 2.5])
>>> f
0.0
>>> d.warnflag, d.nit
(0, 2)