scijit.optimize.fmin_cg¶

scijit.optimize.fmin_cg(f, x0, fprime=None, args=(), gtol=1e-05, norm=inf, epsilon=1.4901161193847656e-08, maxiter=None, full_output=0, disp=1, retall=0, callback=None, c1=0.0001, c2=0.4)¶

Minimize a function with the Polak-Ribiere+ conjugate gradient method.

Reaches the same core as minimize(method='CG'). Callable from Python and from inside @njit; both entries run the same compiled core.

Parameters:

fcallable: A plain @njit f(x, *args) -> float. No @cfunc, no .address.
x0array_like: Initial guess. Any rank is flattened.
fprimecallable or None, optional: A plain @njit fprime(x, *args) -> 1-D float64 array. None (default) selects forward differences with step epsilon. A result whose length is not len(x0) raises ValueError.
argstuple or ndarray, optional: Extra parameters. A tuple is unpacked into separate arguments after x, so args=(a, b) calls f(x, a, b) and fprime(x, a, b). Its elements may be of any type a compiled call accepts, arrays and strings included. Default (), which calls f(x). An ndarray or a list arrives as ONE argument instead, f(x, args). See Notes.
gtolfloat, optional: Stop when vecnorm(g, norm) <= gtol. Default 1e-5.
normfloat, optional: Order of the gradient norm. inf (default) is max|g_i|, -inf is min|g_i|, any other value is the p-norm.
epsilonfloat or ndarray, optional: Absolute step for the forward-difference gradient. An array gives a per-element step and must have length len(x0) or length 1.
maxiterint, float or None, optional: Iteration cap. None (default) means N*200. np.inf is accepted and means unbounded.
full_outputbool, optional: False (default) returns xopt. True returns (xopt, fopt, func_calls, grad_calls, warnflag). Compile-time constant inside @njit: it selects the return type.
dispint, optional: 1 (default) prints a summary. The status line appears only on success and becomes an OptimizeWarning otherwise; the counters print either way. 0 is silent, and is what a prange sweep wants: see Notes.
retallbool, optional: Append allvecs to the return. Compile-time constant.
callbackcallable or None, optional: Called once per iteration as callback(xk), with a copy of the current iterate. A plain Python function may instead be written callback(intermediate_result) and is handed an object carrying x and fun. Raising StopIteration stops the run and returns the current best. From inside @njit the callback is an @njit callback(xk).
c1, c2float, optional: Armijo and curvature parameters of the line search. Defaults 1e-4 and 0.4. 0 < c1 < c2 < 1 is required, checked at the first line search.

Returns:

xoptndarray: The minimizer.
foptfloat: f(xopt). With full_output.
func_callsint: Objective evaluations, including those consumed by a finite-difference gradient. With full_output.
grad_callsint: Gradient evaluations. With full_output.
warnflagint: 0 converged, 1 maxiter reached, 2 precision loss (the line search failed), 3 NaN encountered. NOTE that fmin uses a DIFFERENT table, where 1 means maxfun. With full_output.
allvecslist of ndarray: xk after each iteration, nit + 1 entries of shape (N,), the first being x0. With retall. From inside @njit this is a numba.typed.List.

Raises:

ValueError: If callback is neither None nor a callable nor an @njit function, or fprime returns a gradient whose length is not len(x0).

See also

scipy.optimize.fmin_cg: The scipy routine this mirrors.
scijit.optimize.fmin_bfgs: Stores an inverse-Hessian estimate; usually fewer iterations at O(N^2) memory.
scijit.optimize.fmin_l_bfgs_b: Bounds, and limited memory at large N.
scijit.optimize.minimize: Dispatches over the Fortran-backed methods.

Notes

Under retall, allvecs is a python list of 1-D arrays, as scipy’s is. From inside @njit it is a numba.typed.List of the same arrays.

args is unpacked into the argument lists of f and fprime, f(x, *args), which is scipy’s contract.

DELIBERATELY DIFFERENT: an ndarray or a list args reaches f and fprime as ONE argument, f(x, args). scipy unpacks those two element by element. The arity of a compiled call is fixed when it compiles, so a sequence whose length is known only at run time cannot be unpacked: f(x, *args) on an ndarray inside @njit is a TypingError. An objective written to scipy’s shape refuses the ndarray rather than reading it wrongly.

full_output and retall must be compile-time literals inside @njit: they select the return type.

callback takes a plain Python function or an @njit function, and both halt on StopIteration. The Python spelling reaches the compiled core through a numba.objmode block over a module-level slot: it takes the GIL once per iteration, and two concurrent solves that both install one overwrite each other, so a prange loop over it serializes and is not reentrant.

Two differences from scipy follow from the compiled path. A plain Python callable cannot be an argument of a compiled function, so a call written inside @njit passes the @njit spelling and a Python one is refused by numba. And numba matches no exception class, except StopIteration being a TypingError on 0.66, so any exception raised inside an @njit callback stops the run the way StopIteration does, where scipy propagates it. A Python callback keeps scipy’s behaviour: only StopIteration halts, and anything else reaches the caller.

disp reproduces scipy’s summary and scipy’s warning, at scipy’s default, so a bare call writes to stdout and warns on a limit path. The warning takes the GIL through a numba.objmode block and the printing through Python’s stdout, so with disp set a prange loop over this routine still gives the right answer but serializes. disp=0 is the parallel path.

A gradient whose length is not len(x0) raises. scipy 1.18 separates the two cases. A LONGER gradient raises ValueError: operands could not be broadcast together. A SHORTER one is broadcast and the run finishes: warnflag is 0 with x at the starting point when the gradient is small enough that the norm test halts first, and 2 otherwise. The point can also move, measured [1.524, 1.524] from x0 = [0.5, 0.5] with a length-1 gradient of -0.001.

Pure @njit, so it is safe to call from a numba.prange loop with disp=0 and with callback left at None or given as an @njit function.

https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cg.html

Examples

>>> import numpy as np
>>> from numba import njit
>>> from scijit.optimize import fmin_cg
>>> @njit
... def rosen(x):
...     s = 0.0
...     for i in range(x.size - 1):
...         s += 100.0 * (x[i + 1] - x[i] ** 2) ** 2 + (1.0 - x[i]) ** 2
...     return s
>>> @njit
... def rosen_g(x):
...     g = np.zeros(x.size)
...     for i in range(x.size - 1):
...         g[i] += -400.0 * x[i] * (x[i + 1] - x[i] ** 2) - 2.0 * (1.0 - x[i])
...         g[i + 1] += 200.0 * (x[i + 1] - x[i] ** 2)
...     return g
>>> @njit
... def run():
...     return fmin_cg(rosen, np.array([0.0, 0.0]), rosen_g, disp=0)
>>> np.round(run(), 6)
array([0.999992, 0.999984])