Custom minimizer based on Levenberg-Marquardt in scipy.optimize.basinhopping

Question

I'm having troubles to minimize a complex non linear function in python. This function is actually the chiSquare of a fitting model used to fit experimental data. In order to get the global minimum, I'm using the basinhopping function in scipy. This function is a wrapper of the minimize() function that adds some perturbation to look for different local minima. Right now my problem is that it has troubles to find the local minima.

There are a bunch of solvers that can be used in minimize(), and since I'm using bounds I chose between 'L-BFGS-B', 'SLSQP' and 'TNC'. None of them really find local minima. Is there a method based on the popular Levenberg-Marquardt algorithm that can be use to minimize? Maybe this does not make sense otherwise it would be already implemented, but I can't understand why.

My original idea was actually to use the leastsqbound function(https://pypi.python.org/pypi/leastsqbound) that I know is very good at providing accurate covariance matrix despide the bounds, and include it in a larger algorithm that would look for global minima (like the basinhopping function). Do you know if something like this already exist?

Thanks a lot for you advices!

Answer 1

Scipy has a Levenberg-Marquardt implementation: scipy.optimize.leastsq. It does not have the right return type to use with minimize (and therefore basin_hopping). However, it appears this could be remedied fairly straightforwardly.

Though I have not run it, this should do the trick:

def leastsq_for_minimize( *args, **kwargs ):
    results = leastsq( *args, **kwargs )
    optimize_results = scipy.optimize.OptimizeResult()
    # Some code here to correctly copy results to optimize results
    return optimize_results

scipy.optimize.basinhopping(
    # your arguments here
    minimizer_kwargs=dict(method=leastsq_for_minimize),
    )

• minimize documentation
• basinhopping documentation:
• OptimizeResult documentation