Scipy ValueError: object too deep for desired array with optimize.leastsq

Question

I am trying to fit my 3D data with linear 3D function Z = ax+by+c. I import the data with pandas:

dataframe = pd.read_csv('3d_data.csv',names=['x','y','z'],header=0)

print(dataframe)

            x          y          z
0   52.830740   7.812507   0.000000
1   44.647931  61.031381   8.827942
2   38.725318   0.707952  52.857968
3    0.000000  31.026271  17.743218
4   57.137854  51.291656  61.546131
5   46.341341   3.394429  26.462564
6    3.440893  46.333864  70.440650

I have done some digging and found that the best way to fit 3D data it is to use optimize from scipy with the model equation and residual function:

def model_calc(parameter, x, y):
    a, b, c = parameter
    return a*x + b*y + c

def residual(parameter, data, x, y):
    res = []
    for _x in x:
        for _y in y:
            res.append(data-model_calc(parameter,x,y))
    return res

I fit the data with:

params0 = [0.1, -0.2,1.]
result = scipy.optimize.leastsq(residual,params0,(dataframe['z'],dataframe['x'],dataframe['y']))
fittedParams = result[0]

But the result is a ValueError:

ValueError: object too deep for desired array [...]
minpack.error: Result from function call is not a proper array of floats.

I was trying to minimize the residual function to give only single value or single np.array but it didn't help. I don't know where is the problem and if maybe the search space for parameters it is not too complex. I would be very grateful for some hints!

Answer 1

If you are fitting parameters to a function, you can use curve_fit. Here's an implementation:

from scipy.optimize import curve_fit

def model_calc(X, a, b, c):
    x, y = X
    return a*x + b*y + c

p0 = [0.1, -0.2, 1.]
popt, pcov = curve_fit(model_calc, (dataframe.x, dataframe.y), dataframe.z, p0)  #popt is the fit, pcov is the covariance matrix (see the docs)

Note that your sintax must be if the form f(X, a, b, c), where X can be a 2D vector (See this post).

(Another approach)

If you know your fit is going to be linear, you can use numpy.linalg.lstsq. See here. Example solution:

import numpy as np
from numpy.linalg import lstsq
A = np.vstack((dataframe.x, dataframe.y, np.ones_like(dataframe.y))).T
B = dataframe.z
a, b, c = lstsq(A, B)[0]