Performance w/ calculating Hessian

Question

[edit] The part about "f" is solved. Here is what I did: Instead of using:

X = (F * W' - Y);
f = X' * X;

I'm now using:

X = F*W;
A = X'*F*W;
B = -2*X'*Y;
Y1 = Y'*Y;
f = A + B + Y1

This will give a massive speed up. Still, the problem with the Hessian of f remains. [/edit]

So, I'm having some serious performance "problems" with a quadratic optimization problem I'm trying so solve in Matlab. The problem is not the optimization per se, but the calculation of the target function and the Hessian. Right now it looks like this (F and Y aren't random at all and will have real data, also it is not neccesarily unconstrainted, because then the solution would of course be (F'F)^-1*F'*Y):

W_a = sym('w_a_%d', [1 96]);
W_b = sym('w_b_%d', [1 96]);

for i = 1:96
    W(1,2*(i-1)+1) = W_a(1,i);
    W(1,2*i) = W_b(1,i);
end
F = rand(10000,192);
Y = rand(10000,1);
q = [];
for i = 1:192
    q = [q sum(-Y(:).*F(:,i))];
end
q = 2*q;
q = double(q);

X = (F * W' - Y);
f = X' * X;

H = hessian(f);
H = double(H);

A=[]; b=[];
Aeq=[]; beq=[];
lb=[]; ub=[];
options=optimset('Algorithm', 'active-set', 'Display', 'off');
[xsol,~,exitflag,output]=quadprog(H, q, A, b, Aeq, beq, lb, ub, [], options);

The thing is: calculating f and H takes like forever.

I'm not expecting that there are ways to significantly speed this up, since Matlab is optimized for stuff like this. But maybe someone knows some open license software, that's almost as fast as Matlab, so that I could calculate f and H with that software on a faster machine (which unfortunately has no Matlab license ...) and then let Matlab do the optimization.

Right now I'm kinda lost in this :/

Thank you very much in advance. Even some keywords could help me here like "Look for software xy"

Answer 1

If speed is your concern, using symbolic methods is usually the wrong approach (especially for large systems or if you need to run something repeatedly). You'll need to calculate your Hessian numerically. There's an excellent utility on the MathWorks FileExchange that can do this for you: the DERIVESTsuite. It includes a numeric hessian function. You'll need to formulate your f as a function of X.