How to plot the trajectory that fminsearch follows?

Question

I am trying to optimize rosenbrock's function with fminsearch and also drawing the point that gives the minimum value with point size being proportional to the iteration number at each iteration on the 2-D contour plot of rosenbrock's function, however that's not a good idea. As the point size gets bigger it's difficult to see other points. Instead I would like to plot the trajectory fminsearch follows so that I can see the path clearly. How would one do that?

I couldn't find a way to do it as you only get a single point passed to the outputFcn.

Here is my plot:

Here is an example of how I would like it to look like (just to clarify what I want):

options = optimset('outputFcn', @out, 'Display', 'iter');
x = [3.5    2.5     0.5     0.5];
y = [0      -2      -2      0];
[x,fval,eflag,output] = fminsearch(@rosenbrock_wrapper, [x(4), y(4)], options)
title 'Rosenbrock solution via fminsearch'

%Rosenbrock Function
function val = rosenbrock(x, y)
% a = 1.5, b = -1
val = (1 - x + 1.5) .^ 2 + 100 * (y + 1 - (x - 1.5) .^ 2) .^ 2;
end

%Rosenbrock Wrapper
function val = rosenbrock_wrapper(X)
val = rosenbrock(X(:, 1), X(:, 2));
end

%Output Function
function stop = out(x, optimValue, state)
stop = false;
switch state
    case 'init'
        fcontour(@rosenbrock, [0 3 -3 3], 'MeshDensity',50, 'LineWidth', 2, 'LevelList', 1:5:300);
        hold on;
    case 'iter'
        plot(x(1), x(2), '.', 'MarkerSize', optimValue.iteration + 1);
end
end

Answer 1

If you want to plot the progress of the optimization problem real time, you can use the PlotFcn option of fminsearch (see docs for more info).

options = optimset('PlotFcn', @myplotfnc, 'Display', 'iter');

x = [3.5    2.5     0.5     0.5];
y = [0      -2      -2      0];
[x,fval,eflag,output] = fminsearch(@rosenbrock_wrapper, [x(4), y(4)], options)


%Rosenbrock Function
function val = rosenbrock(x, y)
% a = 1.5, b = -1
val = (1 - x + 1.5) .^ 2 + 100 * (y + 1 - (x - 1.5) .^ 2) .^ 2;
end

%Rosenbrock Wrapper
function val = rosenbrock_wrapper(X)
val = rosenbrock(X(:, 1), X(:, 2));
end

% plot function
function stop = myplotfnc(x,optimValues,state)
    persistent pline; % make handle to line peristent between function calls to add new data points
    stop = false;
    switch state
        case 'init'
            fcontour(@rosenbrock, [0 3 -3 3], 'MeshDensity',50, 'LineWidth', 2, 'LevelList', 1:5:300);
            hold on;
            pline = plot(x(1), x(2), '.-', 'linewidth', 2);
            text(x(1), x(2), num2str(optimValues.iteration));
        
        case 'iter'
            % append data to line object
            pline.XData(end+1) = x(1);
            pline.YData(end+1) = x(2);
            text(x(1), x(2), num2str(optimValues.iteration));
    end
end

Alternatively, you can let the output function assign the x,y data to the base workspace after all iterations. Then you can just plot them like you are used to.

%Output Function
function stop = out(x, optimValue, state)
persistent xy_data
stop = false;
switch state
    case 'init'
        xy_data(1,:) = [x(1), x(2)];
    case 'iter'
        xy_data(end+1,:) = [x(1), x(2)];
    case 'done'
        stop = xy_data; % get output when calling after simulation
end
end

And to get the data after the optimization, you just have to do:

xy_data = out([],[],'done');
figure(); 
plot(xy_data(:,1), xy_data(:,2));

Answer 2

Besides @rinkert's brilliant answer I would like to post another way of doing this which was posted as an answer to my question on MathWorks by Matt J.

trajectory=doIt();
fcontour(@rosenbrock, [0 3 -3 3],'LineColor',   '#00FFFF', 'MeshDensity',50,...
                                 'LineWidth', 2, 'LevelList', 1:25:300);
hold on
plot(trajectory(1,:),trajectory(2,:),'ks-','MarkerFaceColor','k')
plot(trajectory(1,end),trajectory(2,end),'ro','MarkerSize',20)
hold off
title 'Rosenbrock solution via fminsearch'

function history=doIt
options = optimset('outputFcn', @out, 'Display', 'none');
x = [3.5    2.5     0.5     0.5];
y = [0      -2      -2      0];
history=[];
[x,fval,eflag,output] = fminsearch(@rosenbrock_wrapper, [x(4), y(4)], options);
 
    %Output Function
    function stop = out(x, optimValue, state)
        stop = false;
        switch state
            case 'iter'
                history=[history,x(:)];
        
        end
    end
end
%Rosenbrock Function
function val = rosenbrock(x, y)
% a = 1.5, b = -1
val = (1 - x + 1.5) .^ 2 + 100 * (y + 1 - (x - 1.5) .^ 2) .^ 2;
end
%Rosenbrock Wrapper
function val = rosenbrock_wrapper(X)
val = rosenbrock(X(:, 1), X(:, 2));
end