Matlab parfor and input files

Question

I have an algorithm myAlgo() which uses a parameter par1 in order to analyze a set of data (about 1000 .mat files). The path to the .mat files is some cell array I pass also to myAlgo(). The myAlgo() function contains classes and other functions. For every value of par1 I have to test all 1000 .mat files. So it would be a lot faster if I could use a parallel loop since I have an independent (?) problem.

I use the following code with parfor:

par1 = linespace(1,10,100);
myFiles % cell array with the .mat file location
myResult = zeros(length(par1),1);

parfor k=1:length(par1)
    myPar = par1(k);
    myResult(k) = myAlgo(myPar, myFiles);
end

% do something with myResult

.

function theResult = myAlgo(myPar, myFiles)
    for ii=1:length(myFiles)
        tempResult = initAlgo(myPar, myFiles(ii));
    end
    theResult = sum(tempResult);
end

So for every parameter in par1 I do the same thing. Unfortunately the processing time does not decrease. But if I check the workload of the CPU (i5), all cores are quiet active.

Now my question: Is it possible, that parfordoes not work in this case, because every worker initialized by parfor needs to access the folder with the 1000 .mat files. Therefore they can not do their job on the same time. Right? So is there a way handle this?

Answer 1

First of all, check if you've got a license for the parallel computing toolbox (PCT). If you do not have one, parfor will behave just like a normal for loop WITHOUT actually parallel processing (for compatibility reasons)..

Second, make sure to open a parpool first.

Another problem may be that you are using parallel processing for the outer loop with 100 iterations, but not for the larger inner loop with 1000 iterations. You should rephrase your problem as one big loop that allows parfor to parallelize the 100*1000=100000 tasks, not just the 100 outer loops. This excellent post explains the problem nicely and offers several solutions.