Matlab variable count limit

Question

I have some C++ code that communicates with Matlab via the Engine C API. My code creates temporary variables in the Matlab workspace, which it diligently cleans up via clear calls as soon as possible. However, at some point, my application fails, telling me that it is unable to create the next Matlab temporary variable (usually after ~65530 such operations).

After some experimentation on the Matlab command line, I discovered I could recreate this problem in pure Matlab (that is, independent of my C++ code and its use of the Engine API). Consider the following code:

for i = 1 : 100000
  eval(sprintf('x_%d = %d', i, i));
  whos
  eval(sprintf('clear x_%d', i));
  whos
end

Executing this code on my 32-bit windows laptop with Matlab R2008B (ancient, I know), the loop eventually aborts with the error message:

The current workspace already has too many variables; there is no room for "x_65532".

So, it seems that at least this obsolete version of Matlab has a 64K symbol table limit. Perhaps the limit is bigger on newer (64-bit) versions of Matlab--I'd be interested to hear what results others get.

However, the more interesting question is what effect the clear call is having and how to work around its odd behavior. Here is the output from an iteration a little prior to the abort:

x_65530 =

       65530

  Name         Size            Bytes  Class     Attributes

  i            1x1                 8  double              
  x_65530      1x1                 8  double              

  Name      Size            Bytes  Class     Attributes

  i         1x1                 8  double              

As you can see, the whos output clearly shows that the temporaries from the prior iterations have been removed from the workspace, and the clear seemingly works as expected. Nonetheless, the symbol table has apparently reached capacity.

So, two questions for the SO faithful:

1. How can I work around this somewhat arbitrary limit? That is, what should my clear calls be replaced with?
2. Has this behavior changed with newer and/or 64-bit versions of Matlab?

Answer 1

Repeating my comment in form of an answer:

If you have to stick to the variable naming, you could try to re-use variable names, once they were cleared, avoiding the creation of 65xxx different variable names.

Answer 2

The point is that in matlab you shouldn't need to have that many variables. Use bigger variables rather than many variables.

If each x is a scalar: use a vector

If each x has the same size: use a matrix

If each x is different, use a cell array!

With x{65530} = magic(3) you will not be anywhere near the limit.

Answer 3

The answer to 2) is that this restriction applies to older Matlab versions. While the limit is still present in R2014b, it is not present anymore in R2015b.

Answer 4

I just stumbled on the very same thing. From Python I am creating temporal variables in a Matlab instance, that are later cleared. To make sure I was not overwriting another temporal variable, I made new names form them using the uuid() Python module. This was supposed to be a great idea. However, after a while I got an exception, with the message 'The current workspace already has too many variables'. Executing whos I can see they are just 8 variables, so I must conclude that clear does not actually remove the variable from the workspace. This is really hard to understand, and really inconvenient. It is true I can use names temp001, temp002, temp003, but then when I need a new temporal variable I must make sure I am not currently using that name already. Uuids were invented to avoid this kind of problems, but it seems Matlab awkward behavior is preventing me from using them. Has anyone found a good solution to REALLY delete variables from Matlab's workspace?