MemoryError: Unable to allocate 32.2 GiB for an array with shape (28709, 224, 224, 3) and data type float64

888 views Asked by At

I was trying to normalize FER dataset but it gives me memory error. How can I solve this?

This is the scenario

1

There are 1 answers

0
lejlot On

Since you probably don't want your original data either way you can just normalize in place

x = preprocessing.normalize(x, copy=False)

which will avoid copying data.

If it is not enough you just need to think about your memory allocation, as you are clearly operating at the edge of wha tyour hardware can support. You might have to rely on lazy data loading etc.