Memory Leak in Nd4j INDArray

Question

I am making a algrotim in java with Nd4j. I created an INDarray object and then change the object but when I did this, memory usage is going up. I am asking what is the solution for this memory leak.

example code:

while(true){

        INDArray array = Nd4j.ones(1024,1024);

        try {
            Thread.sleep(10);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

As I know java garbage collecter should delete the previous Indarray memory but in this code it doesn't. But when I add this to code:

array.close()

The memory leak is prevented. But why is this happening? Is there any solution without using close() operation?

Answer 1

I found the answer.

ND4j INDarray takes matrix to out of jvm to make the calculations fast. When I code array = new one, jvm garbage collector can delete the array in jvm part but can not reach the nd4j array memory so it can't delete the memory in there instantly. The programming language nd4j working for calculation in cpu can delete the old memories but it's to slow for codes in java.

For example when I try to create array = new one in 1 ms, garbage collector for nd4j can delete the old ones more than that.

Answer 2

What you are seeing here is because Garbage Collection in the JVM only happens when there is enough memory pressure. The JVM typically will allocate a fraction of the available memory for heap space usage, and only when the used memory is getting close to that limit will GC run.

Because ND4J uses native memory, i.e memory that is not managed by the GC, it doesn't significantly contribute to the memory pressure that triggers a GC run.

In your particular example, the only thing that is allocated on the heap is the pointer to that native memory, i.e. just a single number.

If you ask the JVM to do a GC run (with System.gc()), you will also see that it is freeing the native memory.