Why Linux Free command is not showing less free memory when I run a process which keeps on allocating memory

Question

I have written a C program, in which I continuously allocate memory(of 1 MB size) using malloc. I do not free this memory. While this program is running, I call the linux free command and expect that, the Memory used should increase gradually and Memory free should decrease. But this does not expect. The Free command output remains almost constant. Any idea why the memory allocated using malloc are not showing in the Memory used ?

Answer 1

When you call malloc it in turn requests memory from the kernel (via sbrk or mmap) and the OS just casually gives it the memory, without actually allocating it for the process. This is an optimistic strategy; in effect the OS "hopes" the process will never even use the memory.

When the process eventually writes (or reads) from the memory, it faults and the OS says "ok FINE, if you insist" and actually allocates the memory.

You can see this by gradually writing to the memory:

char *mem = malloc(PAGE_SIZE * 100);
for (i = 0; i < 100; ++i) {
    getchar();
    mem[PAGE_SIZE * i] = 42;
}

One side effect of this is that you can easily allocate more memory with malloc than the system has. By writing to it you will eventually hit a limit and your process wil be killed.

Answer 2

Malloc'ed memory isn't mapped into process memoryspace unless you touch it. This memory will only be ready when the it get a pagefault in the allocated memory, the memory should be mapped in.

For example you can check top, for VIRT column which has complete view of assigned memory by malloc but RES is the real memory usage till that point of time where may not all the malloc memory is mapped.

PID   USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND              
4841  esunboj   20   0 1350m 457m  49m S   25 12.9  43:04.26 firefox