Behavior of the Kernel under high memory load

Question

I have observed the following behavior under Ubuntu 12.04:

On a system with 24GB of RAM and 24 CPUs, if a single process acquires ~12GB of RAM, all other processes belonging to the owner of the high memory process are killed without warning, using what appears to be SIGKILL, and the high memory process is allowed to run until termination. Additionally, attempts to start new processes by the owner will fail.

This is a bit bothersome, but I'm more curious as to why it happens. Presumably this is the result of a resource scheduling decision in the kernel. Is there a place I can find documentation on this?

Answer 1

Sorry to answer my own question, but I like to have a record of solutions. The article linked by sawdust contains what I needed.

• The kernal activates the out-of-memory manager (mm/oom_kill.c in the Linux kernel) when it runs out of allocatable memory.
• The OOM manager uses some heuristics to determine which process should be killed. Total runtime of a process reduces the chance of death, while memory allocated increases it. There are other factors, but they don't matter for me.
• After picking a process, the OOM sends it SIGTERM.

There are two reasons for why, in my case, all processes except the one that's hogging the memory get killed.

1. My process ignores SIGTERM during the region where it is allocating memory. This might be because the process is actively receiving lots of other signals during this time, and/or because the process is blocking for I/O during much of the remaining time. In anycase, it ignores SIGTERM.

2. The process that is hogging all the memory typically has been running for a long time, accumulating RAM for a few hours. Even though it has about 4 times as much as any other process, it's long runtime (several hundred times longer than others) might cause the OOM manager to pick other processes to terminate first.

Solutions:

1. Running:

   ulimit -v memamount

for a particular user changes the maximum amount of memory the user can allocate with a single process to memamount. This can prevent the OOM manager from activating. Instead, malloc calls will fail, which I can detect.

2: Writing a handler for SIGTERM that gracefully cleans up might help, but only if the OOM is actually sending SIGTERM to the process, and the process is ignoring or failing to receive SIGTERM.

3: Set the memory limit from within your code (C):

//resource limit structure with both hard and soft max set to 2GB.
struct rlimit memmax; memmax.rlim_max=0x7FFFFFFF; memmax.rlim_cur = 0x7FFFFFFF;
setrlimit(RLIMIT_MEMLOCK,&memmax); //set maximum virtual memory space to 2GB.