Slow memory allocation in OSX

Question

I'm trying to trace down a memory allocation problem I have in OSX. If I compile and run the following code normally, it will run pretty fast.

#include <sys/mman.h>
#define SIZE 8 * 1024 * 1024

int main(int argc, char const *argv[]) {
  for (int i = 0; i < 50000; ++i) {
    mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0);
  }
  return 0;
}

However, if I compile the same code but linking to some library (i.e.: clang -o test test.c -lpcre) it will randomly either run fast (30ms) or really slow (18 seconds).

Notice that I'm not even using the library, just linking. I also noticed that this seems not to happen with any library.

I'm running OSX 10.10.3. Any ideas?

Answer 1

This is just a bug in the kernel that others have encountered, too. The code in the kernel to find an unused chunk of address space to allocate uses an inefficient search algorithm.

I suspect the reason it seems to depend on whether you link a library is that the dynamic loader (dyld) has to map such a library and that sometimes gets the kernel's VM housekeeping data into the state that triggers the search inefficiency. The reason it doesn't happen on every run probably has to do with Address Space Layout Randomization (ASLR).

I encourage you to file a bug with Apple about this, especially since you have a nice simple test case. (It will probably get closed as a duplicate, since I'm sure you won't be the first person to file it. Still, every new bug report can help isolate it and raises its priority within Apple.)

Answer 2

If you do not require the memory to be initialized, it would be more efficient for you to use malloc() as your usage of mmap() is not only allocating 8 mebibytes of memory for your process but also initializing it.

On OS X, both malloc() and mmap() internally use the mmap() system call (syscall number 197 - http://www.opensource.apple.com/source/xnu/xnu-1456.1.26/bsd/kern/syscalls.master), which asks the kernel to map virtual memory to physical memory.

Answer 3

Sure. Here's an idea. Everybody and his dog allocates memory by calling malloc. Nobody ever in the history of MacOS X has ever used mmap to allocate memory. With that in mind, make a guess which function has been optimised to the max, and for which one the designers of MacOS X don't care one bit about the speed. Use the one that has been optimised.