Can jemalloc be modified to allocate from shared memory? The FreeBSD function dallocx()
implies you can provide a pointer to use for allocation, but I don't see an obvious way to tell jemalloc
to restrict all allocations from that memory (nor set a size, etc).
The
dallocx()
function causes the memory referenced byptr
to be made available for future allocations.
If not, what is the level of effort for such a feature? I'm struggling to find an off-the-shelf allocation scheme that can allocate from a shared memory section that I provided.
Similarly, can jemalloc
be configured to allocate from a locked region of memory to prevent swapping?
Feel free to point me to relevant code sections that require modification and provide any ideas or suggestions.
The idea I am exploring is — since you can create arenas/heaps for allocating in a threaded environment, as jemalloc
does to minimize contention, the concept seems scalable to allocating regions of shared memory in a multiprocessing environment, i.e. I create N regions of shared memory using mmap()
, and I want to leverage the power of jemalloc
(or any allocation scheme) to allocate as efficiently as possible, with minimum thread contention, from those one of those shared regions, i.e. if threads/processes are not accessing the same shared regions and arenas, the chance for contention is minimal and speed of the malloc
operation is increased.
This is different than a global pool alloc with malloc()
API since usually these require a global lock effectively serializing the user-space. I'd like to avoid this.
edit 2:
Ideally an api like this:
// init the alloc context to two shmem pools
ctx1 = alloc_init(shm_region1_ptr);
ctx2 = alloc_init(shm_region2_ptr);
(... bunch of code determines pool 2 should be used, based on some method
of pool selection which can minimize possibility of lock contention
with other processes allocating shmem buffers)
// allocate from pool2
ptr = malloc(ctx2, size)
Yes. But this was not true when you asked the question.
Jemalloc 4 (released in August of 2015) has a couple of
mallctl
namespaces that would be useful for this purpose; they allow you to specify per-arena, application-specific chunk allocation hooks. In particular, thearena.<i>.chunk_hooks
namespace and thearenas.extend
mallctl
options are of use. An integration test exists that demonstrates how to consume this API.Regarding the rationale, I would expect that the effective "messaging" overhead required to understand where contention on any particular memory segment lies would be similar to the overhead of just contending, since you're going to degrade into contending on a cache line to accurately update the "contention" value of a particular arena.
Since jemalloc already employs a number of techniques to reduce contention, you could get a similar behavior in a highly threaded environment by creating additional arenas with
opt.narenas
. This would reduce contention as fewer threads would be mapped to an arena, but since threads are effectively round-robined, it's possible you get to hot-spots anyway.To get around this, you could do your contention counting and hotspot detection, and simply use the
thread.arena
mallctl
interface to switch a thread onto an arena with less contention.