Uncaught Exception java.lang.OutOfMemoryError: unable to create native thread Error while executing jmeter inside single docker container

Question

While running apache-jmeter test on single docker container I am facing below issue.

[489.013s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. Uncaught Exception java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached. See log file for details.

I am not able to generate Load beyond 2k concurrency. Even though in distributed mode on same docker host I am easily achieving 5k users concurrency. It seems single docker container is not fully utilizing CPU and memory available to it.

Below are the configurations:

• RAM of server - 90GB
• No of CPU - 16
• Ulimit of container (ulimit -n) - 1048576
• Java - open jdk-14
• JVM arguments - -Xms80G -XMX80G

Answer 1

unable to create native thread

it means that you cannot create more than 2000 threads on your operating system level, check what ulimit -u and ulimit -T commands produce and if you see something around 2000 there - this is the setting you need to blame.

Also your Heap allocation seems way to high to me for 2000/5000 virtual users, you don't need to allocate all your physical RAM to JVM, you need to tune your JVM.

If the occupancy of the Java heap is too high, garbage collection occurs frequently. If the occupancy is low, garbage collection is infrequent but lasts longer... Try to keep the memory occupancy of the Java heap between 40% and 70% of the Java heap size...

More information:

• Troubleshoot OutOfMemoryError: Unable to Create New Native Thread
• 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure
• IBM WebSphere Application Server Performance Cookbook - Optimal Heap Size
• Docker - Runtime options with Memory, CPUs, and GPUs