Jenkins java.lang.OutOfMemoryError: GC overhead limit exceeded

Question

I am currently working on creating a performance framework using jenkins and execute the performance test from Jenkins. I am using https://github.com/jmeter-maven-plugin/jmeter-maven-plugin this plugin. The sanity test with single user in this performance framework worked well and went ahead with an actual performance test of 200 users and within 2 mins received the error java.lang.OutOfMemoryError: GC overhead limit exceeded I tried the following in jenkins.xml

<arguments>-Xrs -Xmx2048m -XX:MaxPermSize=512m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --prefix=/jenkins --webroot="%BASE%\war"</arguments>

but it didn't work and also noted that whenever I increased the memory the jenkins service stops and had to reduce the memory to 1Gb and then the service restarts.

Had increased the memory for jmeter and java as well but no help. In the .jmx file view results tree and every other listener is disabled but still the issue persists.

Since I am doing a POC jenkins is hosted in my laptop and high level specs as follows System Model : Latitude E7270 Processor : Intel(R) Core(TM) i5-6300U CPU @ 2.40GHZ(4CPU's), ~2.5GHZ Memory : 8192MB RAM

Any help please ?

Answer 1

The error about GC overhead implies that Jenkins is thrashing in Garbage Collection. This means it's probably spending more time doing Garbage Collection than doing useful work.

This situation normally comes about when the heap is too small for the application. With modern multi generational heap layouts it's difficult to say what exactly needs changing.

I would suggest you enable Verbose GC with the following options "-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"

Then follow the advice here: http://www.oracle.com/technetwork/articles/javase/gcportal-136937.html

Answer 2

Few points to note

1. You are using the integrated maven goal to run your jmeter tests. This will use Jenkins as the container to launch your jmeter tests thereby not only impacting your work but also other users of jenkins
2. It is better to defer the execution to a different client machine like a dedicated jmeter machine which uses its own JVM with parameters to launch your tests (OR) use the one that you provide

In summary, 1. Move the test execution out of jenkins 2. Provide the output of the report as an input to your performance plug-in [ this can also crash since it will need more JVM memory when you process endurance test results like an 8 hour result file]

This way, your tests will have better chance of scaling. Also, you haven't mentioned what type of scripting engine that you are using. AS per Jmeter documentation, JSR223 with groovy has a memory leak. Please refer http://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler

Try adding -Dgroovy.use.classvalue=true to see if that helps (provided you are using groovy). If you are using Java 8, there is a high chance that it is creating unique class for all your scripts in jmeter and it is increasing the meta space which is outside your JVM. In that case, restrict the meta space and use class unloading and a 64 bit JVM like

-d64 -XX:+CMSClassUnloadingEnabled.

Also, what is your new generation size. -XX:NewSize=1024m -XX:MaxNewSize=1024m ? Please note jmeter loads all the files permanently and it will go directly to the old generation thereby shrinking any available space for new generation.