Gridgain: java.lang.OutOfMemoryError: GC overhead limit exceeded

Question

I'm trying to set up a Gridgain cluster with 2 servers.

1. Load data from a .csv file (1 million to 50 million data) to the Gridgain using GridDataLoader.

2. Find the min, max, average, etc. from the data loaded,

When running as a standalone application in eclipse I'm getting correct output.

But while making a cluster (2 nodes in the 2 servers + 1 node inside my eclipse environment), I'm getting java.lang.OutOfMemoryError: GC overhead limit exceeded error.

The configuration file I'm using is http://pastebin.com/LUa7gxbe

Answer 1

Changing eclipse.ini's Xmx property might solve the problem. Change it to -Xmx3g

Answer 2

java.lang.OutOfMemoryError: GC limit overhead exceeded

This error happens when the system spends too much time executing garbage collection. There can be multiple causes, it is highly related to your environment details. I don't know Gridgain. Because of your complex environment, I think about VM tuning: if your application waits for the whole memory to be full before running garbage collection, here is your main problem.

A hint can be the -XX:-UseParallelGC JVM option (some documentation is available here), but it should be the default conf in Grigain. I don't understand the proper way to configure vm options in your environment (some options seem to be related to the cache). According to the same doc, a slow network could induce a low CPU. I guess a high network could induce a high CPU (perhaps related to GC) ? To ensure you have an appropriate VM configuration, could you check the options applied when running ?

Answer 3

Edit the bin/ggstart.sh script, set the JVM_OPTS to a higher value. Default is 1 GB,

Change it to JVM_OPTS="-Xms2g -Xmx2g -server -XX:+AggressiveOpts -XX:MaxPermSize=256m" or higher