Java Mission Control says "few profiling samples", why, and what are my other options?

Question

I'm profiling a Java application using Java Mission Control, and it's saying on the main page of the flight recording that "This recording contains few profiling samples even though CPU load is high. The profiling data is thus likely not relevant."

It seems to be telling the truth. I asked it to sample every 10 ms for 3 minutes which should be 18000 samples, but I only see 996 samples.

It goes on to explain "The profiling data is thus likely not relevant. This might be because the application is running a lot JNI code or that the JVM is spending a lot of time in GC, class loading, JIT compilation etc."

Hmm, I don't have any native methods, and it shouldn't be loading classes or doing any JIT at the stage I recorded (well into the repetitive number crunching part of the code.) It doesn't look like it's spending an inordinate amount of time garbage collecting either.

We used to use hprof to profile this product, with much success. Hprof helped immensely in figuring out where we were relying on the main thread execution, so we could parallelize the hotspots into multiple threads. But that tool got discontinued in Java 9 so we're moving onward to Java Mission Control. It has a lot going for it, but if it can't identify what line numbers the VM threads are sitting on at random sample times, it's not very useful. Is there some other tool to use? Or, is there a way to debug this further from within Java Mission Control? It also looks like JVisualVM is no longer included in Java 9.

Answer 1

If you have many more running threads than cores, the sampling thread could be starved and not able to wake up at the interval you specified.

Answer 2

The answer is probably as simple as you having more threads than cores, and thus most of them not being scheduled on CPU at the time of sampling. The JFR method sampler will only keep samples of threads actually on CPU. The idea is to provide you of a view of where you are actually spending the time executing your Java code.

Now, we know that there are cases where you want to get random samples of all threads, no matter what they are doing. We are adding new profiling capabilities/events in JDK 10.