How to use JMH with gradle?

Question

I want to use JMH, an OpenJDK microbenchmark tool, with gradle. However, Im getting the NPE on compilation. On the other hand, JMH works when using from maven.

I am not posting any build.gradle as it is basic - apply java plugin and add dependency on JHM tool (org.openjdk.jmh:jmh-core:0.2).

I have tried whats written here without success.

What else I have to do? I think something with setting the agent, but I still didnt figure it out.

Exception:

:compileJava
java.lang.NullPointerException
at org.openjdk.jmh.processor.internal.GenerateMicroBenchmarkProcessor.validMethodSignature(GenerateMicroBenchmarkProcessor.java:502)

Answer 1

Just finished my "masterpiece". No uber-jars, no plugins, code base separated from main & test, benchmarks compilation hooked to main, but does not run automatically in the mainstream lifecycle. Simple, explicit, and hackable, vanilla gradle.

I run it directly from IntelliJ, to run on a box you probably will need the uber-jar back :-)

Before doing it I have spent a fair amount of time trying to get that plugin work, but it's way too clunky for my taste.

Step-by-step breakdown below.

Define a new sourceSet called jmh with classpath hooked to that of the main sourceSet

sourceSets {
    jmh {
        java.srcDirs = ['src/jmh/java']
        scala.srcDirs = ['src/jmh/scala']
        resources.srcDirs = ['src/jmh/resources']
        compileClasspath += sourceSets.main.runtimeClasspath
    }
}

Define dependencies for it (at minimum JMH and its annotation processor).

dependencies {
    ...
    jmhImplementation 'org.openjdk.jmh:jmh-core:1.35'
    jmhImplementation 'org.openjdk.jmh:jmh-generator-annprocess:1.35'
}

Define a task jmh of type JavaExec

task jmh(type: JavaExec, dependsOn: jmhClasses) {
    main = 'org.openjdk.jmh.Main'
    classpath = sourceSets.jmh.compileClasspath + sourceSets.jmh.runtimeClasspath
}

Hook jmhClasses task to run after classes to make sure benchmarks are compiled with the rest of the code

classes.finalizedBy(jmhClasses)

Answer 2

My bad, I was trying to benchmark a method that has an argument - of course JMH will not know what to pass :) Once when I created a void method with no arguments, everything worked.

My build.gradle:

defaultTasks 'build'

apply plugin: 'java'
apply plugin: 'shadow'

buildscript {
    repositories {
        mavenCentral()
        maven {
            name 'Shadow'
            url 'http://dl.bintray.com/content/johnrengelman/gradle-plugins'
        }
    }
    dependencies {
        classpath 'org.gradle.plugins:shadow:0.7.4'
    }
}

jar {
    manifest {
        attributes 'Main-Class': 'org.openjdk.jmh.Main'
    }
}

repositories {
    mavenCentral()
}


build.doLast {
    tasks.shadow.execute()
}

shadow {
    outputFile = new File('build/libs/microbenchmarks.jar')
}

ext {
    lib = [
        ... other dependencies...
        jmh:            'org.openjdk.jmh:jmh-core:0.2'
    ]
}

dependencies {
    compile lib... other dependencies...
    compile lib.jmh
}

sourceCompatibility = 1.7

Build tests and jar:

gw clean build

and then run them with:

java -jar build/libs/microbenchmarks.jar ".*" -wi 2 -i 10 -f 2 -t 16

UPDATE

From recent versions of JMH, you would also need to add dependency to:

org.openjdk.jmh:jmh-generator-annprocess:0.5.4

and you can use shadow 0.8.

Answer 3

If you're an IntelliJ user, perhaps the easiest way to make it work, without all that workarounds, is to use the IDE plugin:

https://github.com/artyushov/idea-jmh-plugin

• Add the dependencies
• Create your benchmark
• be happy :)

Answer 4

I made a very small example project to clone and modify as you like. It's a full working example:
https://gitlab.com/barfuin/jmh-gradle-example

It requires no shadow Jars and no plugins, while still running the benchmark in a dedicated JVM. The project also includes some extra Gradle tasks for printing the classpath, the JMH options, etc., stuff that may help to understand what's going on.

Answer 5

Currently you can just use dedicated plugin jmh-gradle-plugin

It requires minimal configuration and allows you to run JMH benchmarks as well as build benchmarks artifact