Can't use javax.tools.JavaCompiler when lombok is a dependency - getting NoClassDefFoundError

Question

I'm trying to use JavaCompiler to compile a Java source at runtime:

$ cat src/main/java/App.java
import java.util.LinkedList;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

public class App {
    public static void main(String[] args) throws Exception {
        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

        StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);

        LinkedList<JavaFileObject> compUnits = new LinkedList<>();
        for (JavaFileObject javaFileObject : fileManager.getJavaFileObjects("Foo.java")) {
            compUnits.add(javaFileObject);
        }
        JavaCompiler.CompilationTask compilationTask = compiler.getTask(null, fileManager, null, null, null, compUnits);
        compilationTask.call();
    }
}

$ cat Foo.java
public class Foo {
}

I'm using Gradle to build and run it:

$ cat build.gradle
apply plugin: 'java'
apply plugin:'application'
mainClassName = "App"

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.projectlombok:lombok:1.16.18'
}

But when I try to run it:

$ gradle -q run
Foo.java:1: warning: Can't initialize javac processor due to (most likely) a class loader problem: java.lang.NoClassDefFoundError: com/sun/tools/javac/processing/JavacProcessingEnvironment
public class Foo {
       ^
        at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:83)
        at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:87)
        at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:140)
        at lombok.launch.AnnotationProcessorHider$AnnotationProcessor.init(AnnotationProcessor.java:69)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$ProcessorState.<init>(JavacProcessingEnvironment.java:500)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.next(JavacProcessingEnvironment.java:597)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:690)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
        at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
        at com.sun.tools.javac.main.Main.compile(Main.java:523)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
        at App.main(App.java:18)
  Caused by: java.lang.ClassNotFoundException: com.sun.tools.javac.processing.JavacProcessingEnvironment
        at java.lang.ClassLoader.findClass(ClassLoader.java:530)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at lombok.launch.ShadowClassLoader.loadClass(ShadowClassLoader.java:422)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 16 more
1 warning

Now, when I remove the Lombok dependency it works fine and compiles Foo.java to Foo.class as expected. So it looks like something has gone wrong with Lombok's annotation preprocessing stage, but when I tried googling for it I only found some unrelated issues from few years ago about how Lombok does not support Java 8 - which is clearly no longer the case.

Is there any way I can get this to work and still keep Lombok as a dependency? (eventually I would also like to add the Lombok jar to the compilation of Foo.java)