Interactions between code in two unnamed modules

Question

Premise: I have had 2 classes loaded by different class loaders - one from the app classloader and the other, a custom class loader. The class loaded by the custom class loader is trying to reflectively access a private field in the class loaded via the appclass loader. In this case, I get a runtime error along the lines of "Failed to access class A from class B; A is in unnamed module of loader 'app' and B is in unnamed module of loader 'Custom'. The error goes away after adding an '--add-opens java.base/java.lang=ALL-UNNAMED'

Questions : I understand that there can be up to one unnamed module per classloader in the VM.

1. What part of the JLS or JDK implementation specific to hotspot VM talks about interactions between two unnamed modules across loaders ? I can see a lot of unnamed to named module interactions etc in the document, but not between two unnamed.

2. I understand why add-opens is solving the issue in general( the code is being invoked reflectively and end calls into JDK happen via java.lang.reflect APIs?). However, unclear again as to how the add-opens works across loaders - does --add-opens=ALL-UNNAMED expected to open the packages in the specified module to all unnamed modules across loaders in the VM or only to that loader?

Using Java 17 + hotspot VM.

Answer 1

After a lot of trial and error, I finally got a reproducer - but I had to make quite a few assumptions.

Reproducer:

import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.concurrent.Callable;

class A {
    
    private static final String TARGET = "S3CR3T";
    
    public static void main(String[] args) throws Throwable {
        System.out.println("B should get " + TARGET);
        
        Class<?> bClass = injectClass(A.class.getClassLoader(), "B",
                A.class.getResourceAsStream("B.class").readAllBytes());
        
        Method doIt = bClass.getMethod("doIt", Class.class);
        doIt.setAccessible(true);
        doIt.invoke(null, A.class);
    }
    
    private static Class<?> injectClass(ClassLoader loader, String name, byte[] code)
            throws Throwable {
        Method m = ClassLoader.class.getDeclaredMethod("defineClass", String.class, byte[].class,
                int.class, int.class);
        try {
            m.setAccessible(true);
        } catch (RuntimeException e) {
            return new CustomClassLoader(loader, name, code).loadClass(name);
        }
        return (Class<?>) m.invoke(loader, name, code, 0, code.length);
    }
}

class B {
    
    public static void doIt(Class<?> target) throws Throwable {
        accessField("parameter", () -> target);
        accessField("classForName", () -> Class.forName("A"));
        accessField("direct", B::getAClass);
    }

    private static Class<?> getAClass() {
        return A.class;
    }

    private static void accessField(String desc, Callable<Class<?>> targetSupplier) {
        try {
            Class<?> target = targetSupplier.call();
            Field f = target.getDeclaredField("TARGET");
            f.setAccessible(true);
            Object value = f.get(null);
            System.out.println(desc + ": Got " + value);
        } catch (Throwable t) {
            System.err.println(desc + ": Got exception when trying to access private field: ");
            t.printStackTrace(System.err);
        }
    }
}

class CustomClassLoader extends ClassLoader {
    
    private final String name;
    private final byte[] code;
    private final ClassLoader delegate;
    
    CustomClassLoader(ClassLoader delegate, String name, byte[] code) {
        super("Custom", null);
        this.name = name;
        this.code = code;
        this.delegate = delegate;
    }
    
    @Override
    protected Class<?> findClass(String name) throws ClassNotFoundException {
        if (name.equals(this.name)) {
            Class<?> result = findLoadedClass(name);
            if (result == null) {
                result = defineClass(name, code, 0, code.length);
            }
            return result;
        }
        return delegate.loadClass(name);
    }
    
}

Assumptions:

• A is NOT public This is supported by the error message - because now A and B are in different runtime packages, B can't access a non-public class in the other runtime package. The error message matches.
• B references A directly. Reflective access (Class.forName) would still succeed.
• Before creating a new ClassLoader you (or the library you use) first tries to use ClassLoader.defineClass. This would explain why adding --add-opens java.base/java.lang=ALL-UNNAMED would exhibit different behavior. (Not an uncommon thing

---

Running this code yields:

B should get S3CR3T
parameter: Got S3CR3T
classForName: Got S3CR3T
direct: Got exception when trying to access private field: 
java.lang.IllegalAccessError: failed to access class A from class B (A is in unnamed module of loader 'app'; B is in unnamed module of loader 'Custom' @3b07d329)
    at Custom//B.getAClass(A.java:82)
    at Custom//B.accessField(A.java:70)
    at Custom//B.doIt(A.java:65)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at A.main(A.java:17)

while running it with --add-opens java.base/java.lang=ALL-UNNAMED yields

B should get S3CR3T
parameter: Got S3CR3T
classForName: Got S3CR3T
direct: Got S3CR3T

I already hinted in my assumptions about the cause for this:

A and B are in different runtime packages, and A is not public - which means A is not accessible to B.

This has nothing to do with modules.

---

To answer your other questions:

1. The canonical reference for this part is JVMS §5.3.6: _{read the part at the end in bold, then from the beginning}

5.3.6. Modules and Layers

The Java Virtual Machine supports the organization of classes and interfaces into modules. The membership of a class or interface C in a module M is used to control access to C from classes and interfaces in modules other than M (§5.4.4).

Module membership is defined in terms of run-time packages (§5.3). A program determines the names of the packages in each module, and the class loaders that will create the classes and interfaces of the named packages; it then specifies the packages and class loaders to an invocation of the defineModules method of the class ModuleLayer. Invoking defineModules causes the Java Virtual Machine to create new run-time modules that are associated with the run-time packages of the class loaders.

Every run-time module indicates the run-time packages that it exports, which influences access to the public classes and interfaces in those run-time packages. Every run-time module also indicates the other run-time modules that it reads, which influences access by its own code to the public types and interfaces in those run-time modules.

We say that a class is in a run-time module iff the class's run-time package is associated (or will be associated, if the class is actually created) with that run-time module.

A class created by a class loader is in exactly one run-time package and therefore exactly one run-time module, because the Java Virtual Machine does not support a run-time package being associated with (or more evocatively, "split across") multiple run-time modules.

A run-time module is implicitly bound to exactly one class loader, by the semantics of defineModules. On the other hand, a class loader may create classes in more than one run-time module, because the Java Virtual Machine does not require all the run-time packages of a class loader to be associated with the same run-time module.

_{In other words, the relationship between class loaders and run-time modules need not be 1:1. For a given set of modules to be loaded, if a program can determine that the names of the packages in each module are found only in that module, then the program may specify only one class loader to the invocation of defineModules. This class loader will create classes across multiple run-time modules.}

Every run-time module created by defineModules is part of a layer. A layer represents a set of class loaders that jointly serve to create classes in a set of run-time modules. There are two kinds of layers: the boot layer supplied by the Java Virtual Machine, and user-defined layers. The boot layer is created at Java Virtual Machine startup in an implementation-dependent manner. It associates the standard run-time module java.base with standard run-time packages defined by the bootstrap class loader, such as java.lang. User-defined layers are created by programs in order to construct sets of run-time modules that depend on java.base and other standard run-time modules.

A run-time module is implicitly part of exactly one layer, by the semantics of defineModules. However, a class loader may create classes in the run-time modules of different layers, because the same class loader may be specified to multiple invocations of defineModules. Access control is governed by a class's run-time module, not by the class loader which created the class or by the layer(s) which the class loader serves.

The set of class loaders specified for a layer, and the set of run-time modules which are part of a layer, are immutable after the layer is created. However, the ModuleLayer class affords programs a degree of dynamic control over the relationships between the run-time modules in a user-defined layer.

If a user-defined layer contains more than one class loader, then any delegation between the class loaders is the responsibility of the program that created the layer. The Java Virtual Machine does not check that the layer's class loaders delegate to each other in accordance with how the layer's run-time modules read each other. Moreover, if the layer's run-time modules are modified via the ModuleLayer class to read additional run-time modules, then the Java Virtual Machine does not check that the layer's class loaders are modified by some out-of-band mechanism to delegate in a corresponding fashion.

_{There are similarities and differences between class loaders and layers. On the one hand, a layer is similar to a class loader in that each may delegate to, respectively, one or more parent layers or class loaders that created, respectively, modules or classes at an earlier time. That is, the set of modules specified to a layer may depend on modules not specified to the layer, and instead specified previously to one or more parent layers. On the other hand, a layer may be used to create new modules only once, whereas a class loader may be used to create new classes or interfaces at any time via multiple invocations of the defineClass method.}

It is possible for a class loader to define a class or interface in a run-time package that was not associated with a run-time module by any of the layers which the class loader serves. This may occur if the run-time package embodies a named package that was not specified to defineModules, or if the class or interface has a simple binary name (§4.2.1) and thus is a member of a run-time package that embodies an unnamed package (JLS §7.4.2). In either case, the class or interface is treated as a member of a special run-time module which is implicitly bound to the class loader. This special run-time module is known as the unnamed module of the class loader. The run-time package of the class or interface is associated with the unnamed module of the class loader. There are special rules for unnamed modules, designed to maximize their interoperation with other run-time modules, as follows:

• A class loader's unnamed module is distinct from all other run-time modules bound to the same class loader.
• A class loader's unnamed module is distinct from all run-time modules (including unnamed modules) bound to other class loaders.
• Every unnamed module reads every run-time module.
• Every unnamed module exports, to every run-time module, every run-time package associated with itself.

(Emphasis mine)

2. Your other 2 questions:

I understand why add-opens is solving the issue in general( the code is being invoked reflectively and end calls into JDK happen via java.lang.reflect APIs?)

No, java.lang and java.lang.reflect are two different packages. They have nothing to do with each other - except that they are in the same module, part of the platform...

It is more likely that you or a library that you use first tries to hack into java.lang.ClassLoader.defineClass - which resides in the java.lang package - and falls back to creating their own ClassLoader.

However, unclear again as to how the add-opens works across loaders - does --add-opens=ALL-UNNAMED expected to open the packages in the specified module to all unnamed modules across loaders in the VM or only to that loader?

Good question. ALL-UNNAMED implies ALL unnamed modules.
We can test this with this code (and reusing the CustomClassLoader from the reproducer:

import java.lang.reflect.Method;
public class C {
    public static void main(String[] args) throws Throwable {
        Class<?> d = new CustomClassLoader(C.class.getClassLoader(), "D",
                C.class.getResourceAsStream("D.class").readAllBytes()).loadClass("D");
        Method doIt = d.getDeclaredMethod("doIt");
        doIt.setAccessible(true);
        doIt.invoke(null);
    }
}
  
class D {
    public static void doIt() throws Throwable {
        System.out.println("is java.lang open to me: "
                + Object.class.getModule().isOpen("java.lang", D.class.getModule()));
        System.out.print("Try to set ClassLoader.defineClass accessible: ");
        Method m = ClassLoader.class.getDeclaredMethod("defineClass", String.class, byte[].class,
                int.class, int.class);
        m.setAccessible(true);
        System.out.println("Done");
    }
}

When we run this code with --add-opens java.base/java.lang=ALL-UNNAMED this yields:

is java.lang open to me: true
Try to set ClassLoader.defineClass accessible: Done

So the answer is: Yes, ALL-UNNAMED opens the package to all unnamed modules, for every ClassLoader.