How to deploy JNI bindings / C libraries (so/dylib/dll) along with a Java library using maven

Question

We want to offer a Java JNI Wrapper for our core product which is published as a C library for different platform. Our customers should be able to download the Java Wrapper from a maven repository together with the required C binaries (so they don't need to download them separately). While the Java Wrapper is setup to use maven for building, the C binaries come out of a different pipeline. Until now we offer a ZIP package for download that includes the C binaries and the jar file for the Java Wrapper, but since that requires users to manually add the library to their product we want to publish our artifact on maven in the future.

So our project currently looks something like this:

java project dir
┣╸pom.xml
┣╸lib           // these C binaries come from a different pipeline
┃ ┣╸linux-x64
┃ ┃ ┗╸libSdk.so
┃ ┣╸win-x64
┃ ┃ ┗╸Sdk.dll
┃ ...
┣╸src
┃ ┣╸main
┃ ┃ ┗╸java      // actual java wrapper code
┃ ...

How can we deploy all together with maven?

Answer 1

I would like to share my own experience here, because I found very little documentation on this on the web. Please note that the answer might be opinionated based on our use case...

The answer is same as in adding artifacts to standard maven deploy, but since it is a different use case and I was struggling to find a solution quickly I want to answer the question here again...

Ways to package C binaries along with your jar

While there are probably more ways, how the C binaries and the jar can be packaged I want to just list a few different approaches I investigated during my research:

1. zip file including jar & C binaries

   This is probably the simplest approach and was our starting point. However it is cumbersome, since content needs to be extracted first and manually added to the project.

2. jar file containing the C binaries

   You might think: "Why not just put the C libraries into the jar?" Of course this is possible, but there are some issues with that, since loading the C library from with the jar is not possible out of the box and requires extracting the library to the file system first, which again is cumbersome. (see Load library from jar and Extract and load DLL from JAR for more details).

3. having C binaries as separate artifacts

   This led me to idea that the easiest way of using the jar file and the libraries would be if the C binaries are just provided as they are so the user does not need to extract them or anything else. Ideally they would also download using maven, so users don't have to deal with that manually.

   After some research I found that there are basically two options for that:

   1. C binaries are individual artifacts that are resolved via dependencies

      This led me to the question how I should structure the dependencies:

      • Should the C binaries depend on the Java wrapper or vice versa?
      • How would users control which C binaries they need for their platform? (Should they include what they need? Or exclude what they don't need?)

   2. C binaries are side artifacts to the actual jar

      It was new to me that this is possible. Of course I've seen javadoc and sources attached to a jar, but I've never seen such thing like a dll or so or dylib next to a jar. But it turned out that the mechanism for that is the same. The only difference is the classifier and type specified for the artifact.

Conclusion: My preferred way of doing it...

I thought that option 3.2 sounds right for me. But what does that actually mean?

Imagine you have a normal project folder on a maven repo. Usually this will look something like this (I'll omit sha, md5 and asc files for now):

maven_repo_project_directory
┣╸nameoftheartifact-1.2.3.pom
┣╸nameoftheartifact-1.2.3.jar
┣╸nameoftheartifact-1.2.3-javadoc.jar
┗╸nameoftheartifact-1.2.3-sources.jar

But you can make it look like this:

maven_repo_project_directory
┣╸nameoftheartifact-1.2.3.pom
┣╸nameoftheartifact-1.2.3.jar
┣╸nameoftheartifact-1.2.3-javadoc.jar
┣╸nameoftheartifact-1.2.3-sources.jar
┣╸nameoftheartifact-1.2.3-linux-x64.so
┣╸nameoftheartifact-1.2.3-win-x64.dll
...

Meaning all files are in the same folder and can simply be consumed like this in your pom:

For linux-x64

<dependency>
    <groupId>org.yourcompany</groupId>
    <artifactId>nameoftheartifact</artifactId>
    <version>1.2.3</version>
</dependency>
<dependency>
    <groupId>org.yourcompany</groupId>
    <artifactId>nameoftheartifact</artifactId>
    <version>1.2.3</version>
    <classifier>linux-x64</classifier>
    <type>so</type>
</dependency>

or respectively for win-x64

<dependency>
    <groupId>org.yourcompany</groupId>
    <artifactId>nameoftheartifact</artifactId>
    <version>1.2.3</version>
</dependency>
<dependency>
    <groupId>org.yourcompany</groupId>
    <artifactId>nameoftheartifact</artifactId>
    <version>1.2.3</version>
    <classifier>win-x64</classifier>
    <type>dll</type>
</dependency>

But now to the actual question:

How to deploy like this?

Again there are different ways of doing this. But I figured out that one solution works especially nice as it integrates seamlessly with other workflows:

using the build-helper:attach-artifact

The build-helper-maven-plugin can attach any artifacts to your build. This not just ensures that the files end up in the structure shown above, but it also means that they are part of the process for any other steps that follow such as deploy or install...

The following config in your pom will attach the C binaries to the build:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>3.5.0</version>
    <executions>
        <execution>
            <id>attach-artifacts</id>
            <phase>package</phase>
            <goals>
                <goal>attach-artifact</goal>
            </goals>
            <configuration>
                <artifacts>
                    <artifact>
                        <file>lib/linux-x64/libSdk.so</file>
                        <classifier>linux-x64</classifier>
                        <type>so</type>
                    </artifact>
                    <artifact>
                        <file>lib/win-x64/Sdk.dll</file>
                        <classifier>win-x64</classifier>
                        <type>dll</type>
                    </artifact>
                </artifacts>
            </configuration>
        </execution>
    </executions>
</plugin>

Note: you do not need to provide the binaries in the output file name nameoftheartifact-1.2.3-classifier.type style. The build will handle this for you. Your libraries can be just located anywhere.

Note: you can of course also use deploy:deploy-file but I found this more cumbersome as it only deploys these files and does not add it to other build steps such as install or deploy when using the nexus-staging-maven-plugin.