gradle force version of transitive dependency not working. No exclude, override or force seems to apply

Question

I have a conflict with a transitive dependency. Overriding, excluding or forcing does not help. What else can I do to get the right version of the library into the jar? The full code dan be. found https://github.com/geoHeil/gradle-dependency-resolution but the main parts of it are described below.

problem description

executing

./gradlew shadowJar

with geomesa dependencies (which pull in an outdated version of typesafe/lightbend configuration library) disabled:

dependencies {

    compile "com.github.kxbmap:configs_2.11:0.4.4"
    //compile "org.locationtech.geomesa:geomesa-hbase-spark-runtime_2.11:2.0.1"
}

and executing the jar

java -jar build/libs/gradleThing-all.jar                         

outputs

hello
my config is: Success(Job1Configuration(frequencyCounting))

When enabling the dependencies:

./gradlew shadowJar
java -jar build/libs/gradleThing-all.jar                         

it fails with

hello
Exception in thread "main" java.lang.Exception: Failed to start. There is a problem with the configuration: Vector([extract] com.typesafe.config.Config.hasPathOrNull(Ljava/lang/String;)Z)

which is related to an outdated version of the config library, how to resolve NoSuchMethodError on typesafe config?. Also confirmed via:

gradle dependencyInsight --dependency om.typesafe:config

> Task :dependencyInsight
com.typesafe:config:1.3.1 (conflict resolution)
   variant "runtime" [
      Requested attributes not found in the selected variant:
         org.gradle.usage = java-api
   ]
\--- com.github.kxbmap:configs_2.11:0.4.4
     \--- compileClasspath

com.typesafe:config:1.2.1 -> 1.3.1
   variant "runtime" [
      Requested attributes not found in the selected variant:
         org.gradle.usage = java-api
   ]
+--- org.locationtech.geomesa:geomesa-convert-avro_2.11:2.0.1
|    \--- org.locationtech.geomesa:geomesa-convert-all_2.11:2.0.1
|         +--- org.locationtech.geomesa:geomesa-tools_2.11:2.0.1
|         |    \--- org.locationtech.geomesa:geomesa-hbase-spark-runtime_2.11:2.0.1
|         |         \--- compileClasspath
...

How can I fix the transitive dependency to my desired version of 1.3.3?

my solutions

Trying to set:

compile("org.locationtech.geomesa:geomesa-hbase-spark-runtime_2.11:2.0.1") {
                exclude group: 'com.typesafe', module: 'config'
            }

and re running the jar fails again with the same issue.

Also an constraint https://docs.gradle.org/current/userguide/managing_transitive_dependencies.html#sec:dependency_constraints like:

implementation("com.typesafe:config")
    constraints {
        implementation("com.typesafe:config:1.3.3") {
            because 'previous versions miss a method https://stackoverflow.com/questions/40610816/how-to-resolve-nosuchmethoderror-on-typesafe-config'
        }
    }

or using force https://docs.gradle.org/current/userguide/managing_transitive_dependencies.html#sec:enforcing_dependency_version like :

implementation('com.typesafe:config:1.3.3') {
        force = true
    }

or like:

configurations.all {
    resolutionStrategy {
        force 'com.typesafe:config:1.3.3'
    }
}

does not give the right version.

result

All variants fail with the same error

What is wrong? There must be a way to force the desired version.

Answer 1

The problem you have is the exact opposite of what you are describing.

The different things you tried in your project all make sure that the version of com.typesafe:config is a 1.3.x.

However, when adding org.locationtech.geomesa:geomesa-hbase-spark-runtime_2.11:2.0.1 you pull in some dependency that already has the classes from com.typesafe:config but from the 1.2 line. And when you create your fat jar, that version overwrites the classes from the 1.3 line.

This can be seen by decompressing your fat jar and running:

$ javap com/typesafe/config/Config.class | grep hasPath
  public abstract boolean hasPath(java.lang.String);

showing that indeed the hasPathOrNull method is missing.

That shading issue is hinted at in https://issues.apache.org/jira/browse/SPARK-9441.

Given this, the easy path - if possible for you - would be to downgrade "com.github.kxbmap:configs_2.11 to a version that relies on com.typesafe:config:1.2.x

The other solution is to find out exactly which fat jar is including these and see if you can exclude it from your own fat jar.