Spark 3.0 & Scalatest incompatibility issue

Question

I am trying to upgrade my application from Spark 2.4 to Spark 3.0. My code compiles fine but the Scala tests don’t work. I get this message:



Exception in thread "ScalaTest-main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)[Ljava/lang/Object;
        at org.scalatest.tools.Runner$.main(Runner.scala:820)
        at org.scalatest.tools.Runner.main(Runner.scala)
    
    

So I tried using various versions of Scalatest but none of them work. For example when I use this:



3.1.0 => Same error.

3.2.0 =>

<dependency>
    <groupId>org.scalatest</groupId>
    <artifactId>scalatest_2.12<artifactId>
    <version>3.2.0</version>
    <scope>test</scope>
</dependency>

 




[ERROR] error: error while loading <root>, Error accessing /Users/achitre/.m2/repository/org/scalatest/scalatest-core_2.12/3.2.0/scalatest-core_2.12-3.2.0.jar
[ERROR] error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.
       




How do I get around this?