java.lang.NoSuchMethodError in Kotlin Script

Question

I wrote a little piece of code using skrape.it in Kotlin (Github Repository Link). I wrote it in a standard gradle project and it works perfectly fine.

fun main() {
    val result = skrape(HttpFetcher) {
        request {
            this.url = "http://example.com/"
        }

        response {
            document.a {
                findFirst {
                    eachLink
                }
            }
        }
    }

    println(result)
}

Output is: {More information...=https://www.iana.org/domains/example}

However when I try to run the code as a Kotlin script like this, it no longer works. The file is called test.main.kts and I run it using kotlin test.main.kts:

@file:DependsOn("it.skrape:skrapeit:1.2.1")

import it.skrape.core.document
import it.skrape.fetcher.HttpFetcher
import it.skrape.fetcher.response
import it.skrape.fetcher.skrape
import it.skrape.selects.html5.a

val result = skrape(HttpFetcher) {
    request {
        this.url = "http://example.com/"
    }

    response {
        document.a {
            findFirst {
                eachLink
            }
        }
    }
}

println(result)

I get the following exception:

java.lang.NoSuchMethodError: 'org.jsoup.select.Elements org.jsoup.select.Elements.select(java.lang.String)'
    at it.skrape.selects.DomTreeElement.applySelector$html_parser(DomTreeElement.kt:90)
    at it.skrape.selects.CssSelector.applySelector$html_parser(CssSelector.kt:22)
    at it.skrape.selects.CssSelectable.findAll(CssSelectable.kt:36)
    at it.skrape.selects.CssSelectable.findByIndex(CssSelectable.kt:39)
    at it.skrape.selects.CssSelectable.findFirst(CssSelectable.kt:58)
    at it.skrape.selects.CssSelectable.findFirst(CssSelectable.kt:101)
    at it.skrape.selects.CssSelectable.findFirst$default(CssSelectable.kt:100)....

How can that be? Shouldn't it just run the same? It's such a minimal example.

kotlin -version gives this output: Kotlin version 1.6.21-release-334 (JRE 11.0.13+8-LTS)

Answer 1

It might be that you have multiple different versions of jsoup library on the classpath and they conflict with each other - when you are it with Gradle it works (probably a coincidence) but when you try it as kotlin script another (very old) version of the library gets preference and it breaks.

It would be a funny coincidence but I found that the Elements#select method was called filter for a while:

• https://github.com/jhy/jsoup/commit/314fa9d2351741b47c34337f57faacc8fd5448da
• https://github.com/jhy/jsoup/commit/48a63ff1edb2b35098ca14ff9e03733dd9fab0a6

I suggest you examine a dependency tree of your project - I don't use Gradle, but it seems that gradlew app:dependencies --scan should do the job: https://wajahatkarim.com/2020/03/gradle-dependency-tree/

If you see jsoup library in the output multiple times then check what versions are brought in and if that can cause the conflict.

Answer 2

The problem is very probably related to the Kotlin Main Kts module embedding (parts of) a few libraries, starting from version 1.6.0. You can see the difference in size for example:

• 1.5.32: 1.6 MB
  https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-main-kts/1.5.32
• 1.6.0: 6.4MB
  https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-main-kts/1.6.0

I don't understand why these external libraries haven't been shaded / relocated.
Actually, some of them have been relocated under org.jetbrains.kotlin:

But unfortunately not JSoup and SLF4J — precisely the 2 libraries that I use in my app...

See a related issue reported on https://youtrack.jetbrains.com/issue/KT-50378.

Edit
I've just opened a bug report at JetBrains: https://youtrack.jetbrains.com/issue/KT-53283.