Is there a simpler way to test all permutations with Kotest property-based testing?

Question

I'm working with kotlin + Kotest property testing and trying to test all permutations of 2 parameters with list generators like this:

"Some test"{
        forAll(4 ,
                Exhaustive.collection(listOf(
                        "a",
                        "b")),
                Exhaustive.collection(listOf(
                        "1",
                        "2"))
        { begins_with, contains ->
            println("$begins_with:$contains")
            ... some validation code...
        }

I hoped that using the exhaustive generator would generate them in such a way that with 4 iterations all possible permutations would be covered. Like this:

a:1
b:2
a:2
b:1

Instead, exhaustive generators always go in the listed order like this:

a:1
b:2
a:1
b:2

which means I'm testing the same case multiple times.

I've tried switching some generators to Arbs, and that DOES switch the order up, but not optimally. To increase the likelihood of hitting all cases I have to test more than I would if the right order was used.

I've also considered listing the same element multiple times like this

"Some test"{
        forAll(4 ,
                Exhaustive.collection(listOf(
                        "a",
                        "b")),
                Exhaustive.collection(listOf(
                        "1",
                        "1",
                        "2",
                        "2"))
        { begins_with, contains ->
            println("$begins_with:$contains")
            ... some validation code...
        }

but that doesn't seem sustainable, especially when I want to add more parameters or values later.

Is there a way to generate the exhaustive permutations, rather than just continuing to loop over each list?

Answer 1

It seems that you want to merge two exhaustives. Currently, one way of doing this is using Exhaustive.times, which will produce the cross product of two exhaustives:

Exhaustive.collection(listOf("a", "b"))
.times(Exhaustive.collection(listOf("1", "2"))
.checkAll(4) { (begins, contains) ->
    println("$begins:$contains")
}

However, if your usecase is similar to the one you presented with your question, I would suggest Kotest Inspectors instead:

listOf("a", "b").forAll { first ->
    listOf("1", "2").forAll { second ->
        println("$first:$second")
    }
}