Managing multiple arbitrary generators for various tests in PBT

Question

I have many tests, which are using sample input files. Single file contains single example for some test. I would like to make tests to use jqwik for generating test data. From single example of hard-coded file for single test case, I would like to go to PBT approach and make all tests to be properties and check multiple input files generated for me by jqwik framework.

Sidenote: file from test resource is in all test de-serialized to POJO instance (not a single class, but multiple possible types with single parent (abstract class). So I am generating given instances, instead of files.

First approach could be to use arbitrary builders, like stated in documentation here https://jqwik.net/docs/current/user-guide.html#combining-arbitraries-with-builders So sample usage in test can be something like:

// this is actual "test"
@Property
boolean sentencesEndWithAPoint(@ForAll("someYoungPerson") Person youngPerson) {
    return person.calculateAge() < 21;
}

// this is customized arbitrary provider for given test.
// As all tests are in different files, similar, relatively
// complex, single method will be present in all tests
@Provide
Arbitrary<Person> someYoungPerson() {
        Arbitrary<String> insuranceNumberForStackowerflowInsurance = DomainSpecificArbitraries.stackOwerflowInsuranceNumber();
    Arbitrary<String> names =
        Arbitraries.strings().withCharRange('a', 'z').ofMinLength(3).ofMaxLength(21);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 22, null);

    return Builders.withBuilder(() -> new Person(null, -1))
                   .use(names).inSetter(Person::setName)
                   .use(ages).inSetter(Person::setAge)
                   .use(insuranceNumberForStackowerflowInsurance).inSetter(Person::setAge)
                   .build();
}

Another approach can be perhaps using multiple custom domain classes https://jqwik.net/docs/current/user-guide.html#domain-example-american-addresses I.e. for each test case I would prepare single domain context base implementation and use it in given test.

    @Property
    @Domain(OldPersonsDomain.class)
    void oldPersonCannotBeInsured(@ForAll Person person) {
                //...
    }

So I'll end up with too many *Domain classes.

I would like to be able somehow in clean way express something like this:

    @Property
    void oldPersonCannotBeInsured(@ForAll Person person) {
                person.assume()
                    .insured(InsuranceCompaniesEnum.STACKOVERFLOW_ACME_INSURANCE)
                    .midAge()
                    .unemployed()
                    .telephoneNumberPattern(TelephoneNumbers.SVK.ORANGE);
    }

PS: Using https://jqwik.net/docs/current/user-guide.html#assumptions will not be possible, because of huge space of possibilities. Assumption would filter out many percent of generated cases unfortunately.

I have tried to make custom providers with names and used given names in @ForAll annotation. This does not scale, because it does not search in another classes. Also I have taght of another solutions, but all of them seems too "long" and not maintainable.

Answer 1

My recommendation is to program your own configurable Arbitrary class. It's similar to test data builders but the jqwik version. Here's a start - simplifying a bit your example since you didn't provide all the details:

class PersonArbitrary extends ArbitraryDecorator<Person> {

    private Arbitrary<String> nameArbitrary = Arbitraries.strings().alpha().whitespace()
                                                         .ofMinLength(2).ofMaxLength(20);
    private Arbitrary<Integer> ageArbitrary = Arbitraries.integers().between(0, 150);
    private Arbitrary<Person.Insurance> insuranceArbitrary = Arbitraries.of(Person.Insurance.class);
    private Arbitrary<String> telephoneArbitrary = Arbitraries.strings().numeric().ofMinLength(5).ofMaxLength(10);

    @Override
    protected Arbitrary<Person> arbitrary() {
        return Combinators.combine(nameArbitrary, ageArbitrary, insuranceArbitrary, telephoneArbitrary)
                          .as(Person::new);
    }

    PersonArbitrary withInsurance(Person.Insurance insurance) {
        this.insuranceArbitrary = Arbitraries.just(insurance);
        return this;
    }

    PersonArbitrary midAge() {
        this.ageArbitrary = Arbitraries.integers().between(40, 60);
        return this;
    }

    PersonArbitrary withTelephoneNumber(String telephone) {
        this.telephoneArbitrary = Arbitraries.just(telephone);
        return this;
    }
}

class Person {
    private final String telephone;

    enum Insurance {
        PRIVATE, PUBLIC, NONE
    }

    private final String name;
    private final int age;
    private final Insurance address;

    public Person(String name, int age, Insurance address, String telephone) {
        this.name = name;
        this.age = age;
        this.address = address;
        this.telephone = telephone;
    }

    @Override
    public String toString() {
        return "Person{" +
                   "name='" + name + '\'' +
                   ", age=" + age +
                   ", address=" + address +
                   ", telephone='" + telephone + '\'' +
                   '}';
    }
}

Usage would be like that:

@Property(tries = 10)
void oldPersonCannotBeInsured(@ForAll("oldPerson") Person person) {
    System.out.println(person);
}

@Provide
Arbitrary<Person> oldPerson() {
    return new PersonArbitrary()
               .midAge()
               .withInsurance(Person.Insurance.PRIVATE)
               .withTelephoneNumber("+421 12345678");
}

And of course you can combine that with a domain to simplify access without having a lot of specialized provider methods.