One caveat that some developers argue about TDD with Example-based tests is the possible lack of every valid input handling.
Let's take a simple example where those developers could argue:
Make a program that add two numbers.
Bob, a developer starts writing the first test:
Given 1 + 3, then result is 4.
Implementation: def add(x: Int, y: Int) = 4
Nice, it passes.
Now second test:
Given 2 + 2, then result is 4.
Implementation still the same: def add(x: Int, y: Int) = 4
So other developers would come and tell him:
"Now you see Bob that Example-based testing is not enough with your poor 2 examples !
You didn't test every output and looking at your actual implementation, it will fail for other inputs whose result is distinct from 4!
Now, listen to us and start writing some property-based tests to cover all valid inputs.
So let's start with commutativity test, associativity etc.. that are specific to the addition: properties of the addition !"
But but but.... Bob's TDD practice is really bad!
Indeed, he wanted to apply triangulation but he wrote a test that already succeeded, since the implementation hadn't to be altered!
To lead to Triangulation, one should write a test that fails.
And triangulation is one of the master key of the practice of TDD.
It allows the main step: refactoring since you should manage two paths that lead to 2 distinct results!
=> As long as the tests get specific, the code gets more generic thanks to refactoring.
So my question is:
If we practice strict TDD with good practice of triangulation, what is the real benefit of using Property-based testing, asserting invariants that are in 99% of cases already cover by a good TDD?
Indeed, supposing that developers have a good IQ and build implementation that makes sense ;)
My example is taken from those slides.
property based testing is very helpful when edge cases are hard to find or there is so many of them that programmer can easily miss one. i used it for example when i was implementing hirschberg's algorithm. there is no obvious way to divide algorithm into smaller, trivial, easily TDD-testable pieces. and it's hard to hand-craft input that cover all possible algorithm paths. the best way is to generate large number of different input to cover all the paths. and when automatic checking find the bug, add that specific input to your regression tests