Is a postcondition a (type of) unit test?

Question

I'm trying to incorporate some design-by-contract techniques into my coding style. Postconditions look a lot to me like embedded unit tests and I'm wondering if my thinking here is on the right track or way off-base.

Wikipedia defines a postcondition as "a condition or predicate that must always be true just after the execution of some section of code or after an operation in a formal specification. Postconditions are sometimes tested using assertions within the code itself".

Is that not very similar to what you do in a unit test that verifies state directly (doesn't use mocks)?

If that's the case:

1) By using post-conditions, aren't I now sort of embedding testing code in my production code, and isn't that frowned upon?

2) Should using postconditions change the structure of my unit tests? My first thought is that the assertion logic is moved from the tests to the postconditions. That is, tests will use the same inputs and I'm still testing everything I was testing before, but now instead of making assertions in the unit tests I'm making a simple binary assertion about the postconditions passing or not.

3) My second thought is that postcondition code might have control flow and is therefore not ideal for test code, which is supposed to be simple and avoid control flow. But, if I test the postconditions, can I then rely on them in my unit tests?

4) It seems difficult to test postconditions because if I understand them correctly they basically pass or fail and you would have to repeat the logic of the postcondition itself to check that it did the right thing. So, how do you test a postcondition? Do you check them by not utilizing them in your unit testing and ensuring your unit tests and postconditions pass or fail together?

5) My unit tests sometimes verify that a method has caused changes to state in collaborators. In standard practice, do postconditions cover collaborator state or just the state of the class they are defined on?

Answer 1

You are on the right track.

It is true that post-conditions serve a similar purpose to unit tests. The key difference is that the post-condition always runs, while the unit test only runs against a known set of data. This means that the post-condition is less likely to overlook the corner case you didn't think of, but is more expensive at run time.

Here are answers to your specific questions.

1. There is a run-time penalty to post-conditions. However (depending on your environment), it may be possible to drop assertions for speed. (In C you can use an #ifdef, in Java look up AOP, in Python anything in a assert only runs if you pass the --debug flag, etc.) Should you get a performance problem from your assertions, it is solvable. However my preference would be to leave them on until you have a reason not to.

2. Some of your logic will naturally move from the unit test to the post-condition. However it is worthwhile to make sure that you have unit tests that run through all of the cases of interest for your post-condition. This is particularly true if you are dropping assertions in production for speed.

3. Post-conditions are not unit tests. Write them in whatever way that makes sense for what they do. (In general they should be somewhat simple.)

4. In general you test post-conditions as described in #2, by passing in a set of inputs of interest where the post-condition might possibly be violated, and check that it isn't. If you want to test the logic of the post-condition itself, then you can set up code that can violate the post-condition, but which will only run during tests. For instance have a global variable that tests can set which, if it is set, replaces the data to be returned with whatever you want. Now you can cause the post-condition to receive any input you want.

5. I'm not going to give you a hard and fast rule. They are your contracts. They should say what makes sense for what the function is doing. That said, what you are describing can lead to tight coupling between those objects. Tight coupling is something you should only do with good reason.

Answer 2

As with any methodology or coding style - there is no single correct answer. However, one thing I found to be true so far is that there is never a 'one size fits all' solution.

So, if you implement these assertions into a logics of every single postcondition in your design, I'd consider it to be wrong.

My own opinion is that such assertions should be used only if failure to meet postconditions leads the entire system to a dangerously inconsistent state. So, if something like that happens, I'd definitely like the system to do something like: send email/sms to admin, halt production execution, run diagnostics or whatever should be done for that particular system. Note, that this would be an actual feature which purpose is increased security, it's not a unit test code.

On the other hand, if you're coding assertions after every single method call, then as you noticed only thing you are doing is hardcoding test cases into production code. That doesn't serve any real purpose, other than to make your codebase a big mess.

Answer 3

Contracts aren't a form of unit-testing. Rather they're a way of specifying (in an executable format) what conditions should hold before and after a particular function or method is called, and may also specify invariants of objects.

You still need tests when you have contracts since just because you've specified what the functions are supposed to do doesn't mean they'll actually do it. But you'll find that your contracts will help you debug - because by having code that can check that what's happening at run-time is what was expected means that any logic or programming error will cause a failure near to the code that contains the error.

You may find that with contracts you're happy to have fewer smaller tests and more larger-scale tests since the contracts will let you narrow down the source of an error even if the test is broad. Also, there's less need for unit tests to play the role of a specification of how the logic is supposed to work, further limiting the value of the smaller tests.

Contracts are like assertions in that you may choose to or choose not to have them enabled in production code. My opinion is that contracts tend to be more expensive than assertions and so you'll tend to have them disabled in production.