Coding

A blog article on testing versus correctness

What I find odd about TDD is this. Most tests are just the same as exercising the contract that you would write if your language supported DBC (design by contract).

i.e. They consist of setting up data conforming to the preconditions, passing the data to the routine, then asserting the post condition as the success or failure of the test.

TDD then just tests a very small subset of the possible combinations.

DBC on the other hand is the equivalent of TDD full time (until disabled). All the time you are running this style of testing on your code, for all the scenarios that you exercise.

There are three approachs then to automating the testing for this style.

For Eiffel there's a research project Eiffel Test Studio that generates test data automatically that tries to generate all valid combinations of data that meet the pre-conditions, to see if you break the post-conditions. It should also be possible to see which post-conditions are used, and maybe even of there are possible missing post conditions.

There's a microsoft reasearch project Pex doing a similar thing. Exercise the function with lots of data automatically, but watch the program as it executes. Try and set the data up so you exercise each path through the code. ie. Explore all the cyclometric complexity of the code.

Lastly, there is the old fashioned code analysis route, much helped by DBC.

ie. Most TDD should be automated. Programmers need to move beyond just writting the tests.

We need to move to the point where we are writting the difficult tests, not the easy tests.

Its also clear that programmers are starting to use programs to automate their development, just as the program they write are designed to automate or aid other people's work.