Wednesday, April 26, 2006
The Problem with Unit Testing
One issue with unit tests is the limited number of paths through the software the tests test. Secondly, I'm not convinced that writing the test first, is the best way of writing code.
Taking each of these points in turn. You can address part of the limit number of paths by combining a code coverage tool with your unit tests. If you don't get code coverage, you can add more tests. However, what you really need is something that tests all the time.
Secondly, writing the test first focuses on a particular case, not on producing a good abstraction or good code. If you are not careful, the code is produced to meet a particular test case, not the set of test cases, or even the set of possible data.I've worked for a long time with Eiffel, and it has the interesting feature of design by contract. This combined with unit tests takes things to the next level. A lot of people who comment on DBC, have never used DBC in practice.
First, you change the way you program. It isn't write unit test first, it is contract first. Then write code / unit test. The change of emphasis is produce code that covers all cases. All cases are defined by the contract. Now it becomes clearer what the test cases are. You need to exercise the cases defined by the contract. Whether you write the tests, or the code first doesn't matter. Importantly, if you run the code with assertion checking on, you are testing the contracts as you go. Instead of testing just specific cases, you get universal testing. It is far more powerful.
There are some other alternatives that I can see coming in the future. FxCop and other static code analyzers are getting more sophisticated. Software such as the proposed NStatic are also interesting. Combining these with DBC would make something really powerful. I can also see the day when run time analysis during testing feeds back into the static analysis
Subscribe to Posts [Atom]