When you still can’t develop quality software from Requirement Specs, how do you build it right using Test Specs ?

Test-Driven Development (TDD) is the new silver bullet in town. While several ‘obvious’ advantages are being cited in several articles, I fail to understand one simple fact: when we have not yet quite figured out how to develop good-quality software from requirement specs, how do you really use TDD to develop it using Test Specs ?

Bertrand Meyer has a point in case in a recent article, “Seven Principles of Software Testing” in IEEE Computer, August 2008:

…Test-driven development, given prominence by agile methods, has brought tests to the center stage, but sometimes with the seeming implication that tests can be a substitute for specifications. They cannot. Tests, even a million of them, are instances; they miss the abstraction that only a specification can provide.

While there might be several ‘right’ situations where TDD might cut down the effort and time to test, especially regression test, a software, I refuse to believe it is the panacea for all maladies. For one, if a test case passes, there could be two scenarios:

1. the software behaved correctly as per the specs

2. the test case was wrongly designed and did not quite check the functionality as per the specs

Any student of reliability engineering is familiar with Triple Modular Redundancy (TMR): when you use two time clocks, you will always get two different times, and to figure out the correct time, you need an arbiter, or a third clock. In case of software development, we are more known to produce anything faulty than correct, at least in the first try. So, when a test case passes, if is equally possible (if not more) that someone wrote a bad test case than someone writing a correct software ! Why ? Well, I have at least two arguments to support it:

1. when a software engineer can’t write an engineering├é┬ácode 100% correctly, how can you believe his test code will not have statistically similar number of defects ?

2. a software engineer is trained to write good engineering code and not to write good test code. Just like there is a sea of difference between an average developer and a world-class developer, there is also a huge difference between an average developer and world-class tester. Remember, you hired him on his programming skills – if you wanted him to write test code, you probably would have hired someone else !

So, this somehow doesn’t make sense, or it is likely that I have completed misunderstood this subject (more likely, but possible !). Time willing, I plan to write a paper for the upcoming Software Testing conference to explore this subject further. If you have any opinion, experience, anecdotal data, or rocksolid eveidence one way or other, I would love to hear from you.

Meanwhile, stay tuned…

One thought on “When you still can’t develop quality software from Requirement Specs, how do you build it right using Test Specs ?

Leave a Reply