It’s been reported that most developers don’t believe in striving for complete (100%) unit test coverage. I won’t surmise whether or not that’s true. I feel that it’s largely irrelevant to any discussions about unit test coverage.
So, despite the title of this blog, I won’t be talking about what most developers believe. I will offer a few opinions based on many experiences with a wide range of programmer teams and probably a hundred or more systems. I was a long-term programmer on some of those teams, and virtually all were building “real” and sizable software. It doesn’t matter to me that most programmers don’t do what these teams do. What does matter is that they deliver high-quality software and make the business happy.
Is striving for complete unit test coverage dangerous? What if we spend too much time focusing on unit testing, while neglecting other things important to software delivery?
I have yet to hear anyone really say that absolute 100% unit test coverage is realistic. They do say that it’s a goal, not an absolute destination that they approach with blinders on. I’ve worked with many test-driven systems exhibiting coverage in the high 90s, and have heard of many more. And contrary to some claims that this is dangerous, realizing such extensive coverage hasn’t deemphasized other things critical to shipping product.
In agile shops, which is likely where most of TDD is being done, paying attention to customer need is paramount. I’ve never seen a team give too much emphasis to TDD (I have seen it as a problem in a few, very tiny percentage of programmers.) More typically, I’ve seen businesses lose significant money (in QA, on production issues, in development time, on support desks, etc.) because the programmers didn’t write enough tests.
Some of the more experienced programmers out there suggest that we are all smart enough to figure out what doesn’t need to be tested. Unfortunately, that’s not the case. Unlike, say, Google, where the majority of developers are very sharp and knowledgeable, the majority of developers in most organizations know little about design. Most programmers are too quick to make the judgment call that their test coverage is just fine.
In reality, the vast amount of code out there is poorly designed and riddled with bugs. I see this constantly when working with client systems. I’ve worked with far too many cocky developers who thought their code didn’t stink, and made excuses every time they shipped a defect to QA about why it really wasn’t their fault.
Ultimately, our decisions about when not to test are wrong more often than not. We skip tests at the worst possible time: crunch time, when programmers are rushing and carelessly completing their work, or stump time, when we can’t figure out how to test something. Many times, we decide to give up on testing because the code is just not designed well, exhibiting poor dependency management. Even the most senior developers on teams make less than optimal decisions about when to test.
Arguments against increasing test coverage are also off the mark for another reason: they suggest that that TDD is primarily about testing. It’s not, it’s primarily about driving the design and development of your system. (Whether or not it produces a “better” design is a different debate.)
Over time, all systems degrade in quality. Having high levels of code coverage enables the sustainment or improvement of design quality through refactoring. In contrast, systems that have fewer tests are refactored less frequently (in my experience); also, the risk associated with refactoring such systems increases dramatically. Therefore systems with fewer tests more rapidly degrade. I’ve yet to encounter a system that didn’t exhibit this behavior.
Not paying enough attention to unit test coverage approaches malpractice. High coverage levels allow us to sustain the life of very expensive software far longer than the average system.
Ultimately, what most other developers (apparently) believe is a poor beacon. If you’ve found a path that has been leading you toward enlightenment, stay on it. Feel free to believe those who are threatened by TDD, but many of us have found considerable enlightenment in it, borne out with many successes over many years.
