Jeff's Blog

Musings about software development, Java, OO, agile, life, whatever.


Tuesday, July 22, 2008 
The Realities of Test-After Development (aka Why POUT Sucks)

"Plain Old Unit Testing," or "POUT," implies that developers are coding unit tests, but they are not doing test-driven development (TDD). In other words, they are following the classic pattern of writing the code, then coming back and coding unit tests to verify the correctness of the code. Instead of POUT, I prefer the acronym TAD, or Test-After Development, which is always a "TAD" too late.

Why does it matter whether tests are written first or last? Theoretically, someone doing TAD could easily produce as high-quality a system with as comprehensive test coverage as produced via TDD. In reality, however, it never happens.

Here are some of the reasons why TAD sucks:
Doesn't promote a testable designMost of the time, testability is in concert with notions of "good," or sustainable, design (coupling and cohesion and all that). Usually, developers discover too late that their design isn't testable. Most often in this case, they quickly give up on trying to test the code, Over time, sure, developers figure out how to produce a more testable design out of the gate, but it's still never a certainty. The real problem is that almost no developer will go back and fix an untestable design in order to be able to write tests.
Is undisciplinedNo rule says that a developer must write tests. Good developers might write enough (usually they do not), but there are plenty of not-so-good developers that will use the lack of a standard as a quick excuse to write as few tests as possible.
Falls victim to ego"My sh*t don't stink, I don't need to bother with writing unit tests, so I'll do the fewest that I can get away with." Enough said. This attitude doesn't belong in a professional development team, but it's sadly common.
Appears unnecessary"I already proved that the code works, I loaded it up in the app server and tried it myself. Why would I bother writing unit tests now?"
Clashes with deadlinesWhen it comes down to deadlines, managers will quickly say, "Never mind with the unit tests, we just have to ship this software."
Decreases average coverageFor many of the reasons already stated, the practical coverage ceiling for TAD is about 70%. This is based on lots of anecdotal evidence, often stated by TAD proponents themselves. I met with Agitar reps about 18 months ago; they confirmed that their TAD-tool (which generates unit tests automatically) has a comparable ceiling. No matter how TAD tests get there, the result is that up to 30% of your app remains unverified at the unit level. Maybe that's not such a big deal if you have higher-level (integration) tests, but...
Doesn't afford adequate refactoringIf up to 30% of your app is not well-covered, that means you cannot safely and effectively refactor almost a third of your application. A third of your system will progressively get worse almost by definition.
Results in difficult testsMany TAD tests are closer to integration tests in nature. "Who cares," say some of the TAD proponents, "about the distinction between unit tests and integration tests?" I don't care about the semantic distinction either, but integration tests are harder to maintain, harder to comprehend, harder to use to pinpoint a problem, and slower.
Isn't very enjoyableFew people enjoy writing unit tests; many people will claim that they love TDD. Honestly, it's not very satisfying to spend time on unit tests when you're pretty sure the code already works.
Questionable return on valueTAD is expensive. If you have to choose between TAD and more end-to-end functional tests, dump the TAD.
Does nothing to advance the craftTAD is a haphazard approach. TAD proponents insist that we're all smart enough to figure out what should be verified at the unit level, and what can be safely ignored. Reality tells us otherwise, but still, that choice may be acceptable in some shops. However, the industry has been screaming for a more disciplined approach to software development that can consistently produce higher code quality. Perhaps TDD isn't it, but TAD most certainly ain't it.


Comments: Post a Comment

Links to this post:

Create a Link



<< Home

RSS Feed (XML)

Archives

February 2004   March 2004   May 2004   September 2004   October 2004   January 2005   February 2005   September 2005   October 2005   November 2005   December 2005   January 2006   February 2006   March 2006   June 2006   August 2006   January 2007   February 2007   March 2007   April 2007   September 2007   October 2007   November 2007   December 2007   January 2008   February 2008   March 2008   April 2008   May 2008   June 2008   July 2008  

This page is powered by Blogger. Isn't yours?