Unit Tests Are Still a Waste of Time

Posted on 1-September-2011 by jlangr

Per a recent Dr. Dobbs article, the only reason to write unit tests is as a “convenience … to track down bugs faster.” If that’s the only value proposition, then I must agree with the author that writing unit tests is a “luxury” that only infrequently makes sense.

You’d be far better off investing the time in improving your integration tests or (much better) bolstering the tests that demonstrate customer acceptance criteria. They run much more slowly, but provide more value than writing unit tests after you code: They not only help prevent you from creating defects in the system, they can also be a key negotiation point, and they can act as living documentation on the behaviors your system supports.

Fortunately, writing tests first can create myriad other benefits that outweigh their cost: reduced system size, tests that improve developer understanding of system behaviors, more decoupled/cohesive designs, cleaner code, and the ability to change code safely and rapidly as it needs to be changed. Yes, there’s some churn along the way, but it’s usually mild (and milder as the quality of design improves). These benefits are why I’ve done TDD for a dozen years now, having witnessed some fantastic successes along the way.

Posted in acceptance tests, TAD, TDD, test-driven development, unit testing | Leave a comment

Agile 2011: Live Speaker Feedback

Posted on 18-August-2011 by jlangr

I delivered a talk on “The Only Agile Tools You’ll Ever Need” at Agile 2011, and had a great time doing so. Most of the written feedback from the session indicated that people were very happy with the talk. Negative feedback, received from a handful or so of the ~60 attendees: It didn’t necessarily meet expectations based on what the session summary said. I apologize for that–I’d prepared the summary months before putting together the talk, and what I’d felt important to say changed during that time.

I also muffed a bit–I made a forward reference to the notion that task tracking is a smell, but didn’t quite fully close that thought out as I talked about limiting work in process. If it wasn’t clear, the key point was: if you minimize work in process, the need for tracking tasks in a software tool diminishes significantly.

I handed out index cards at the outset of the session, with the intent of gathering names so I could give out a couple copies of Agile in a Flash. But that’s boring! What else could I have people do with the cards (to bolster my contention that they’re wonderful tools)?

Aha. I gave the following instructions:

“I’m looking to get live feedback during my session. On one side of the index card, draw a big fat smiley face. If you’re happy with what I’m saying, or you think I’m making an astute point, hold up the smiley face. On the other side, let’s see. If you think the believability of what I’m saying is suspect, put something down to represent that. Hmm. Believability Suspect, just abbreviate that, and write down ‘BS‘ in big fat letters. Finally, put your name below the smiley face for the drawing.”

I got a number of smiley faces throughout, but no BS cards. One guy looked like he was about to hold up a BS sign, but changed his mind–probably the one guy who hated the talk. So, my takeaway is that the silly mechanism worked in terms of getting positive feedback–but I suspect that it’s probably a bit too intimidating for most people to challenge a speaker with negative feedback.

Posted in agile, agile in a flash, index cards, speaking | 4 Comments

Buying Back My Soul

Posted on 8-July-2011 by jlangr

I recently traded away my beliefs for security and comfort. I had taken a three-month contract-to-hire at a local software company.

What had I hoped to gain?

  • The company is three miles away from home, a commute of 6 minutes with only one traffic light on the way.
  • They are stable. I received an offer at the end of three months. Due to the industry (government “defense” software), it’s likely I could have stayed there for many years to come.

What was my alternative?

  • Travel. I ultimately chose to return to a life of sporadic travel, consulting and training for Langr Software Solutions. The work is great–I love what I do and love helping people. But it’s travel. For those who’ve never traveled extensively, it’s entertaining the first couple times out. After that, you quickly realize that living in hotels, suffering the hell of airline travel, eating in often-lame restaurants, and being away from home and family is simply not fun at all.

So what was wrong with the local gig?

  • Money. I’m not all about money, and we don’t live extravagantly, but this was the second time I’d taken a pay cut in two years. It’s always tough to cut back on your lifestyle.
  • Industry. I’m a realist, so I understand that “defense” work is necessary, but I’d just as soon not be part of it.
  • Process. I thought I’d be ok with skipping process for a while (they had close to zero process elements). I’d hoped to help introduce some useful techniques and ideas, but it quickly became clear that the culture wasn’t going to support it. They had been reasonably successful without any real process. Not to say that they couldn’t have taken their game to the next level, but one individual can’t change a multi-hundred-person company with zero support.
  • Culture. The folks were nice enough, but I didn’t feel like I was a part of anything. Developers tended to stay in their cubes and keep to their existing cliques. I worked on a two-man development effort (fun stuff; I learned a good amount of Flex/ActionScript). I saw my team member and my boss (both good guys) a few times a week each. The remaining 97%+ of the week, I had close to no additional human interaction.

I sold my soul. Granted, I got to build software, something I love to do, but it was a lonely existence. Had I felt like I was part of a team, as opposed to a bunch of individuals who just happen to reside in nearby cubes, I might have stayed.

I’ve bought back my soul. What’s interesting is that I’m still not part of a team–as a consultant, you are an outsider when you’re traveling, and on your own when at home. But the tradeoff is worth it: I get to help others experience the joy of being part of a true, effective team.

Posted in agile, team | Leave a comment

Interviewer Tips

Posted on 27-June-2011 by jlangr

About every other week, Monster or some portal publishes an article on how to interview properly–what kinds of questions to expect, what kinds of attitudes to carry, and so forth. You’ll also see articles on what sorts of questions to ask as an interviewer. What aren’t published as often are suggestions on how to avoid being a bad interviewer.

While I run a consultancy, I’ve held several full time positions intermittently. I believe in order to stay relevant when training or consulting, it helps dramatically to be part of a real team from time to time. I also find it wise to interview at least once a year–even if you like your current job, it’s always a good idea to find out what’s going on out there.

I’ve had some interesting interviews over my career. I’ve done well on most, and botched a handful too. I’ve also been subjected to “interviewer abuse” many a time. I personally have interviewed close to a couple hundred candidates in my career, and have tried hard to avoid making these candidates miserable.

Here is a short, not exhaustive, list of recommendations I wish my prior interviewers had read before interviewing me:

  • Don’t treat the interview as an opportunity to blather endlessly and show off your skills. Your candidate should be talking at least as much as you.
  • Don’t ask your candidate to “write code” at the whiteboard. Chances are you have at least one computer in your office that the candidate could use.
  • Don’t fixate on overly clever algorithms or puzzles. Insights needed to solve these may not come, particularly if it’s 3:30pm on a Friday and you’ve already subjected the candidate to hours of interviews. Better: Start with a reasonable problem, keep the discussions open-ended, and be willing to shift gears based on what you learn from the candidate.
  • If you think a candidate’s answer to a technical question is wrong, discuss it. There are often other ways to answer a question, and you might have also not done a good job of asking it. Initiate a discussion. You might even learn something yourself.
  • Don’t insist on any more than a half day’s interview. We all know that the important impressions are made or not in the first five minutes, so demanding significantly more time is inconsiderate and suggests vast room for improvement at interviewing.
  • Provide the candidate with a rough agenda, making sure to include what roles each interviewer has in the organization.
  • Ensure your candidate gets exposed to the culture of your company–make sure the interview process accurately reflects it! Make sure they get to see the team at work.
  • Don’t pigeonhole your candidate. Try having an open discussion and see where things lead, instead of forcing him or her through a predetermined interview sequence.
  • Don’t make a candidate slog through the interview process if it’s clear to all parties that he or she is not a good fit. If you think there’s a problem, have the courage to initiate a dialog with the candidate about the supposed mismatch. Ask them where their strengths lie, and look to take the discussions there. Chances are that you’re not doing a good enough job of asking questions.
  • Don’t deliberately make your candidate uncomfortable. Avoid intimidating behavior, such as staring angrily or morosely and refusing to utter more than a syllable or two. Don’t forget to ask them if they need water or a restroom break.
  • If you insist on “behavioral interviewing,” make it clear to the candidate how his or her answers correspond to your decisions.
  • If you are declining to extend an offer to the candidate, be honest with them about the reasons why.
  • Thank your candidate by phone or email within a day of the interview, and indicate when you will have an answer.
Posted in interviewing | 1 Comment

Pitting Teams Against One Another

Posted on 23-May-2011 by jlangr

I was asked this via email:

I just read your article

http://www.developer.com/tech/article.php/3715196

And, I came across this principle:
Don’t use metrics to compare teams. Metrics are best viewed as probes and indicators of problem areas that may warrant further investigation.

Interestingly, this is one of the drivers that my employer wants to collect metrics.

My responses:

Simply put, no two teams are alike.

From another perspective, comparing two points completely misses the goal of doing agile software development. A good agile team is continually looking for ways to improve their productivity–i.e. within the team itself. Most teams have massive potential room for improvement, never mind worrying about what the other teams are up to. They don’t need distraction and concern about keeping their jobs–what they need instead is good leadership and support.

Competition in the marketplace can be a great way to end up with better product, but creating competition between two teams producing different products is artificial and pointless.

There’s also no “fair” way to compare two teams working on different products. Many factors can be a reason why any one team has a significant advantage or disadvantage over other: difficult vs rapid technology (e.g. C++ vs Ruby), team size, geographical considerations, quality of existing code, weak team members, duration together working as a team, bad managers, fire drills, crankier customers, and so on. How do you provide a good measurement of productivity based on all these factors?

You might come up with something that seems reasonably fair, but you’re better off just focusing on getting each team to use their own past productivity as a baseline. The only realistic measurements are relative (and even those are skewed by things such as departing team members).

It can also be counter-productive to focus on racing other teams: What if one team finishes 50% more stories but delivers 20% more defects, resulting in a customer threatening to drop your product? What if you worry about technical metrics (such as code coverage), and one team gets high coverage but their tests stink (seen this–it’s subjective, and easy to tell for a seasoned TDD developer but not anyone else)?

If management focuses on a small number of metrics, and mandates certain goals for them, you can guarantee that the smarter teams will do what they can to play the numbers game. More often than not, a singular metric will have little to do with legitimate goals (customer satisfaction, on-time delivery, high quality). Due to management mandate, I saw a team rapidly increase their code coverage by rapidly cutting and pasting tests, producing such low quality tests that they eventually abandoned. The end result was a complete waste of investment in the unit tests, because the team wasn’t concerned about what really mattered.

Short-term goals may make the numbers look good, but in the long run you will pay for the misguided focus.

Worse, it can be demoralizing. I worked in a large Fortune 500 company where a VP with seemingly good intentions pitted teams against one another by issuing letter grades (A, B, C, D, F). He insisted on having a meeting with each of 30+ teams in his organization and lambasting those who he perceived to be inadequate. Never mind that there were factors for some of the teams that had nothing to do with team productivity, but instead with customer or internal politics.

The only time I would even consider a team-vs-team competition is if both were high-perfoming, kicking-rear teams. That might be fun. But it’s hard enough to find one top-notch team, and almost impossible to find two in the same place.

Posted in agile, metrics | Leave a comment

Abandoning Pairing

Posted on 29-March-2011 by jlangr

For almost a year, I pair programmed on a daily basis as part of a team working for GeoLearning, a small software company that is no more. My enjoyment was tempered by the fact that I was pairing remotely, online with a headset and sometimes a camera, using Skype and WebEx. The contrast to live pairing was marked.

Remote Pairing Cons

  • Lag. Whoever was not hosting the session suffered enough lag to make the experience a bit painful. We didn’t switch enough, probably because of the cost of switching overhead (not only the desktop switching itself, but also getting the source across, which really isn’t all that big a deal with git).
  • Noise. Some people breathe rather heavily, others swallow loudly, and some like to chomp on snacks too much. I did mention it from time to time, but it always felt awkward. And either I was a very good pair, not making these sorts of noises–I doubt it–or others similarly didn’t want to speak up.

Pairing or not, as a remote developer I didn’t feel like I was a real part of the larger, “live” team. I felt that my influence as a remote person was insufficient.

The remote pairing also carried the same issues that “live” pairing can create, most of which are minor. For example, the team had very few standards, which led to some wasteful debates during many of the pairing sessions, still not that big a deal. But couple a lack of standards with stubborn people, and you have a useless pairing session.

I can get along with just about everybody; not the case this time out. I paired with one developer whose concept of TDD was almost completely opposite mine. We produced virtually no useful product during our two or three pairing sessions (we then ended up on different teams, thankfully). In attempts to stop the arguments, I would often say, “Let’s just go with your preferred way of doing things for now.” No such luck. Some folks are simply contrarians.

Remote Pairing Pros

  • It allowed me to stay at home, very welcome after many years of travel, which had created a significant negative impact on my family life.
  • Learning a new system without the ability to lean on someone for several hours a day is tremendously wasteful and frustrating. Remote and isolated, it’s even worse. Pairing rapidly brought me up to speed to the point where I was able to contribute.
  • I could work with the remote pair and not be uncomfortable about things like sitting too close, bad breath, or my laziness on some days when I didn’t feel like showering.
  • The slow cycle times (waiting on compiles, test runs, etc.) were almost bearable due to the fact that you had someone to talk to for the minute or two or more.

Being remote sucks in so many ways–primarily the isolation from the rest of the team–that I wouldn’t have even considered the position had it not been for the ability to pair. I never felt like I was a real part of the larger, “live” team, and I felt that as a remote person I was unable to influence the team much.

So despite the negative aspects of my remote pairing experience, it was the best that I could expect for this opportunity. Ultimately, having this opportunity remotely was better than not having it. And the pairing brought me a small but significant step toward feeling like I was a real part of something.

Pairing No More

I’m currently working a three-month contract for a local firm. It’s a very stable company, there’s no travel on the near horizon, and my commute is about seven minutes door-to-door (it’s three miles away, with only one stoplight inbetween). Those three elements are great, but just about every other element–ranging from pay to preferred way of working to technology–is a concession.

There’s no pairing. My first “real” week (i.e. past most of the HR and setup BS) was spent sitting in an office, digging through a (test-less) codebase, occasionally IMing my teammate a couple offices over to ask questions. It’s a typical no-process environment.

Here’s how I feel about no longer pairing:

  • Bored. Cycle times are again high. I miss having someone to discuss things with during the waits. Checking email, tweets, RSS feeds, and Amazon book ranks helps only a little.
  • Lonely. Stuck in my office with no one to talk to and bounce ideas off, I don’t really feel like I’m part of a team, just one of a number of individuals who happen to work on the same code base and project.
  • Conflicted. I second-guess myself continually. “Is this the company way? Is this the [unfamiliar-to-me technology] way? Does it violate standards? Is it the best design? Did I just do something really stupid? Am I going to get chewed out or look bad for checking this code in?” Unfortunately, second-guessing produces far weaker answers than simply asking someone who already knows and can quickly tell me the answers.
  • Released. One downside of pairing most of the time is it tends to suppress learning by exploration. While I don’t want to work that way all of the time, I’ve found I often learn better when I probe about and try things. With a pair, I usually feel the pressure (even if I’m really the one creating that pressure myself) to move on. And many people who already know the answer tend to simply prefer to tell you it, or more typically grab control and do it themselves rather than wait while you muck around.
    While this may sound damning, it’s easily rectified in a pairing environment: Set a time limit to pairing. I’ve found that a six-hour window is plenty, but it’s up to you and your team to figure out how much works well for you.

Pairing is not for everyone or every team. The bullets above suggest that there are serious challenges both with pairing and with not pairing. But most of the time, I’ve found pairing to be far more enjoyable and productive than not.

Posted in agile, distributed, pair programming, pairing, remote | 4 Comments

Is TDD Faster than TAD?

Posted on 24-February-2011 by jlangr

I do TDD mostly to support the ability to keep  my code and design clean, with high confidence that I’m not breaking something in the process. Most developers who do not practice TDD refactor inadequately, because of the fear based on the very real likelihood of breaking the software. I can’t blame ‘em for the fear. I’ve helped developers in environments where defects related to downtime had the potential to cost the company millions of dollars per minute.

“Inadequate? How dare you!” Yes, inadequate. The best TADers (test-after developers) claim around 70%-75% coverage. The problem is that there’s usually some good, chewy, gristly meat in that 25% to 30% that isn’t covered, and is thus not easily or safely maintained. Costs go up. The system degrades in quality by definition.

In contrast, I have very high confidence that I can move code about when doing TDD. I can also improve the structure and expressiveness of a system, making it far easier to maintain.

Duplication is sooo easy to introduce in a system. It’s harder to spot, and even harder to fix if you don’t have fast tests providing high levels of coverage. I’ve seen one real case where introducing close-to-comprehensive unit tests on a system resulted in shrinkage down to about 1/3 its original size over a reasonably short period of time. And with most systems I’ve encountered, I can scan through the source code and quickly spot rampant bits of unnecessary duplication.

Good code structure and expressiveness is also lacking in most systems. If you’ve been a developer for more than a few weeks, it’s almost a certainty that you’ve spent way too much time slogging through a multi-hundred or multi-thousand line long function, trying to understand how it works and how to fix it without breaking something else. In a well-structured (cohesive) system, the time to pinpoint and fix things can be reduced by almost an order of magnitude.

TADers simply don’t eliminate duplication and clean up the code to this extent. It’s not a goal of TAD.

Which would you rather maintain? The TAD system, or a system that’s half the size and double the expressiveness?

There are many other reasons TDD allows me to go faster than TAD. The converse of my “why TAD sucks” reasons should hint at many of them.

Posted in agile, TAD, TDD | 9 Comments

Responding to a Comment on “Tradeoffs”

Posted on 24-February-2011 by jlangr

As mentioned two posts ago, it’s not possible to see new comments made on my old blog. Here’s a comment I received on Tradeoffs.

The basic thrust of the “Tradeoffs” blog post is that code & fix cycles–after development thinks they’re done, and software gets run through a test team–we typically hit a cycle where QA encounters no end of problems that must be fixed.

“I think you’re comparing a project that uses TDD to a project that writes no unit tests until, ‘Done,’ right?

“Why are you ignoring the projects that write unit tests after each unit is written?

“This is faster than both the approaches you describe. It’s faster than TDD because you write far fewer tests that you don’t need and faster than your other (waterfall?) approach because you find problems sooner.”

/Schmoo

Schmoo asks a good question–writing unit tests shortly after a “unit” is written could conceivably be a way of minimizing the code & fix cycle.

I’ve seen it tried, however. In reality, it helps a little, not a lot and not nearly as much as TDD. The code & fix cycle is still there, a bit shorter than it would have been otherwise.

Now, add good automated acceptance tests into the mix, and I believe Schmoo’s premise is closer to true. But unit tests? No. The primary reason is that most test-after-development (TAD) unit tests are insufficient with respect to coverage, particularly in the more complex areas. Why? Check out my earlier blog post, The Realities of Test-After Development.

As far as TDD generating more tests that you don’t need, depends on what your goals are. I’ve found that TDD, which requires you to write tests for just about everything, results in much faster development than TAD, even as TAD produces less coverage. More on these topics in a future blog post.

Posted in agile, TAD, TDD | 3 Comments

You Don’t Build Stories

Posted on 24-February-2011 by jlangr

I recently responded to a poster at JavaRanch who was asking about how stories are used in agile. I think a lot of people use the word “story” improperly–they use it in the sense that a story is a feature. “I built 10 stories into the product.” No, a story is a discussion around a feature. A story is also not a card.

While this may seem like a nit that I’m picking, I believe misuse of the word “story” has led to unfortunate insistences, such as rigid formats for cards that capture feature conversations (and I’m going to stop calling these cards “story cards,” which isn’t quite as bad a term, but it does help reinforce the wrong things).

(Most of the rest of this post is taken directly from the exchange on JavaRanch.)

“Story cards” are simply placeholders, or tools. The word “story” is the important thing here. A story is something we tell people; the card is simply a reminder, a summary of that oral storytelling. A card is a convenient medium. I suppose I’m thankful I don’t hear about “story whiteboards.”

Story cards (I tried… it’s not easy!) are not artifacts! Once you complete work on building a feature, the card is meaningless. You might think you could use it to track what had been done in a given iteration, but a 5-word summary of a several-day conversation (between customer and developers and other folks) scribbled on a card will not likely be meaningful to anyone six months down the road.

Instead, we get rid of the conversation piece, and replace it with something that documents how the system behaves once that new feature is in place. The best form for that document is a series of well-designed acceptance tests that anyone can easily read and understand. (See our Agile in a Flash card on Acceptance Test Design Principles for a definition of well-designed.)

Acceptance tests are the closest analogs to use cases (a tool popularized almost 20 years ago now), but with slightly differing granularities. The name of an acceptance test can map to the goal, or name, of a use case; it can also map to a much smaller slice of functionality. This is because features in agile are intended to be very small, taking only a day or so to complete. So someone might tell a story: “allow users to filter sort results by age.” That’s likely not an entire use case, it’s an alternate path on a larger use case.

Otherwise, the focus is the same: Jacobson said you could produce user guides and documentation from well-written use cases. The same can hold true for acceptance tests (with a little assembly work), which have the vastly superior quality of eternal accuracy: As long as all your acceptance tests pass, you know that they describe how the system works.

Stories, on the other hand, are just conversations, and the memory of all conversations eventually fades from memory. The cards act as great tools while the story is in progress–they help us track things and remind us what we were talking about, but other than that, we should just recycle the cards once we deliver the corresponding features.

Posted in acceptance tests, agile, agile in a flash, stories | 2 Comments

A Note on the Old Blog

Posted on 24-February-2011 by jlangr

I converted to WordPress some time last year, since Blogger killed my ability to edit there and post on my personal site. In the meantime, I have two annoyances: One, the spammers to this blog are unrelenting. You don’t see any of the spam comments, but I get an email for each one (it could be a real comment) and have to whack about 25 spam messages a week.

Two, while it looks like it’s possible to add comments to my old blog entries, they really just disappear into a bit bucket somewhere. I might dig around and fix the page so it no longer supports typing in the comments, but in the meantime I’ve had two comments that I wanted to respond to, and have felt guilty because I can’t do so easily.

Perhaps my responses will end up as blog entries.

Posted in Uncategorized | 2 Comments