- Consulting / Coaching
- Jeff’s Blog
Heading into Ten and Twenty Years of Pragmatism (3-Jul-2018)
With the July 2018 issue, PragPub magazine (edited by Michael Swaine) celebrates the beginning of its tenth year. This year also begins the 20th of Andy Hunt’s and Dave Thomas’s classic book The Pragmatic Programmer. In this article, “Heading into Ten and Twenty Years of Pragmatism,” I talk a bit about what it means to be pragmatic, and how both the book and magazine have helped us be pragmatic over the years.
Some articles seem slightly embarrassing to me after I see them published. I thought this might be one, but I think the point should be well-taken: we’re way too invested in dogmatic process trappings. This blog post for Ranorex tells the story of a day in a life where we don’t talk about a process, we just live it, with the singular goal of delivering high-quality software frequently.
I’ve had the idea to write a book on refactoring for some time, and even produced a proposal a couple months ago. The focus would be “refactoring in the context of TDD,” aka continual design. I only got halfway through a sample chapter, which the publisher requests. (Writing the sample also reassures me it’s the book I want to and can write.) This past week I heard that a Big Name is currently working on a refactoring book rewrite. Hmm, maybe this project is not something that I should be doing.
Agile Java has been in print since 2005. It still sells copies–not a lot–and I also still receive thanks for the book from many folks who found it a great way to learn (TDD + OO + Java from the ground up). I feel a bit guilty that people still use a 12+year-old-book featuring a version of Java end-of-lifed for about 8 years! Time for a rewrite, as some have requested?
A dozen years later, what would change?
Otherwise, the tone and general flow would remain the same. Hopefully not the page length: Few folks want to read 750-page books anymore.
I’m the sort to find the stuff I wrote a while back to be lacking, so I might be tempted to rewrite a few things…
There are at least a hundred for-profit or open source tools to help you track your agile / Scrum project. Here is a list, from 2013, of 70 such tools.
If you’ve read Agile in a Flash or some of my blog entries, you’d quickly gather that I’m not a fan of most software-based project tracking tools–you’re better off using whiteboards and/or physical card walls. Introduce a software tool only if you must. And if your need is solely for reporting up to management (hmmm), remember that it’s the SM’s/project manager’s job to capture, consolidate, and distribute such information. Please don’t involve the rest of the team in this.
The tools I seek instead are those that help our team communicate and collaborate better. We’re 15+ years into “agile;” I’d hoped for far more collaboration tools than exist today. (What’s my excuse for not building any of it?)
IDEA, Eclipse, and even VisualStudio (with Resharper added) are powerful. To what extent do these tools directly support TDD? IDEA, Eclipse, and VS all support unit testing out of the box, but that scratches only the surface. While plugins exist for many of these interests, I’d like to see direct support for things like:
With TDD, tests and production code are inexorably linked. It would be nice to work in an IDE that supported TDD as the primary workflow for building software.
Pairing will never supplant loner development. But it’s still a preferred way of working for many developers, including those distributed geographically.
If we’re pairing and you’re sharing your machine and IDE, I’m stuck with your horrifying choices for keymaps (I need my vim!), colors, font, and font size. One or both of us must make concessions to the most-comfortable setup we would choose. How sad.
The problem of disconcerting configs would be reasonably easily solved if the major IDEs supported a universal config-switching scheme. Sets of profiles (one profile per IDE, I suppose) would be stored somewhere accessible via http. The first time I sit on your machine, I’d enter my profile-set location. The IDE would provide a universal hotkey that toggles between active profiles (right now, you would typically need to go through one or more Preferences/Settings pages to switch out all the preferences). My turn to pair? I press the toggle hotkey, and my preferred setup is loaded..
Another focus would include supporting accessibility considerations for pairs. Some folks need a considerably large font, for example. A pair-friendly IDE would provide the ability to easily sport multiple views across multiple monitors.
Using multiple machines might ultimately be a better world than my shared machine ideas above, even if I’m physically sitting next to you.
Screen-shared programming sessions usually suffer latency issues, particularly for the remote pair. If you’re a VIM or Emacs weenie, however, you can run an effective pairing session using tmux/tmate or screen. The character-mode interaction makes it almost as good as being there–particularly when coupled with cameras, quality audio, and a screen share so that you can see the UI interactions on the host machine. This is my preferred remote-pairing setup, but I admit that there are some great things I lose that IDEs like IDEA can provide.
Atom also supports a plugin called Motepair that “enables remote pair programming using Github’s editor.”
Subscription-based cloud IDEs like Cloud9, Codeanywhere (which has a free version), and Codenvy are an option, but for now, most of us would probably prefer to stick with our IDEA or VisualStudio or Eclipse. (And I’m resisting paying for yet another subscription…) Having not used these, I’m not sure how truly collaborative they are. Your experiences are welcome in the comments!
The subscription-based Floobits takes things one step further, offering the premise of allowing clients to use whatever IDE they choose… as long as it’s Emacs, IDEA, Neovim, Sublime Text, or Atom (no Eclipse or Visual Studio yet). I’ve not tried it, but it sounds fantastic. Koding appears to be another tool with a similar model.
Other attempts have been made to create remote pairing plugins for specific IDEs. I remember XPairtise, an Eclipse plugin, which was file-based (it synced at the file system level) and had no end of problems. Now there’s SAROS, a still-active open source initiative.
There’s little worse than watching someone struggle through typing story data into a tool like Rally. The tool often requires lots of clicking about, which can create a distraction for the rest of the team in the room.
In contrast, I’ve found simple virtual card wall solutions like Trello to be fantastic, however. I’ve run and been a party to iteration planning, retrospectives, and other distributed (and non-distributed!) meetings where Trello was the tool of choice. It’s highly tactile and responsive; all parties see changes as they occur. With plugins, things like dot-voting on cards promote Trello to a great general purpose tool for managing and tracking just about any collaborative meeting. Worst case is that you need to export the information to sync up another tool elsewhere.
Ultimately: I wish all applications supported true real-time collaboration. Google Docs is a great example of a tool that effectively supports live editing by multiple people.
If we really believe in agile, let’s insist that the tools we use and build all support collaborative capabilities, and not just as an afterthought.
Your business doesn’t want to hear about how you “delivered stories.”
The business wants you to deliver value, in the form of what they most likely refer to as features.
A story should trigger a conversation around a feature request. Your software product doesn’t contain stories, it contains features that help users accomplish goals.
Wikipedia describes a user story as “one or more sentences” that capture a user’s needs. Those sentences represent a brief description or summary based on the planning and preparation so far, and they are reminders of the conversation that you must continue to have around the feature you will build. The sentences are not the feature. They aren’t even the full story, in most cases, as it often takes the requester several more oral sentences or even paragraphs to say what they really want.
If you think about stories as a simplified form of a requirements document (“a couple sentences”), you’re constraining agile to be a small tweak to how you did things before, with a trendier name. That diminishes the most important aspect of how agile can help you deliver software successfully: Through continual collaboration and oral communication. Without understanding of the importance of collaboration, stories end up being another waterfall-like hand-off document.
On another front: Too many teams waste time over the wording of a story. It’s a discussion point, not an artifact. The “standard” story template should help you clarify feature requests by asking you to think about who needs them and why. But otherwise spending time on precise word-smithing of a story writeup misses the point. You’re going to discuss it, and you can iron out any ambiguities as you do.
Stories provide only minimal value as an artifact. The ensuing discussion is where the details come out. Unfortunately, most of us can’t fully recall details from a conversation a half year prior, or even a half month prior. Granted, the details end up embodied in the software, but source code more often than not does a poor job of expressing such details concisely and simply. Even programmers must sometimes spend hours to determine just what the code does for the smallest of features.
We need a better artifact than the few sentences that represent a story. Many successful teams have found that the best way to capture system behaviors is in the form of acceptance tests, best derived by use of acceptance test-driven development (ATDD) or BDD. The acceptance tests supplant the nebulous conversation with real meat. A test’s name describes the goal it accomplishes, and its narrative provides an example that quickly relates the details of system behavior. The set of test names describes the complete set of designed system behaviors. The tests capture the important detail behind the discussion triggered by the story.
A story is not a feature, and it should always trigger a conversation.
Pausing to determine where you’ve come from and why you are where you are can be a valuable exercise in knowing where you want to go, as well as in understanding the value you can provide. An exploration of my past dozen consulting years taught me that taking an occasional break from consulting has improved my ability to better understand and help subsequent customers.
In 1998, after over 15 years of professional development, I thought I’d be Mr.-Retire-From-Big-Corporation-With-Nice-Pension. In 1998, I had just reached my five-year mark with MCI. When WorldCom came around seeking souls to connect to the hive mind, my research determined that they didn’t provide such a hot place to work. I was in the 1% of shareholders who voted against it. Sometimes being right isn’t gratifying (what a debacle).
I moved on to a local dot-com, the now-defunct ChannelPoint. In 1999, I learned about XP** and did what of it I could–mostly some TDD–while isolated as a professional services developer in my shared office. Cool stuff!
In 2000, with ChannelPoint’s collapse pending, I excitedly began my foray into the world of consulting. I worked proudly for Object Mentor for two years. I then joined an XP team and enjoyed pair-programming day-to-day as “just another developer” for seven months.
An important revelation hit me: Promoting XP as a consultant and practicing XP as a team member are two completely different things. I’d preached about a few things–pairing, most significantly–that I’d done only as an external party, not as a true team member. Until you experience what pairing is like, day-in day-out for an extended period of time, you can’t possibly understand what it really feels like. And pairing felt good, but I also learned to recognize some problems with it.
I knew I had gathered very relevant information, and started consulting on my own. Cool! Well, the wife didn’t think so; she preferred me home. In 2004, I took on a local position again as a developer, and ended up a developer/manager for the better part of a year. I learned a lot more about what makes teams excel–I had a kick-ass development team, and they weren’t even doing much in an XP sense.
The call of consulting took me again on the road to help other teams learn what I’d learned. I transitioned to being an internal consultant at a large shop pushing the mandate of Thou Shalt All Be Agile All At Once (it doesn’t work, believe me). I started feeling out of touch, and the unending barriers presented by a rigid corporate structure made things all the more frustrating.
I returned to development as a remote programmer for GeoLearning for most of 2010. More learning about what works and what doesn’t! After GeoLearning was bought out and all remote individuals terminated, and after a local 3-month “just a developer” contract, I figured it was time to spread words again.
(The sneaky part: Employee vacation time over the years provided an opportunity to take on short training and consulting engagements here and there, largely to give me a fix for my books and training addictions.)
I’ve been back in full consulting swing for about 15 months. I wonder when that will need to end again. Yes, I could remain a full-time consultant and still provide good value for my customers. But I’ve found I can be even better by finding a way to occasionally immerse in a development team, to remain current, open-minded, and relevant.
** Extreme Programming. That’s for you, J.B. It’s not dead yet.
Tim, a modern urbane developer (i.e., he willingly practices TDD), talks about the reality of living in his team, circa 2008, where his organization was attempting to grow TDD.
I am not gonna lie, this place is crazy with buzz words and terms. They use words like Agile and Scrum as adjectives as opposed to ways of thinking and common sense still. I hear, “Well, we are doing Agile, so we don’t have time for design.” My jaw hits the floor… and then I start drinking heavily.
The team I am on currently, is a “center of excellence,” so to speak. We consult development teams within the organization (along with owning development projects, components, and standards). I feel pretty confident in my abilities to help teams out. I have no problem saying that I do not think of myself as an expert or perfect (yet). I have always been of the opinion that this should be a constant learning experience. I find it difficult to relay my thoughts, though, to the rest of the guys on my team. It is basically the three of them attacking me. It’s not totally me vs. the world, but it turns into a joke. I hear:
“…so basically you want me to double my development…”
“…there is no design in TDD…”
“…there is no empirical evidence that says TDD improves code…”
“…I have beens in shops before where the test cases were bogus…”
So…I think I know how you must feel sometimes 🙂 I never once say that it is a silver bullet. What I tell them is I didn’t realize how little I knew until I started using it as an approach. Primarily from a design and coding point of view. Don’t get me wrong, I have some smart-ass retorts, but I am just outnumbered 🙂 It’s a tad difficult for me to try and persuade them, because we all are strong developers (I think) on the same hierarchy, and everyone has their own opinions.
Fortunately or unfortunately, they will have to come on board though, because this is coming from the top down. The quality of code, and overall abilities of the development teams here is borderline atrocious for most projects. At least the ones we see on a regular basis. The good teams obviously don’t need our help. To put it bluntly, I have seen one too many 5000 line classes than I care to ever see again.
I think they (my teammates) have just had either bad personal experiences with TDD/Agile/etc., or bad education on it.
A commenter on last week’s post indicated that he viewed being the minority as an opportunity to show how much better things can go with TDD–fewer defects and getting code done sooner. That’s true, and it can feel gratifying, but you’re not likely to gain new converts by effectively showing them up–at least not immediately.
In a den of dogs with old tricks, some of the smart ones eventually come around, but in the short term they’re unlikely to admit that the new tricks might be better. Tim himself was sure his old way of coding (no TDD) was better until well after I’d left his scene.
In any case, from Tim we once again hear how bad experiences with TDD and agile can sour people. (Remember, this is four years ago.) What can we do to help? Here are a couple thoughts:
In “TDD Is Not a Silver Bullet” I reproduced part of an interview with Tim, a sharp young corporate developer that I’d paired with very briefly in 2006. In 2008, he sent me an email about his experiences in the interim. Tim’s stories are what prompted me to want an interview with him. Here’s a portion of the email.
Much to my surprise, it is being conveyed from the head of our development that we need to be “stressing” TDD. My surprise was… I thought we already were practicing TDD on an enterprise level. Regardless, I guess they had brought in this guy named [Very Big Agile Name deleted]. (I am sure he runs in your circles.) If you are friends with him, take what I am about to say not as an insult, rather my perception; but if he would have been the expert to introduce me to TDD, I would never even thought of adopting/agreeing with it. And, as it turns out, the teams he consulted felt the same way.
I have been telling my boss about my experiences with you, and told her it would be like night and day. I respected the fact, that while TDD was the stressed point with you, you also provided more than just using TDD as a silver bullet per se. Basic things… like knowing the language, having fun, communicating, and so on.
I got in touch with Tim this week to make sure he was ok with me publishing this stuff four years later. No surprise, Tim is still doing (and presumably enjoying) TDD.
If you’re a “real” programmer, you find coding fun at least some of the time. Of course, it might not necessarily be fun when the code fights you all the way, hiding bugs and throwing nasty dependencies in your path. Still, I happen to enjoy coding always, despite the challenges. Test-driving is fun, too–it’s more code, and it gives me confidence to craft cleaner code, which is even more fun.
Maybe learning doesn’t have to be fun to be effective, but I have to figure sucking the fun out of it on purpose can’t possibly work. (I’m disappointed that a Very Big Agile Guru could do that.) Yes, the sticky challenges of real code mean that you can’t teach TDD with a few trivial, “non-real-world” examples. That’s why I employ a deliberately horked-up, fairly realistic codebase in my training. But the class also learns that it’s only a challenge, not a barrier, and they quickly learn a few simple techniques to surmount the challenge. They have a good time learning TDD.
Ultimately, my students leave the classroom enthusiastic about the prospects for TDD, and confident that they at least have the basic understanding needed to begin tackling their challenges. After that, it’s a matter of their having good support for the practice. (Being the minority in a team, as I’ll relate in a later blog post through Tim’s stories, is a great way to lose that enthusiasm.) When I get to pair with developers to help them through learning TDD, both of us have a good time.
If you’re interested in coming up to speed on TDD, make sure you pick trainers and consultants who don’t suck the fun out of it.
As a writer, I’ve left a few incomplete efforts by the wayside, promising I would someday return and complete them. The one I feel most guilty about is an interview with a promising young developer named Tim.
In 2006, I had helped train and coach a development team in TDD at a large corporation’s headquarters in the U.S. In fact, the training and coaching were separated by a month or so. I initially worked with a cool manager who brought me in to run training, but by the time I returned for the follow-up coaching, the manager had departed. In his place was a humorless, dictatorial manager. While working with the team, I focused primarily on helping the individuals see how TDD might help them build quality software. Unfortunately the temperament of the manager meant I departed with few hopes for the success of agile or TDD at the team level.
When pairing with Tim, I could tell he was very sharp but also very skeptical and resistant to TDD. Two years later, I received an email from him. He had some interesting things to say. I asked him some questions on TDD, with the intent of shaping it into an interview I would publish. He asked that I not mention the company at that time. Well, four years later, I still think these are wonderful answers, and so I plan on publishing them over a couple or more blog entries. (Tim–sorry I took so long!)
Here’s one of the questions and answers from Tim.
Q #5. What did you finally see or recognize in TDD that you initially couldn’t see?
A. I have somewhat of a loaded answer to this question. I’ll preface with this… I didn’t go out and research TDD on my own. I had never heard of it, or knew what it meant. I wasn’t chomping at the bit to change the way I had been writing code for the 5+ years before that. It was directed from the top down in my organization, and my team was a test bed, so to speak. Right off the bat, I had preconceived notions as to why I had to learn this approach, but why should I? So I had no expectations as to what I was supposed to see other than tangible things that management probably thought were the only byproduct of TDD (i.e. JUnit/HttpUnit test cases).
So I can only answer your question with what I have found with using the TDD approach. It’s rather simple, and yet not so simple. I have to bullet them:
- I have found myself writing cleaner, more efficient, maintainable, and better “architected” code.
- I have a better understanding on topics as simple and innate as the Java specs, to more complex such as patterns, writing frameworks, and abstraction.
- Some would laugh, but I never followed the whole “code to the interface, not the spec” statement. I cannot imagine not following this now.
- My skills with the editor I use have become extremely advanced. So much so, that I would challenge anyone who claims TDD will slow you down that I can write the same thing simpler, faster, and tested.
- I automatically have a spec/documentation (the test cases) since that’s where I start. Any developer who says they keep up with documentation is flat out lying!
- I have more confidence.
- I have the feeling of “getting it”.
- And I have more fun.
I can admit that I didn’t fully understand TDD until a couple of years ago. So if you do the math, it took @2 years to really sink in, but I know I am better now than I was then (and I thought I was pretty damn good). I can only surmise that in 2 more years, I will feel the same compared to now.
By no means am I saying that I can solve any and every development endeavor and never get frustrated or challenged, all because of TDD. Development is challenging. If I didn’t get challenged than I would probably, really hate my job. I could do without the frustration, but that goes with the territory… and builds character. I feel like I have to say this because (and I think I mentioned it before) some developers I talk to think that TDD is a silver bullet to solve these issues just because you have some test cases and a build server, and as you can see (I feel) it’s much more.
One thing that stands out for me is Tim’s notion that he has “more fun.” More on that in an upcoming post.