I’ve interviewed a significant number of candidates recently for a developer position at my current customer. I’ve never boiled down interviewing to an exact science, but I think I’ve managed to get a lot of bugs out of the process. The up-front work is where I’ve been concentrating most of my efforts, due to the large number of resumes. By “up-front,” I mean the efforts leading up to the decision as to whether or not to bring in a candidate for face-to-face interviews.
Prior to even looking at resumes, putting out a good job description is an important step. That’s a topic for another blog entry. Once you have a stack of resumes, your job is to whittle them down to a small enough number. My recommendation is to set aside two weeks for screening candidates. Over that two weeks, expect to run an average of 2 phone screens a day. (Don’t try to do more than a couple a day, unless you want to go insane!) This allows you to cover up to twenty candidates if necessary.
Each phone screen should last from 30 minutes to an hour. This is a feeling-out process for both of us. I try to give the candidate all the negatives up front and give them the opportunity to beg off. Most don’t. I ask a number of conceptual questions, mostly design or process related. I look for the ability to carry a technical conversation. Does the candidate have knowledge of significant development concepts (patterns, OO, unit testing, etc.) that is commensurate with their claimed experience?
For a while, I would ask technical questions over the phone. It’s awkward, not very effective, and ultimately frustrating. I brought in a few people for face-to-face interviews; these people ended up being good at answering technical questions over the phone but had little additional value.
Recently I began a practice of sending out an email challenge to each candidate that I was still interested in. The email has two parts. The first part is a number of simple questions and answers regarding the language of choice (Java here). Answers to these few simple questions can reveal a lot about the core knowledge of each candidate. You’d be surprised, for example, how many candidates have a tough time explaining the difference between .equals and ==. Even with Google at their disposal!
For the second part, I grab a prototypical sludge method from the system. The candidate’s job is to figure out what the messy code does. They can define a contract for the method, write a unit test, add comments, or do whatever they choose. I also suggest they rewrite the method. The nice part is that they can cheat as much as they want–they still have to be the ultimate arbiter of what is quality code.
I give the candidate two days to return the email. From there, it’s a dialogue. I challenge the candidate as to their answers and as to why they coded things a certain way. A candidate might get a number of questions wrong, or have questionable programming style, but that doesn’t preclude them from further consideration. I look for their ability to think, to justify their answers, to perhaps open their mind up to different ways of looking at things. I bring in candidates for face-to-face interviews based upon the results of this dialogue.
I used this technique many years ago when doing a slew of interviews. It worked well then, and it still works well now. One other benefit is that it gives candidates an opportunity to shine if their phone chat didn’t go so well. And most candidates actually enjoy it, since most good developers are geeks who like a challenge.
The email challenge also does a good job of ferreting out the troublemakers. One recent candidate (I’ll call him Joe Blow) interviewed for a lead developer/architect position on the team. All went well on the phone. But answers in Joe’s email response were commensurate with a mid-level developer. A couple of answers to the Java questions were dead wrong: Joe demonstrated complete misunderstanding of the difference between instanceof and getClass(), for example. The code he sent back was pretty confusing in one spot, and only adequate elsewhere. Further, Joe hadn’t bothered to compile it, which I had explicitly required per the instructions. I pointed out the incorrect answers, as well as the code issues, including items for which there is no “right” answer, just opinion.
All of the other candidates were able to cope with me picking at their code. Some of them challenged me back–good!–or acknowledged that they could have done things differently, and why or why not it might be a good idea to do so. Unfortunately, Joe Blow couldn’t handle the fact that someone might actually see things a different way. He ended up withdrawing from consideration for the position immediately after providing his very defensive responses. Apparently he couldn’t take the heat. He wrote a lengthy email to his recruiter, and also accused me of looking for people who coded in a specific Java “dialect.”
Touchy and perhaps even whiny: this is not the kind of candidate you want for your lead developer! Still, I learned from the process. I added a few more disclaimers to my comments (“these are not intended to suggest there is one right answer,” for example) and tried to show a bit more compassion in my already-sensitive answers.
Ultimately, when you’re looking for a developer, you want to know if they can write maintainable code. Asking technical questions rarely gives you the answer. The only way to find out is to get them to write some code. Sitting and pairing in a live interview accomplishes this effectively. But if you want to save the time and expense of bringing in candidates for face-to-face interviews, the email challenge is a great tool.
