A technical interview can be a learning experience for all, or it can be a grim, soul-sucking grind. The first path will lead to an effective team, and the second will lead to a dysfunctional team; this is an exorable, near-mathematical inevitability. Think deeply about how your interview process will lead to the team your company deserves.

This post talks in general about the second interview in the overwhelmingly popular two-interview format, where the first is a 'culture fit' interview of an hour or two, and the second is the technical interview of about the same duration.

Here I discuss both a general philosophy and a specific, actionable approach to technical interview design, to make interviews effective, enjoyable and ethical.

Effectiveness #

Briefly, let's define an effective interview process for a particular job as that which will identify the best, or even an adequate, candidate for that job. An interview can do this by drawing as accurate a "map" as reasonably possible of what a candidate can do, technically speaking.

A secondary goal, but still important, is to leave the candidate with a positive feeling about your company, even if the process does not end in a job offer.

The Map Is Not The Territory #

There is a limit to how effective any evaluation can be, in the same way that a map cannot effectively show everything that's in the land it represents. In a map, all fractal irregularity of coastlines, street cracks, ant-hills and rotten post fences are subsumed in an abstract overview best suited for getting a car from here to there. There is nothing in a map about the beauty of a forest pond, the intimidation of a steep incline, the dangerousness of a particular intersection, nor who makes the best mojitos in town.

In the same way, an interview is not the job, with all of its daily grinds, challenges and triumphs. A job interview rarely covers, for instance, that which interviewers do not know they do not know, the "unknown unknowns"; on the contrary, interviewers almost always confirm their own and their employer's biases. The interview process format of having a culture fit interview of 2 hours or so, followed by a technical interview, is about as effective at finding a great match as going on a great first date and then deciding to move in together.

Understanding and accounting for this inherent lack of accuracy or precision must be part of any interview process. That said, there are some things we can do to make the whole situation easier.

Humility and Respect #

A quick note about courtesy and respect: have it. Even, and especially, for those candidates who do not do well during the interview. There are many reasons a candidate might flounder that have nothing to do with their technical abilities. Resist any temptation to revel in your power over your temporary ability to give another person a hard time: this is the moment you shine by being humble and kind. You are representing your company and your own personal morals. Not much more need be said. Either this already makes sense to you, or your life has already gone off the rails.

It also absolutely needs to be said, if the candidate is talented at what they do, you need to internalize well that they are evaluating your company at the same time. Even if they are nervous and submissive and seem rather desperate, and even if you believe your interview process is infallible, you must jettison the notion that they are a petitioner for your scarce resource. A candidate's rejection for the position is very rarely viewed as a failure, but it could very well be. Talent is a scarcer resource than your unfilled positions. Stay humble.

Similarity #

One goal of a well-designed interview process is to compare candidates. Only by having a similar interview format, with candidates having similar experiences, discussion topics and questions, can you even hope to have a basis to compare candidates accurately. If each interview is different and depends wholly on the mood and whims of the interviewers, there is really no basis to compare candidates' technical skill (in contrast to the culture fit interview, which can be more unique and wide-ranging).

Agenda #

Have an agenda, and have it be reasonably the same for each interview, in keeping with the principle of similarity. Write it down. Have the interview agenda be documented, and decided, around the same time you write the job description. Hand a copy of it to the candidate at the beginning of the interview. Everyone should know what is going to happen. It can even have some of the technical questions and terms you will cover during the interview.

Here is one template for an agenda:

General introductions #

The candidate is offered water, coffee or snacks. The interviewers and candidate introduce themselves, talk about who they are. This is really just to get people chatting. It's not very important what is said at this point. These are the normal introductions by which strangers transform into acquaintences. Ice-breaking, jokes, stories are great! Then the interviewers present the agenda and go over it with the candidate. The goal here is to relax. Remember, you want the candidate at their best.

I like to state up front that, while there is time at the end of the interview for candidate's questions, that this is a conversation, and the candidate is interviewing the company as well. "Please, feel free to ask questions of us at any time, not just at the end."

General tech discussion #

Ease the candidate into the topic of tech by asking them to talk about anything at all they find interesting in tech: perhaps a recent controversy, observation, trend, intriguing library or exciting project. Anything, really. As with the introduction, what is said here is not extremely important; the goal here is to ease the candidate's transition from outside concerns into a tech mindset. The interviewers can ask follow-up questions, but again, the goal is not test the candidate's knowledge, but to have a relaxed candidate. Nothing the candidate says here will be wrong. Also, bear in mind: having a candidate talk about their past projects is one of the least effective ways to evaluate their skill.

That said, there are some things the interviewers can start to notice. This is an opportunity for them to talk about what they are passionate about in technology, if anything. Even so, you don't know this person very well at this point, so you don't know what 'passion' looks like for them, you don't know what they sound like when they're passionate. Do they talk in a monotone about everything? Are they excitable people who sound passionate about literally everything? You don't know, and so, do not place too great an emphasis on this section.

Resume / CV / past experiences #

Here the interview can get a bit more detailed, covering specific questions about duties or projects in the past. Have these written down, and these will necessarily be different for each candidate. While you can start with softball questions like "What is your proudest technical achievement?", you should begin to turn to specifics. What were the projects you worked on in your last job? Why did you make those technical choices you did in your last project? Did you consider other options? If so, why did you reject them?

Still friendly, always, but the interview has begun. We are now about 15 or 20 minutes into the interview, and this phase goes for 10 or 20 minutes, depending on their experience.

Break? #

Offer a break of 5 to 10 minutes. It's about to get intense.

Technical questions #

If you're looking for a specific kind of technical expertise, or if the candidate claims expertise, this is where you probe their knowledge. Ask questions with clearly right or wrong answers: "What is the event loop?" "What is a monad?" "What is the difference between a left outer join and an inner join?"

The temptation that many interviewers seem to have, here, is to assume a stern expresson and fire questions at the candidate in an intentionally unsettling manner, interrupting before they are finished. Resist doing this. The process is unsettling enough, and you are not evaluating a candidate's ability to withstand a hazing. You want the candidate at their best. Unsettling the candidate reflects badly on the company and is ineffective at finding clear technical thinkers. I have met geniuses who are rattled by a 'hello', but are sweet and dedicated.

Never tell someone they have done well with a question when they have not.

This part can take anywhere from 20 minutes to 45 minutes or longer.

Break! #

Mandatory break. 5-15 minutes. It's about to get really intense. Encourage them to have a snack. Keep water in the room.

Coding exercise #

It is essential to have a coding exercise as a part of a technical interview process. Not doing this is a crap shoot. Or Russian roulette. Coding is where the rubber meets the road. Not evaluating a programmer's ability to program is an extremely poor decision, no matter their background.

That said, some interviewers resist giving any kind of coding exercise because they point out, not without merit, that a coding exercise is so unlike the job itself that a coding exercise is no better than a dice roll. I see that point. Still, give the simplest coding exercise you can and then disregard it completely, if you like. It's closer to the needs of the job than just talking about tech.

If the coding exercise is during the interview, be very clear and consistent about what is to happen, whatever that is. e.g. "We will present a problem. Outline a solution on the whiteboard. We are less interested in a correct solution than in seeing how you think through problems, so communicate verbally as you solve it." Or, "Here is a small coding problem. We will leave you alone for an hour as you solve it."

The topic of coding exercises is vast, and I won't do more than touch on it here. Entire articles, books, blogs and video channels are dedicated to the topic, so it does at least deserve its own (necessarily incomplete) section. More about that below.

Break! #

Yes, another 5 minute break. They might need a few minutes to collect themselves.

Debrief, feedback #

Give them the opportunity to talk about the interview process itself. Solicit feedback about how the interview process itself could have gone better. They also might have some feelings about their performance. This is where you can identify ways to improve and even lend a sympathetic ear. If they have done poorly, do not rush them out the door or be impatient. Answer their questions until the end of your allotted time. Tell them about what next steps will be.

If you like them, talk about how great it is to work at your company.

Departure #

Regardless of your evaluation, shake their hands and look them in the eye. They honored you with their time. Respect that.

Coding exercise #

As said before, some kind of coding exercise is essential. And, likewise, doing poorly on any particular exercise does not mean that they would be bad at the job, or bad at programming: only that they did poorly on an exercise. It is possible that the interview process or the test itself is poorly designed. I view coding exercises as but one channel to understand a candidate; an important channel, but only one.

There are many different formats for coding exercises: whiteboard exercise, take-home projects, multiple-choice tests, pair-programming, verbal questions about terminology. Here I'll only write some general principles, and get specific only about take-home projects for now.

Generally #

• Coding exercises should be the same for all candidates, in accordance with the Principle of Similarity, in order to accurately compare performance. An ad-hoc exercise, custom-designed for a particular candidate will not lead to an accurate evaluation.
• If the coding exercise has a clear, unambiguous, correct answer, then consider having unit tests. This eliminates biases about coding style, doubts about whether some particular solution can work, and misinterpretation of the result.
• If the coding exercise does not have a clear, unambiguous correct answer, then consider having clear, unambiguous acceptance criteria.
• Overall, leave more time for any exercise than might be usual. Yes, the question might be very easy under normal circumstances, but these are not normal circumstances.

Take-home projects #

A take-home project reveals a lot about your company, so consider it carefully.

Unless you pay the client, a take-home project or exam must not be intended to take more than an hour or so. It might take the candidate more than an hour, but that's their business. It must - must - be designed sincerely to take no more than an hour. Do the project yourself, or have a respected developer do it, and if it takes longer than an hour, scale it back.

It must have very clear acceptance criteria, preferably actual unit- or end-to-end tests. It must only evaluate the skill-set the candidate will be hired for. No UXD for a front-end engineer, for instance.

I will explain why in the form of advice to your candidate: If a company sends you a project that takes hours, or does not have clear acceptance criteria, this is a company that is unambiguously and clearly telling you that they will not respect your time and will have unclear, shifting and non-existent acceptance criteria in their day-to-day operations. If you receive a test like this, run. You have been warned.

Needless to say, if your take-home project has a deadline, ask the candidate when they would like to begin.

Preparation #

Apparently, it needs to be said: be well-prepared and on time. Here is how to do that:

• Go through the candidate's application and CV, and decide what experience-related questions you will ask.
• Print out any needed materials like agendas, copies of the CV, and coding exercises.
• Make sure that the interview room is booked and has fresh dry-erase markers, erasers, and a pitcher of water.
• Send an email to the candidate about what they can expect. Attach the agenda if possible.
  • If the candidate needs to bring any materials, then tell them.
    • e.g. portfolio, source code, links to previous work
  • DO NOT rely on the candidate's having laptop unless you have specifically requested they bring one.
  • Do not request that candidates bring a laptop, anyway. Supply one, if necessary.
    • Perhaps they develop on a desktop. They might have to borrow or buy one. This is no poor reflection on them.
• If the candidate will need a computer or other equipment, make sure that it is set up and working.

Miscellany #

• Do not mention how many interviews you need to get through.
• Do not hold interviews if you have no intention of filling the job.
• Do not ask candidates to rate their experience with some specific technology. The Dunning-Kruger Effect guarantees your experienced candidates will underrate themselves and your less experienced candidates will overrate themselves.

Conclusion / TL;DR #

Be kind. Be relentless. Be human. The way you treat people in the interview will exactly reflect the team you build. Dysfunction begins and ends with the hiring process. If your company wants highly effective, talented developers, then craft an interview process that reflects a highly effective, talented company.