When hiring software developers, it’s important to know whether or not they can actually solve problems with code. This can be harder to get right than you think.

A quick chat

When faced with the problem of how to achieve this, most interviewers first try something along the lines of a general, fairly freeform conversation about code and technical matters. As they talk, they use their intuition and common sense to get a feel for how experienced and competent the candidate is. I’ve been on both sides of this kind of interview and seen them produce mixed results.

The first problem with the unstructured chat is that you’re not assessing the most important thing. You’re not hiring this person to talk about code, you’re hiring them to write it. You run a high risk of striking a false positive and hiring someone who knows all the buzzwords but can’t actually apply the concepts in practice. Worse, you’re likely to produce a lot of false negatives. You may pass on people who get anxious or flustered when discussing abstract technical concepts in a high pressure interview situation, but could produce excellent work in the actual job.

The next issue you’ll find with these kinds of interviews is that they’re very susceptible to interviewer bias. The baseline is never the same between two different candidates because the conversation is steered by the interviewer. We all have cognitive biases and the bias towards tribes or in-groups is especially deeply rooted. In my experience watching interviewers, these discussions can often end up down a rabbit hole of some esoteric technical detail that is important to the interviewer; but that the candidate is unfamiliar with. Often when I see this happen the detail has little or nothing to do with the job at hand. Our knowledge of how cognitive biases work suggests that this is more likely to happen when the interviewer and candidate are more different from one another.

A questionnaire

A common reaction to the above is a checklist-style questionnaire. The interviewer seeks to reduce the opportunity for bias by asking the same set of questions to each candidate and then giving them a score or ranking based on the results.

I argue that this throws the baby out with the bathwater. A qualitative approach is critical in interviews, as no two candidates will compete along exactly the same vectors. There is usually a description of a role that you’re hiring for, but the ultimate goal is to hire someone who will make the company more successful. You need to be open to respond to an explore the particular strengths of the individual. No two candidates are alike and you must be prepared to recognise all of the skills, talent and experience that they could bring to the role.

Some code

The most effective interviewing tool I’ve come across is to get a sample of the candidate’s work. What better way to see how well they build something than by looking at something they’ve built? What better way to guide a technical discussion than by having some actual technology to discuss?

It should be as close as possible to the kind of work you would actually like them to do in a normal day in the role you’re hiring for. Once you have this, you no longer need an abstract conversation about whatever is on your mind or a preconceived list of questions. You can talk to them about the real work product in front of you both.

You get to ask your candidate why they made particular trade-offs. Why did they choose this abstraction over another? How would they write that differently if it became important to optimise it for performance? What would they change if the requirements were different in this way or that?

In this way, you can see what they are really capable of. You can gauge their experience and their technical values. You can see how flexible they can be in the face of changing requirements. You can see how well they’re likely to do the job by simulating them doing the actual job.

A challenge

The gold standard would be to put them in your actual team, doing actual work, for a meaningful period. Ain’t nobody got time for that. Your team don’t have time to on-board that many people. Your candidates have other interviews to go to and probably an actual job to juggle.

Another common approach is to look at some of the candidate’s past work. This is often a non-starter as all of the candidate’s previous meaningful work is owned by whomever was employing them to do it. Some developers have open source portfolios you can draw upon. For those that do, however, it’s often in a different language or framework than what you use at your company. It’s almost certainly built to solve a totally different problem than the one you’re hiring them to solve.

Where that leaves us is getting the candidate to write some code specifically for this interview process. The requirements should look as much as possible like something they’ll actually be asked to do in a normal day on the role you’re considering them for. You can ask them to build out the solution using tools and approaches that match what your team is using. You can give each candidate the same set of requirements in order to establish a baseline.

I’ve hired a lot of software developers. This is why I use challenges. In the next post, we’ll explore what makes a good interview challenge.

— David Banham, Takehome