Why code challenges?

When hiring software developers, it’s important to know whether or not they can actually solve problems with code. This can be harder to get right than you think.

A quick chat

When faced with the problem of how to achieve this, most interviewers first try something along the lines of a general, fairly freeform chat about code and technical matters. Over the course of this chat, they use their intuition and common sense to get a feel for how experienced and competent the candidate is. I’ve been on both sides of this kind of interview and seen them produce very mixed results.

The first problem with the unstructured chat is that you run a high risk of missing what makes that candidate special. You’re not hiring this person to talk about code, you’re hiring them to write it. You run a high risk of striking a false positive and hiring someone who knows all the buzzwords but can’t actually apply the concepts in practice. Worse, you’re likely to produce a lot of false negatives. You may pass on people who can produce great work, but get anxious or flustered when discussing abstract technical concepts in a high pressure interview situation.

The next issue you’ll find with these kinds of unstructured chat interviews is that they’re very susceptible to interviewer bias. The baseline is never the same between two different candidates because the conversation is steered by the interviewer. We all have cognitive biases and the bias towards tribes or in-groups is especially deeply rooted. In my experience watching interviewers, these discussions can often end up down a rabbit hole of some esoteric technical detail that is important to the interviewer but the candidate is unfamiliar with. Often when I see this happen it’s about something that has little or nothing to do with the job at hand. Our knowledge of how cognitive biases work tells us that this is more likely to happen when the interviewer and candidate are more different from one another.

A questionnaire

The other side of this coin that many interviewers take is a checklist-style questionnaire. The interviewer seeks to reduce the opportunity for bias by asking the same set of questions to each candidate and then giving them a score or ranking based on the results.

I argue that this throws the baby out with the bathwater. A qualitative approach is critical in interviews, as no two candidates will compete along exactly the same vectors. There is usually a description of a role that you’re hiring for, but the ultimate goal is to hire someone who will make the company more successful. You need to be open to respond to an explore the particular strengths of the individual.

Some code

The most effective interviewing tool I’ve come across is to get a sample of the candidate’s work. What better way to see how well they build something than by looking at something they’ve built? What better way to guide a technical discussion than by having some actual technology to discuss?

It should be as close as possible to the kind of work you would actually like them to do in a normal day in the role you’re hiring for. Once you have this, you no longer need an abstract conversation about whatever is on your mind or a preconceived list of questions. You can talk to them about the real work product in front of you both.

You get to ask your candidate why they made particular trade-offs. Why did they choose this abstraction over another? How would they write that differently if it became important to optimise it for performance? What would they change if the requirements were different in this way or that?

In this way, you can quickly get to the heart of what they are capable of. You can gauge their experience and their technical values. You can see how flexible they can be in the face of changing requirements. You can make an earnest attempt at determining how capable they are of doing the job.

A challenge

The gold standard would be to put them in your actual team, doing actual work, for a meaningful period. Ain’t nobody got time for that. Your team don’t have time to on-board that many people. Your candidates have other interviews to go to and probably an actual job to juggle.

Another common approach is to look at some of the candidate’s past work. This is often a non-starter as all of the candidate’s previous meaningful work is owned by whomever was employing them to do it. Some developers have open source portfolios you can draw upon. For those that do, however, it’s often in a different language or framework than what you use at your company. It’s almost certainly built to solve a totally different problem than the one you’re hiring them to solve.

Where that leaves us is getting the candidate to write some code specifically for this interview process. The requirements should look as much as possible like something they’ll actually be asked to do in a normal day on the role you’re considering them for. You can ask them to build out the solution using tools and approaches that match what your team is using. You can give each candidate the same set of requirements in order to establish a baseline.

I’ve hired a lot of software developers. This is why I use challenges. In the next post, we’ll explore what makes a good interview challenge.