Improving coding interviews

I see this question all the time show up in one way or another - Why do we need coding interviews?

Just to clarify, when I'm speaking about coding interviews, I'm specifically referring to an algorithms & data structures type of interview. I'm sure most of you know this famous tweet from Max Howell:

This story went viral back in 2015 when Howell, creator of Homebrew, had an interview at Google and was rejected after failing a coding round involving inverting a binary tree on a whiteboard.

Furthermore, many engineers feel that this style of interview is not representative of their skills and doesn't translate into anything they'd do in their daily jobs.

As an interviewer myself, I've conducted over 100 of these interviews, so I understand their pros and cons. I'm not saying that coding interviews are good or bad, they're just one tool that companies can use in hiring (and many do). And even if you like them or not, joining the majority of tech companies, especially FAANG, requires that you pass one or more coding rounds. In this article, I want to discuss a few ways in which companies can optimize coding interviews to leverage their strengths and minimize the frustration on the side of candidates.

The not so good parts

Let's start with why many candidates dislike these types of interviews:

• Irrelevant - This is probably the one you'll hear most often and I agree to some extent. It's one thing to know fundamental algorithms and data structures, but many of these questions require you to have a lot of practice in solving similar algorithmic style problems, which let's be honest - almost none of us do at the job.
• Time pressure - Typically candidates are given at least one, but more often than not two coding questions in the span of 45 minutes. This usually equates to 15-20 minutes spent on each question, which is not a lot of time. Engineers don't need to solve problems in 15 minutes at their work, which puts an unnecessary stress on candidates.
• Bias - Usually the more senior you get, the less you deal with algorithms and the more you deal with designing and architecting systems. This means that senior engineers lose touch with these types of problems and tend to do poorly in coding interviews unless practiced on a regular basis. On the other end, candidates straight out of university tend to have an advantage, as they have this knowledge fresh in their minds.
• Rewards repetition - You can argue that this might be a pro or con, but you can't deny that a candidate that prepares for this type of interview will be wildly more successful than others that don't. This leads to the interview feeling like an exam, where you can become very good at passing tests, but not necessarily good at applying that knowledge in real-world scenarios.
• Constrained setup - You don't really write code at a whiteboard in real life, but engineers are expected to do so in coding interviews. Or sometimes now, in the age of COVID, in a Google doc (looking at you Google). Although I understand the intent behind this - candidates being comfortable with their chosen language and writing bug-free code without the help of an IDE - I don't believe this gives enough useful signals to interviewers to warrant using this approach.

The good parts

That being said, there are a lot of pros as well in my opinion. Let's look at a few of them:

• They test problem solving skills - No other type of interview tests abstract problem solving skills in such a short amount of time. Even though the problem itself is not something candidates would solve regularly, the process of coming up with a solution demonstrates their overall cognitive abilities and problem solving. It allows us to evaluate candidates on how well they can take a complex, perhaps ambiguous problem, break it into smaller problems, implement them step by step, motivate and defend their decisions and explain the different trade-offs.
• Rewards preparation - The types of questions are well known and thanks to sites like Leetcode candidates have all the necessary tools to prepare and become better at these interviews. This rewards candidates that have put in the time and effort to prepare.
• Consistency at scale - One big reason that companies like Google have been using coding interviews for many years is that it enables them to get consistent signals from candidates with a wide range of backgrounds and experiences. Problems can be calibrated in terms of complexity and time and candidate evaluation can be made consistent by setting multiple milestones. This is what allows big tech companies to consistently hire a lot of great talent over the years.
• Hiring new talent - Deciding if you want to hire a junior engineer cannot be done based on their CV alone, so companies need a fair tool to assess their potential instead. Coding interviews are just that - something that evens the playing field for engineers who don't have a lot of experience.
• Equal playing field - This is one of the biggest strengths of coding interviews in my eyes. No matter if the candidate worked in backend or frontend, cloud technologies or micro-controllers, this interview gives them an equal opportunity to demonstrate their skills.

Can we do better?

Taking into account the pros and cons mentioned above begs the question - can we do better? Can we modify or optimize the process in such a way that we keep most of the good parts and eliminate some of the most frustrating ones? In my opinion yes, there are several things that we can do in order to make the most of coding interviews.

Proper environment

This is a low hanging fruit. I don't think coding on a whiteboard, sheet of paper or basic online document brings any value to the interview. In can maybe give you a sign of how reliant is the candidate on their IDE and other tools, but let's be honest - engineers have so much flexibility and customization in the tools they use nowadays, that it's normal to get used to features like syntax highlighting, auto-complete etc. Forcing candidates to use a minimal setup only serves to slow the overall pace of writing code.

There are so many good collaborative tools out there, such as Coderpad, CodePen, Hackerrank etc. that there is no reason not to use them. It allows candidates to use their favorite programming language, test and debug the code in real-time and it also allows you as the interviewer to pre-configure the environment in terms of problems, test-cases etc.

Start easy

From what I've seen, one of the most common mistakes that interviewers do is jump straight into the problem after a very brief intro. Especially if the problem is hard, this puts the candidate under immediate stress. It's hard to be logical and methodical when your mind is racing under pressure.

In my opinion, it's very important for the candidate to relax. The last thing we want is to reject good candidates because they were stressed and unable to show their best self. As an interviewer, you should start the discussion with a few things about yourself, smile and display a general positive yet professional attitude right off the bat.

Furthermore, the first thing the candidate hears shouldn't be a hard problem, even if you intend to work towards that eventually. Start with a relatively easy problem and go from there. Solving an easier question will boost the candidate's confidence level and put them in the best position to answer more challenging questions.

Pick one (good) problem

This ties directly into the point above of starting things easy. In my opinion, two different medium or hard questions in 45 minutes is too much - usually only very prepared candidates can solve both optimally. The issue is that starting with a problem where the candidate gets stuck from the beginning usually leads to a poor performance and not enough signals for you as the interviewer to assess them.

What worked very well for me is picking problems which have multiple solutions and good follow-ups in increasing order of complexity. Here are a couple of examples of problems I've used in the past.

If the candidate solves the question quickly, another follow-up that I consider useful is asking things like:

• How would you implement this algorithm if you had N machines instead of one?
• How would you support a very large input data set that doesn't fit in memory?

I've found questions like these lead to very interesting discussions and can give the interviewer different signals - practical applications of algorithms, concurrency, distributed systems etc. While technically out of the scope of the coding interview, oftentimes I find it very useful especially for more senior candidates.

It's more than coding

I see many companies focusing only on the coding aspect of the interview and ignoring a lot of other important aspects.

Start with the code itself. How does the candidate name the variables? Are they duplicating logic or abstracting it away? Do they have a good grasp of their programming language of choice? Do you see any anti-patterns?

Look at the process. Does the candidate jump straight into the code or ask clarifying questions first? Do they consider multiple approaches and trade-offs? Do they stop once they found a solution or try to optimize it further? Do they walk through the code before running it or hit run every time? Do they test the code to make sure it works as expected?

Lastly pay attention to communication. Do they treat the interviewer as a colleague/partner to solving the problem? Are they able to clearly articulate their solution and decision making process? Can they defend their ideas? Can they explain their thought process along the way and keep the interviewer engaged? Are they receptive to hints and feedback?

Don't overlook these aspects when evaluating a candidate. Besides their raw problem solving skills, these signals are very relevant when deciding whether to hire someone.

Candidate point of view

I hear what you're saying - these ideas are nice and all, but as an interviewee I have upcoming coding interviews, so what can I do in the meantime?

The biggest thing to realize is that companies and interviewers are aware of the shortcomings of coding interviews. You are not expected to be perfect at this. Actually many companies that I know treat this more of a pass/fail interview, where your level is determined by subsequent interviews.

That being said, I have one tip for you - prepare. At the end of the day, it's a skill like any other and preparation is the key to success. There are many online (free) resources that you can leverage to practice and also learn. Spend some time regularly on this and you'll eventually succeed in passing even the hardest of interviews.

Conclusion

While coding interviews are not perfect by any means, I think they are a powerful tool that we can leverage to hire great talent. Coding interviews got a bad reputation in the last few years and this is mostly because some companies fail to adapt it to new market conditions and candidate expectations.

The biggest thing I want to leave you with is - don't overdo it, either as a candidate or as a company. Use it where it makes sense, but make sure you're not losing great candidates because they don't know how to invert a binary tree in 20 minutes.