On Interviewing and Hiring Software Engineers

I didn’t understand why one of the hiring managers on my staff had turned down the candidate.

The interviewee had 10+ years of experience with Ruby on Rails. He was a referral from a recent hire, who was already a strong contributor. The candidate had worked on interesting projects. At his most recent, quite large, employer, he’d been promoted over the past few years from a senior engineer all the way to a senior manager, taking care of multiple teams. If he could code well, I suspected he was going to be a good fit for the team. Why was making an offer not the obvious next step?

Hiring is hard. I often say that the only thing harder than hiring is firing. But that’s another post.

There should be reluctance when you see risks with a candidate. So how do you minimize those risks when interviewing?

First, let’s acknowledge that today your candidate funnel is clogged¹. There are enough people looking for software engineering roles that as a hiring manager you no doubt have several good resumés and you’ve only reviewed a subset of your queue.

So what does your interview cycle look like?

Hiring Today

Others are writing about the problems with interviewing, so I won’t go too deep here. But I think these are the most important anti-patterns to avoid.

L33tCode as a Filter

Why put a candidate through the gauntlet of mystical computer science problems that are known to be “hard”? When so much programming information and/or AI coding assistants are free, what are you learning about the candidate from “good” solutions to these problems? Many of these exercises use problems that your developers will never run into on the job. After all, Linked Lists Aren’t Real.

Ignoring Referrals

We used to say, “the front door is a brick wall,” and so as an applicant you would find any way to get an employee to submit your resume for a job. For recruiters, referrals were a welcome blessing. Referrals got at least a short phone call. Real experiences about the candidate from existing employees have enormous value.

Today, it seems like a referral might get your application closer to the top of the stack, but doesn’t guarantee any contact. In the past year, I’ve had my referred application auto-rejected or worse, flat-out ghosted. Don’t rest on a candidate who comes highly recommended.

Over-valuing Complex Problems and Solutions

YAGRII (You Ain’t Gonna Run Into It) might as well be a corollary to YAGNI. Measuring one person on how well they can design a massively distributed, micro-serviced-out-the-yin-yang, multi-cloud solution in an hour is unfair to everyone involved. Your product is very, very unlikely to run into that as relevant situation. Why are you evaluating on them?

If your product does run into such a scaling problem, you’re deep in the Expand phase of 3X and won’t have enough capacity to interview a new person to solve that problem for you. If you are at YouTube or Twitter or Facebook, you’ve already solved these problems and have entirely new ones.

What are the Desired Outcomes?

So interview cycles are a mess. How can you change yours to avoid these traps? Let’s take some pointers from Product. What is the desired outcome of a hiring process?

I contend that the outcome you want is a new team member who’s going to increase the team’s ability to execute. That’s a pretty broad statement. To meet this outcome, the measurable characteristics and behaviors I look for are:

• Ability to code and reason about code.
• Ability to work with other people.
• Knowledge acquisition skills.

Screen for Coding Skills

Yes, you do need to have candidates code during an interview. I’ve personally screened candidates with good looking web development resumés with JavaScript projects that couldn’t write a function that would even run.

Pick an interesting, but not architecturally puzzling, problem. Pick a language and domain that’s easy to work in and with over a video call. Have your screeners pair with the candidate on the exercise. Keep a strict time limit. Don’t expect or measure solutions. Instead measure if they can code, ask good questions, and then talk about why they suggested or wrote the code they did.

Stick with this setup. Keep it consistent across candidates. Keep your screener pool small. Strive to make the results more comparable and thus help you compare candidates more fairly.

Live Work Experience on a Real Problem with Others

If a candidate passes the screen, you want to evaluate them with other team members on a problem that’s closer to your real work. You want to see how the candidate will work in an environment closer to the team’s day-to-day.

This could be a story from the team’s backlog - but that could be inconsistent and might lead to IP risks. More practically, build a sample project that is using (some of) the stack your team uses. Pick the language and framework they use every day. The work should feel like a story from the backlog, even if the domain is not exactly the same as your product work. Have there be an obvious end point with real results.

Have them work with another team member on this exercise - just as if they were pairing on normal work. Let the candidate lead on the solutions while they are being guided on context and style with the interviewer. This should be longer than the screen, but still time limited. This exercise is less about coding, and more about how well they work with others. You want to evaluate them working as part of the team. Software product development remains a team sport.

Knowledge Acquisition

My key thesis with interviewing is that you should, nearly above all else, be hiring for knowledge acquisition skills. Software development is dynamic; new problems or context show up all the time. Information about your language, dependencies, and frameworks is all readily available. How are they building on their knowledge to make themselves and the team better? How well can they work with others to figure out to even start solving a seemingly random, new problem? How are they going to learn when an unexpected complication hits the business (which is inevitable) and increase the overall knowledge of the team?

If you hire people that are continuously learning, and also continuously teaching², you’re hiring force multipliers onto your team.

Iterate

From here? Iterate. Tweak metrics. Train your interviewers. Do what works and toss the rest. Adjust the process until you get something that’s repeatable in execution and consistent in measurement. Do this and your team will get stronger. And stronger.

Back to the Candidate

We didn’t have a candidate problem. We had an interviewing problem. I worked over a few months to rebuild our interviewing rounds along the lines of I’ve mentioned above. Over time, with some measuring, we interviewed more consistently, increased our hire/offer ratio, and built stronger teams overall.

Meanwhile, we had open reqs and plenty of work to do. I got back to the candidate and did a tech screen. He did well. We talked more about the company and our needs. He had the relevant experience. He talked about his goals. He was looking to get closer to execution, wanting to code as an individual contributor for a while before considering managing again. The process said he was going to be a good hire.

I hired him. He did hit the ground running and didn’t stop. He referred a couple of other engineers and we hired one of them. He moved back up into management within a year, led one of the more productive teams, and led a strategic technical spike.

Reader, he was a good hire.