Back to Best Of

Why Job Interviews are Flawed

Why does anyone believe they are good at interviewing people? If you ask experienced managers you’ll often her a set of well worn pet theories about how they interview people. Yet most interviewers, no matter how experienced they are, make instinctive judgments based on cognitive and other biases. And then afterwards, when reporting on the qualities (or lack their of) of the candidate, they back-fill logical reasons to support the intuitive responses that they’re in denial about. It’s no surprise that most job interviews are deeply flawed and unfair experiences.

Contributing factors include:

  1. Few are self-aware enough to sort out their biases. Few people posses the self awareness to realize why they instinctively like or do not like someone they’ve just met. And even fewer, especially in business and engineering circles, feel comfortable with their feelings. It’s considered unacceptable to say ‘the guy did well but I didn’t like him for reasons I can’t explain’. It’s much easier to hide that feeling inside unfair judgments, using whatever flavor of corporate jargon can be found in the official hiring criteria (Lacked intellectual horsepower, couldn’t deal with ambiguity, didn’t know the secret handshake, etc.)
  2. Talking about doing is not doing. Most interviewers focus on trying to extract a prediction about someone’s ability through having them talk about their ability. This is ridiculous. Could you evaluate an NFL running back by asking them questions about how they run? (e.g. “I run really really fast”, “Great, you’re hired.”). Better interviewers work hard to put candidates in problems and situations like the real ones they’ll face, and watch. They collaborate on real problems during the interview, as that’s what much of work is. Over time they’re able to calibrate what it means for a candidate to do well, given real problems, in an hour. But this requires skill and patience few interviewers have. And even when they do, the candidates are in an awkward and artificially stressful environment that does not approximate real work well, unless the interviewer is diligent on compensating for these issues (tip: hiring candidates for a trial project is often a better use of everyone’s time. Get a sample of them actually doing the job).
  3. Interviews work better as a filter. The job interview loop is more effective at eliminating bad candidates than identifying good ones. The bet is by the end only good candidates remain, but that’s not true. Like bacteria responding to antibiotics, strains of bad candidates that are immune to your process survive as well, and are hard to distinguish from good ones. The process can be prone to false negatives too (people who get rejected but would have thrived).
  4. Recommendations are underestimated. Since interviews are mostly bullshit, it makes sense to put more weight on a recommendation from a trusted person (not necessarily the names on the candidates resume) who has worked with the candidate somewhere else. They have first hand experience on the millions of things that can only be witnessed outside of the interview room. If you trust them, and they trust the candidate, that may have more predictive ability than 60 awkward minutes in your office.
  5. No one else saw what happened. Interviewers are free to lie and distort, intentionally or not. All interviewers are free to invent pet theories on which questions work best, or how good they are an extracting the value of a candidate. They are the only record of what they asked, how they asked it, and how the candidate performed. If they have bad habits that bias the candidate, no one will ever know, as the candidate has almost no ability to report on the interviewer. Every interview is a cat and mouse trapped in the room, and the mouse is motivated to do whatever it can to survive the cat, no matter how cruel or unfair the cat is.
  6. We never go back a year later and evaluate. The hiring loop at nearly all companies is broken, as there is no feedback loop. No one forces you to go back 6 months or 2 years later and see: how many of the hire decisions you made worked out well, and how many of the people you rejected kicked ass at other companies with similar cultures and needs. With no data, the value of any interview process is guesswork, not rigor.

Despite my affinity for this theory, I believe groups that take interviewing seriously, and leaders who reward interviewers for putting more time and careful thought into interviews, end up with better teams. The choice to hire someone is the most important decision you make that month, or year and the wise know this. At a start-up it can make or break the company. And the more seriously people take the process, even if it’s flawed, the higher the odds they’ll recognize the natural shortcomings above and invest in minimizing them.

While I don’t think all interviews are a crapshoot, I agree with the Winchester theory – in most interviews, most of the time, it’s mostly bullshit as a tool in truly evaluating how well a person would perform in the job, even if the people doing the interviews don’t intend it to be.

The real story is in who you recruit to interview in the first place. Better candidates in, better candidates out: see my essay how to interview and hire people. What is your take on interviewing? How do you work against the Winchester theory?

[This post was inspired by Royal Winchester who, when asked about interviews, offered: they’re mostly bullshit]