The Black Box of Interviews: We Don’t Know What’s Happening, and That’s a Problem

Let’s be real: interviews today are a black box. We assume a lot about them, but we know surprisingly little about what really happens when that virtual or physical door closes. We put our trust in hiring managers, leaders, and panelists to make critical decisions—decisions that shape the future of our teams and our company—but do we have any real insight into how those decisions are being made?

Here’s the uncomfortable truth: we don’t. And that’s not good enough.

Interviews are often treated as private, subjective experiences. But what if we looked at interviews for what they actually are—a high-stakes mission to gather evidence that will shape whether someone belongs in our company? What if, instead of relying on gut feelings, we started demanding transparency, accountability, and continuous improvement from everyone involved in the interview process?

It’s time to get real about how we’re interviewing—and who’s doing the interviewing. Because even the best leaders need feedback, and even the best interviews can be improved.

Schrödinger’s Interview: Was It Great or Was It Terrible?

Right now, every interview we conduct is like Schrödinger’s Box. Until we “open” it—by seeing how the candidate performs on the job—we just don’t know if it was effective. Was it insightful? Was it a waste of time? Was the interviewer fair? Did they ask the right questions? We’re dealing with too many unknowns.

The problem is that by the time we finally get the answers, it’s too late. We either missed out on a brilliant candidate or, worse, we hired someone who’s not a fit, all because our interviews were not designed to extract meaningful, actionable evidence.

And let’s be clear: this isn’t just a problem of bad interviewers. This is about the system we’ve built, where we trust interviews without holding them to objective standards. No one questions it when a hiring manager says they had a “good feeling” about someone, but let’s be honest—gut feelings aren’t a strategy. If your hiring process depends on vibes, you’re taking risks you can’t afford.

Interview Training Is Not Enough

Here’s another truth: training alone doesn’t make someone a great interviewer. It’s easy to sit through an hour-long training session and then go back to interviewing the exact same way as before. Real growth doesn’t happen in a training room—it happens in the interview room, with real-time feedback and opportunities to improve.

Think of interview training like learning to drive a car. You can read the manual, watch someone else drive, and take a couple of lessons. But if you never have an instructor sitting beside you, giving you pointers on the road, are you really going to get any better?

The Solution: Shine a Light Inside the Box

If we want to stop operating in the dark, we need to bring visibility to the interview process. Here’s how we do that:

1. Transcribe and Share Interviews:It might seem radical, but interviews should be transcribed and, where possible, shared with candidates as an artifact they can use. Imagine getting an interview back—not just to remember what you said, but to see where you could improve, to understand how you presented yourself, and to grow from the experience.
2. Giving candidates their interviews as an artifact isn’t just transparency—it’s respect. It says, “We value your time, and we want this process to be valuable for you, too.”
3. Evaluate the Interviewers Too:Let’s stop pretending that interviewers are above evaluation. Every interview is a two-way street, and we should be holding managers and panelists to the same level of scrutiny we give to candidates. Did they ask meaningful questions? Did they adapt based on the candidate’s responses, or were they rigidly sticking to a script? Did they make the candidate feel comfortable enough to share real insights?
4. Interviewers should be evaluated on their ability to gather evidence, create a fair environment, and drive meaningful conversations. The idea that a good leader automatically makes a good interviewer is flawed. Interviewing is a skill—a skill that requires accountability and coaching just like any other.
5. AI Coaching for Real-Time Improvement:This is where AI-driven Interview Coaching comes into play. Imagine having AI listen to an interview, not just passively, but with an active rubric that can flag areas for improvement. Was the candidate given enough space to speak, or did the interviewer dominate the conversation? Were follow-up questions dynamic and responsive, or were they stale and rehearsed?
6. AI can provide near real-time feedback—insights that help interviewers grow. Did the interview have a natural flow, like a conversation between future colleagues, or did it feel robotic? These aren’t small questions; they’re the ones that determine whether we’re making the right hiring decisions.

Do We Really Know What’s Happening in Our Interviews?

We need to ask ourselves a tough question: do we really know what’s happening in our interviews? Most of us assume we do, but the reality is, unless we’re gathering data, analyzing performance, and making our interviewers accountable, we’re in the dark.

We trust that our panelists are doing the best they can—and I believe they are—but trust isn’t a strategy. If we want to hire exceptional talent, we need more than good intentions. We need real feedback, real accountability, and a system designed to improve with every single interview.

Good Interviews Aren’t Good Enough—We Need Great

Let’s stop aiming for “good enough” interviews. Let’s set our sights higher. Interviews aren’t just about vetting candidates—they’re a reflection of us as an organization. They show how much we value people, how much we invest in making fair, informed decisions, and how willing we are to look at our own processes and improve.

The truth is, every interviewer can improve. And if we’re serious about building the best teams, we need to treat interviews as the dynamic, evolving skill they are. We need to crack open the black box, bring transparency to the process, and commit to growth—not just for candidates, but for interviewers too.

It’s Time for Radical Transparency

It’s time for action. We need to bring radical transparency to interviews. To use AI to coach interviewers, to hold ourselves accountable, and to turn every interview into a learning opportunity for everyone involved.

Because gut feelings and vibes are not a hiring strategy. It’s time we did better—by our candidates, by our companies, and by ourselves. Let’s treat interviews as what they truly are: evidence-gathering missions, where we’re committed to fairness, growth, and making the right choices for our future.

‍