How much do you actually trust an AI-written job-match score?
by•
Working on scoring candidates against job postings with AI, and I keep going back and forth on this: a bare percentage feels untrustworthy no matter how good the model is, but a full paragraph of reasoning is too slow to skim across 50 profiles.
Landed on a point-by-point checklist (what's met, what's missing) as a middle ground, but curious how others handle this. If a tool told you "73% match," would you trust it without seeing why? Where's the line between useful signal and noise for you?
32 views
Replies
A percentage without context feels like a credit sore for a human 😂. I’d much rather see the app the top3 things that drove the score.
I ran 500+ assessments and 200+ tech interviews, and honestly humans don't trust their own scores either. That's why we always made people write the reasoning first and the number second. Number first and you spend the rest
of the review defending it.
So for me the question isn't "do I trust 73%". It's what the number is for. If it sorts 50 profiles so I read the right 10 first, I trust it fine, being roughly right is enough for ranking. If it decides anything on its own, no amount of reasoning text would save it.
Your checklist is the right call, but I'd flip the emphasis: show the misses, not the matches. When I skim 50 candidates I'm not looking for reasons to say yes, I'm looking for the one gap that makes it a no. Three red items I can scan in a second beat a balanced summary I have to read.
The thing I'd actually want: let me disagree once and have it stick. Mark "missing Kubernetes" as irrelevant for this role and never see it counted again. Trust doesn't come from better explanations, it comes from the tool losing an argument.
The score should help me decide where to look, not decide for me. I would find a short evidence based breakdown much more useful than a percentage, especially when one missing requirement can completely change a candidates suitability.
A single 73% hides too much. I’d separate hard requirements from soft-fit signals, then use the percentage only for ranking. Someone missing one critical requirement shouldn’t look equivalent to someone with a few minor gaps. That would make the score much easier to trust at a glance.