My LLM never crashed. It just told a marketing manager to claim 8 engineers.

by•

I build LinkedInRank, which rewrites LinkedIn profiles and tailors resumes with an LLM. Nothing ever crashed. The output just read well and was quietly wrong.

Some real ones from my commit log:

  • A marketing manager's profile check suggested writing "Managed team of 8 engineers…" with a 25% improvement. There were no engineers anywhere in that profile.

  • A nurse with 7 years of experience and a BSc was written up as "Aspiring".

  • Bullets came back as "Architected managing…" and "Led led".

  • Connection notes took the recipient's job as the sender's own story.

  • Cover letters kept ending sentences with filler like "…, ensuring consistent publishing".

Prompting alone never fixed these. What worked was checking after generation, in code: every number, skill and employer in the output has to appear in the person's own resume or profile, or it's flagged or dropped. A bullet has to be supported by the role it sits under, or it's removed. Example text uses [brackets] instead of plausible fake numbers. The prompt asks; the check enforces.

How do you catch hallucinations in your apps that read perfectly fine? Evals, a second model, deterministic checks, or users reporting them?

16 views

Add a comment

Replies

Best

The agent lost context mid-task and silently rebuilt a helper that already existed two files away under a different name. It didn't throw an error or crash. It just branched the codebase into two diverging implementations.

Curious to know how long it took you to first figure out and then to fix that?
The models are becoming so good, and really showing so much confidence that catching these things is increasingly harder. We are building something in that space, but I find spec-driven pretty decent at avoiding them, evals have also limitations - you don't know what you don't know therefore hard to set the evaluation for some of the scenarios