Your LLM judge probably shares a base model with the thing it is judging. That is not a judge.

LLM as judge has quietly become the default way people check agent output, and almost nobody states which model is doing the judging. In most setups it is the same one that produced the answer, or a sibling from the same family. That arrangement fails in a specific and predictable way. The judge does not catch the hallucination because it finds the same thing plausible, for the same reasons, out of the same training. You have not added a check. You have asked the same mind twice and written down that it agreed with itself. Errors that are correlated between producer and judge are exactly the errors that survive, and they are also the ones you most need caught, because a plausible wrong answer is the only kind that ships. There is a second version of this that shows up inside agent loops. A builder agent grading its own work grades against its own model of what "done" means, which is the same model that generated the work, so it cannot see the gap by construction. And the moment your tester can read the builder's own claim of success, it anchors on it. Two things that seem to hold. Run the judge on a different model family than the producer, even when the judge is the weaker model, because independence beats capability for that job. And give the judge the artifact and the original spec only. No producer reasoning, no summary, no "I have verified this works". The reason people do not do this is cost and latency, and those are real. But a judge that agrees with the producer for free is not cheaper than no judge. It is more expensive, because it manufactures confidence you did not have before. What is your judge running on, and did you pick it on purpose?