A benchmark without a failure taxonomy is just a scoreboard

A single score tells you who won. It rarely tells you what broke. For model or product evaluation, I want failures grouped before I trust the ranking: factual error, stale source, bad tool choice, latency collapse, unsafe action, or a task the system should have refused.

That taxonomy changes the next engineering decision. A two-point gain driven by easier cases can hide a worse regression in the failure mode users actually care about.

What failure category has changed how you read a benchmark?