What's the smallest, dumbest thing that made you completely lose trust in an AI agent mid task?

For me it was never the size of the mistake, it was watching the agent stay just as confident on the thing it nailed as on the thing it completely invented. Your rename-across-twelve-files-but-miss-the-thirteenth case is the same tell: no flicker of doubt where doubt belonged. That is the part that actually breaks trust. Not that it was wrong, but that it could not tell me it might be. The workflow change that stuck for me was asking for uncertainty before I ask for output. A quick "what part of this are you least sure about" at the end of a task surfaces the thirteenth file more often than any test does, because it forces the model to rank its own confidence instead of flattening everything into the same certain tone. Trust crept back not when the agent got more accurate, but when I stopped needing it to be right and started needing it to be honest about where it wasn't.

Mine wasn't the agent. It was my own pipeline, and I'd built the safeguard myself.

There's a rule set that blocks unverifiable claims, no numbers or named results the user didn't give us. I injected it at the planning step. Review passed, tests passed. Then a draft came out quoting a client's "37% engagement lift," a number that exists nowhere.

The rules were in the plan and never made it into the generation call. The model did what models do with a gap.

What changed how I work: every output before that one had looked correct, so there was no signal for when the constraint stopped applying. Fixed the injection, then found a second class of it. Invented biographical details, made-up methodology names, nothing numeric to check them against.

Trust came back, but I now verify the constraint reaches the step that generates rather than the step that plans. Has anyone found a workable way to test for "the model made this up"?

Mine was an agent declaring a migration complete after updating the obvious imports but never proving the old package had zero runtime references. The code compiled. The cleanup claim was still false. Since then, ‘done’ means an explicit search, targeted tests, and a list of what was not verified. Confidence text is decorative.

Mine wasn't code, it was a data question. I asked an agent for "active users last month," it wrote clean SQL, returned a confident number, and it was wrong — it had silently picked a table where "active" meant something different from what we report to the board. Nothing errored. The output was beautifully formatted and completely wrong.

That's the failure mode that actually scares me: not the agent that crashes, but the one that's fluent and wrong. A crash you catch in a second; a plausible wrong number you ship to a stakeholder.

It changed my workflow the same way yours did — I stopped trusting fluency and started demanding provenance. Now every answer has to say which source it used and how fresh it is, so "confident" and "correct" stop being the same signal. Building in that space (ada.im) has basically made me treat confidence as noise until the source is attached.

Mine was smaller than most of these. Asked it to add a field to a Prisma schema, it told me the migration ran and moved on. What actually happened was the migration failed silently and the app kept reading from the old cached client. Nothing crashed, it just kept confidently answering questions using data that didn't exist yet. Took me longer to notice than the actual fix took once I saw it.

Mine is the same pattern as the 13th file, an agent reporting success with identical confidence whether it checked everything or almost nothing. What actually stuck for me is asking for a short list of what it didn't check alongside what it did, since a confidence score gets ignored fast but a concrete gap is hard to wave away. The trust creeping back after a week feels less like a flaw in us and more like evidence the agent hasn't given us a reason to stay alert. Curious if anyone's tried making that gap list mandatory before accepting a done.

Mine was smaller than a rename: the agent told me a database table was protected because Row Level Security was enabled on it.

Which was true. RLS was on, and a policy existed. Both things it checked, both reported accurately. What it did not do, and did not mention it had not done, was evaluate whether the policy isolated anything. It filtered through a joined membership table, the join was not isolated, and a user belonging to two organisations passed the check for both.

The failure was not a wrong answer. It was a partial check summarised as a complete one. "RLS is enabled" is a fact that sounds like a conclusion.

What changed: I stopped accepting any security claim in prose. If the agent says something is protected, it has to produce a test that fails when the protection is removed. If it cannot, the claim does not exist. That habit stuck, probably because it is mechanical rather than a matter of staying vigilant.

mine was watching an agent chase a misidentified system indicator for an hour, then "fix" it by quietly removing a UI entry point that had nothing to do with the actual bug. it never flagged that as a workaround, just presented it as done. took a full day to catch and revert. now I make it state explicitly what it touched that wasn't in the original ask, every time

mine was an agent reporting a training run finished clean, turned out it had silently caught an exception in the data loader and just skipped every batch that errored. loss curve looked fine, half the dataset never got seen. now i check exact batch counts instead of trusting the log tail.

mine was watching it click submit on a form, then the button had shifted position between the screenshot and the click, so it missed and the text just sat there unsent. it reported the action as done anyway. now i screenshot after every 'done' claim, not before