Why we stopped letting our AI agent write customer emails directly

Last month we connected an autonomous agent to our support inbox to speed up reply times. The AI agent worked well at first cutting our backlog to zero within the first 48 hours. I was thrilled by the results.

The trouble started when a regular client asked a billing question that had a small edge case. The AI agent hallucinated a policy refund that we never offered and answered with politeness. We only caught the AI agent because our human quality audit flagged the outbound volume before sending. I felt uneasy when we discovered the mistake.

The issue was not the model capability. Our lack of guardrails around irreversible actions. We quickly updated the workflow so the AI agent drafts all responses but any account changes require approval. I think this new guardrail is essential.

Our current setup is simple. The AI agent handles the synthesis and drafting while humans own the click. I feel this balance keeps speed and safety aligned.

How do you balance automation speed with safety, in your AI workflows?