How do you evaluate AI agents before trusting them with real users?

Building an AI agent is one thing, but deciding when it's actually ready for real users is much harder.

Unlike traditional software, an AI agent can perform perfectly in testing and still behave unexpectedly when it encounters new situations in production.

I'm curious how other teams approach evaluation before deployment.

Do you rely on benchmark tasks, automated evaluations, human reviewers, simulated user interactions, or something else?

What signals give you confidence that an AI agent is reliable enough for production? Have you found any evaluation methods that looked promising but didn't reflect real-world performance?

I'd love to hear how different teams are approaching this, since evaluation feels like one of the biggest challenges in production AI.