The rules we wrote after our AI agent started answering real prospects

When we first let our AI agent answer real prospects, I thought the hard part would be making the replies sound good. The replies were actually fine. The scary part was all the mail that isn't really a person.

Out-of-office messages. Bounces. "Please remove me from your list." To a model all of these look like someone wrote back, and someone writing back looks like a conversation. So it wants to be helpful and answer. Picture a polite follow-up landing in the inbox of someone who's on holiday, and then another one when their auto-reply fires again. Not great.

So the first rule was the boring one. Some kinds of mail never get a reply at all. Auto-replies, bounces and unsubscribes get filed and that's it.

The second one came from my own nerves more than anything. There's a hard limit on how many emails can go out in an hour, even when everything looks fine. If something goes weird at 3am I'd much rather wake up to a short queue than to an apology tour.

Third, anything about price or refunds never goes out on its own. The agent writes a draft, a person reads it and presses send. Those are the emails where one wrong word costs the most, so it felt silly to let a model have the last word there.

And the one I like most now is the simplest. Before anything important goes out, the agent shows what it's about to send, to whom and when. Seeing it laid out like that catches more than I expected. Most of the time it's fine. Every now and then you go "wait, why is it emailing them??"

None of this is clever honestly. It's mostly us admitting that a model will happily keep going until something tells it to stop, and somebody has to write that something down.

How did you handle this with your own agents? Did you start strict and loosen up over time, or let it run and add rules after the first surprise? 😅

2 views

Add a comment

Replies

Be the first to reply

Have a question or a thought to share? Add a comment above to start the conversation.