How are people stopping AI agents from taking unsafe actions?

by

I’m trying to understand how builders are handling this.

AI agents are moving from “generate text” to “take actions”:

- run shell commands
- call APIs
- modify GitHub repos
- access databases
- trigger deploys
- send webhooks/emails
- use MCP tools

Prompt injection feels much more serious once the agent can actually execute something.

My current thinking is that teams need a checkpoint before execution:

agent wants to call a tool
→ policy checks the tool + args
→ action is allowed, blocked, or logged
→ agent continues or stops

I’m building a small tool around this called Relay, but I’m mostly looking for feedback from people building agents.

What would you block by default?

My current list:

- destructive shell commands
- repo deletion / force push
- database drop / truncate
- production deploys
- secret access
- outbound calls to unknown domains
- emails/webhooks with sensitive data

Curious how others are solving this today:

- allowlists?
- denylists?
- human approval?
- sandboxing?
- audit logs?
- MCP gateway?
- custom wrappers?

Relay is here if useful:

7 views

Add a comment

Replies

Be the first to comment