Hardening barely moved our LLM guardrail's bypass rate. Is round-1 red-teaming theater?

by

I've been building AEGIS, an open-source defense layer for LLM apps/agents (prompt injection detection, policy enforcement, tool-permissioning for agents). Before I launch it here Tuesday, I want to share something I almost didn't publish.

Most guardrail projects report a round-1 catch rate against a fixed test set — which tells you very little about a real attacker who adapts. I ran our detectors through adaptive red-team campaigns that breed new attacks from whatever got past the last round. The honest result: hardened, real-model detectors converge to roughly the same ~48% overall bypass rate as unhardened stub detectors by round 3. Full methodology and numbers are in the repo if anyone wants to pick it apart.

Genuinely asking this community: is round-1-only red-teaming basically theater at this point? Has anyone here seen a detection approach that actually holds up past round 2-3 against an adaptive attacker, rather than just looking good on a static benchmark?

7 views

Add a comment

Replies

Be the first to comment