Hamid Matiny

Hi everyone I'm Hamid, a software engineer with a bit over a decade of experience. The last few months I've been heads-down building AEGIS: an open-source (Apache 2.0) defense-in-depth layer for LLM apps and agents prompt injection detection, policy enforcement, and tool/agent permissioning so a prompt injection can't quietly get an agent to misuse a tool.

What pulled me into this specifically: most guardrail tooling I looked at only reports how it does against a static, round-one test set, which doesn't tell you much about what happens once an attacker actually adapts. So I built an adaptive red-teaming harness alongside the defense layer itself, and published the honest result even though it wasn't flattering.

Launching on Product Hunt this Tuesday. Mostly here to learn from others working on similar problems what's everyone else building or running into on the LLM security / agent safety side?

I've been building AEGIS, an open-source defense layer for LLM apps/agents (prompt injection detection, policy enforcement, tool-permissioning for agents). Before I launch it here Tuesday, I want to share something I almost didn't publish.

Most guardrail projects report a round-1 catch rate against a fixed test set which tells you very little about a real attacker who adapts. I ran our detectors through adaptive red-team campaigns that breed new attacks from whatever got past the last round. The honest result: hardened, real-model detectors converge to roughly the same ~48% overall bypass rate as unhardened stub detectors by round 3. Full methodology and numbers are in the repo if anyone wants to pick it apart.

Genuinely asking this community: is round-1-only red-teaming basically theater at this point? Has anyone here seen a detection approach that actually holds up past round 2-3 against an adaptive attacker, rather than just looking good on a static benchmark?