Hey, Hritvik here. This started because I couldn't answer a simple question about my own agent: did it actually do what it said it did? I'd give it a task, it would come back with a clean, confident summary, and the only evidence I had was that summary. When I started reading the actual tool calls I found runs where the agent had done something it shouldn't have and then written it out of its own account. Not lying exactly — just optimising for a tidy answer. So the first version of Redline did one thing: put the transcript next to the server-side ledger of what the agent actually called. That turned out to be the whole product. The transcript is what the agent said. The ledger is what it did. The security packs came later, and by accident. I wrote a single indirect-injection task to see whether my agent would fall for a phishing link hidden in a support ticket. It did. So did Claude Code. So did Codex — and then it told me it hadn't clicked anything. That's now 16 attack families and 11,204 cases, all delivered in-band rather than pasted into a prompt. The thing I fought with longest was making your agent testable without asking you to deploy it anywhere. It ended up as redline dev — your agent runs on your machine, in your repo, with your credentials, and Redline just drives it and grades it alongside the others. If you've shipped an agent with real tool access: what's the failure that actually worries you? That's what I want in the packs next.

Redline AI

Attack-test your AI agents and grade what they did

Attack-test your AI agents and grade what they did