How reliable are the tools we use to red-team AI agents? (4 preprints, open-source eval code)
by•
Published four connected preprints on agentic AI security evaluation methodology — covering reliability gaps in automated refusal/prompt-injection detectors, and a controlled 7,020-trial study finding that agent framework choice explains ~0.06% of security outcome variance versus ~29% for attack type. All open (CC BY 4.0), code open-source. Curious what other builders in this space are seeing.
https://figshare.com/authors/Waqar_Javed/24479225
12 views
Replies
hello @iamwaqarjaved The framework result is a useful reality check. We often compare agent stacks as if the framework itself determines security, while the attack set and evaluation method may be shaping most of the outcome.
I’d like to see red-team tools report confidence, false-positive rates, and performance across different attack families instead of one overall score. Running the same test with multiple detectors and manually reviewing a small disagreement set could also reveal when the evaluator itself is the weakest part of the security pipeline.