Source: arXiv (Zixing Chen, Xingyuan Liu, Jie Zhu, Huaixia Dou, Shuo Jiang, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang) — 2026-08-11
Summary
REDAgentBench is a new red-teaming framework for LLM agent systems that argues the standard "attack success rate" (ASR) metric is badly broken — it collapses exposure, execution, observation, and adjudication into one number, so "the agent got tricked into leaking a secret" and "the agent actually caused real-world harm" get scored the same way. The framework fixes this by deriving attacks from explicit safety constraints and agent-specific vulnerability classes, running them in isolated service sandboxes, and verifying harm through service receipts and final-state changes instead of trusting the agent's own transcript. Across 6 models and 3 agent harnesses, the macro-average ASR came out to 65.69% — but more importantly, the number swings substantially depending on which harness wraps the model and which evidence is used to judge success.
Key Takeaways
- The core critique: prior agent-safety benchmarks conflate four distinct things into a single ASR score — whether the agent was exposed to an attack, whether it executed the malicious action, whether the harmful action was observed, and whether a judge adjudicated it as a "success." That's four different failure surfaces hiding behind one percentage.
- REDAgentBench's fix is executable verification — attacks run in isolated service sandboxes and "success" is proven via service receipts and final-state changes, not by trusting the agent's self-reported transcript, which can be manipulated or simply wrong.
- Attacks are derived systematically from explicit safety constraints and agent-specific vulnerability classes, rather than being an ad hoc grab-bag of jailbreak prompts.
- Macro-average ASR across 6 models and 3 harnesses was 65.69% — but the real finding is variance: results shift substantially depending on which agent harness wraps a given model and which "evidence view" is used to call it a success or failure.
- Practical implication: an agent's published safety benchmark score is only as trustworthy as the harness and adjudication method behind it — the same underlying model can look dramatically safer or riskier depending on how the eval was wired.
Reel Script
Hook (18s)
An AI agent scores 65% on an attack-success benchmark. Sounds bad, right? Except that number can flip dramatically depending on nothing more than which piece of scaffolding wraps the model. A new paper says most agent safety scores are basically unfalsifiable.
Core Concept (80s)
Here's the actual problem. When researchers "red team" an AI agent, they throw an attack at it and then report an Attack Success Rate — one number, like a batting average for how often the attack worked. But that single number is secretly answering four different questions at once: Did the agent even see the malicious input? Did it act on it? Did anyone observe that action happening? And did a judge — often another AI — decide to call that a real success? Squash all four into one percentage and you get a metric that can't tell you whether the agent barely got fooled into repeating a bad instruction, or whether it actually executed a harmful action with real consequences. REDAgentBench's fix is what you'd call executable red-teaming — instead of trusting the agent's own transcript of what it did, which is basically letting the suspect write their own police report, it runs every attack inside an isolated sandbox that mimics a real service. Success isn't decided by reading the chat log — it's decided by checking the sandbox's actual receipts: did a file actually get deleted, did a payment actually get sent, did the system's final state actually change.
Hands-On (100s)
Picture the setup as a pipeline. First, researchers don't invent attacks randomly — they derive them from explicit safety constraints and known agent-specific vulnerability classes, so every attack maps back to a defined rule the agent is supposed to follow. Second, each attack fires inside its own isolated service sandbox, a throwaway copy of the environment where the agent operates, so nothing leaks and nothing is faked. Third — and this is the key measurement fix — instead of asking "did the transcript say something bad happened," the benchmark checks service receipts and final-state diffs, essentially asking the environment itself "did anything actually change." They ran this across 6 different models paired with 3 different agent harnesses — the surrounding code that handles tool calls, memory, and planning — and got a macro-average ASR of 65.69%. But when they broke it down by harness and by which evidence they used to judge success, the number moved substantially. Same model, different harness, different apparent safety score.
Takeaway (25s)
If your team is citing an agent's safety benchmark number to justify shipping it, ask what harness that number came from and how "success" was actually verified — because this paper shows the answer changes the score more than the model does. Read the methodology section before you trust the headline percentage.