Tool-using agents often read untrusted external content, which exposes them to indirect prompt injection (IPI). In this attack, hidden instructions embedded in that content can hijack the agent's tool calls. To assess how secure agents are against such attacks, benchmarks commonly report the attack success rate (ASR).
The new paper, titled "Ignored, Not Refused," argues that a low ASR can be misleading. The authors contend that agents may achieve low success rates not because they actively refuse injected instructions, but because they simply ignore them. This distinction matters: ignoring is not the same as a deliberate security decision, and it may not hold up under more forceful or better-crafted attacks.
The implication is that ASR alone can overstate an agent's robustness. The paper points toward a need for more nuanced evaluation that separates ignoring from refusing, so that security metrics reflect genuine defenses rather than accidental behavior.