Attack Success Rate (ASR)
Quick Answer
Attack success rate (ASR) is the fraction of adversarial attempts that achieve the attacker's goal against a target system, given a defined success criterion. Despite the shared name, ASR is not one quantity: benchmarks vary the unit of trial, the success judge, the attacker's adaptivity, and the denominator. Numbers from different artifacts are therefore not comparable unless the underlying definition is checked.
Attack Success Rate (ASR)
Attack success rate (ASR) is the fraction of adversarial attempts that achieve the attacker's goal, measured against a defined target and success criterion. It is the load-bearing metric in prompt-injection and agent-security benchmarks, but it has no standard definition. Artifacts vary four parameters: the unit of trial (fixed test case versus free-form human attempt), the success judge (output string match versus automated judge versus human review), the attacker's adaptivity, and the denominator. Change any one and the same name yields a different, irreconcilable number. A reported ASR should always be read alongside those four parameters — a defense showing near-zero ASR against a static attack corpus can face greater than 50% ASR under an adaptive attacker, so a single scalar systematically overstates efficacy.
See also
- Indirect prompt injection — the attack class whose success ASR most often quantifies
- Tool hijacking — a concrete attacker goal encoded in ASR success criteria
- What is indirect prompt injection — the parent explainer for the attack class these benchmarks measure
- Security benchmarks for prompt-injection defenses — the source audit establishing ASR non-comparability