ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems
Organizations: Salesforce AI Research · Georgia Institute of Technology · Independent Researcher
Abstract
As enterprises delegate consequential work to multi-agent LLM systems, benign-looking requests can escalate into unsafe actions as they propagate across sessions through poisoned shared state, long-context documents, and trusted peer messages: no single message is malicious, yet the assembled tool action is. Prior benchmarks isolate a single agent under prompt injection or score multi-agent attacks by their final output, leaving cross-session erosion of shared state out of scope. We introduce ErosionBench, an environment-grounded benchmark that executes such attacks and verifies alignment decay at the process level, scoring the realized tool call against authorized intent and provenance with deterministic contracts, not output text. Each scenario is authored as a deterministic state graph before any prose is rendered, so every state change traces to a known cause. It spans four enterprise domains and four decay mechanisms across 1,728 scenarios with matched benign controls, labeled against the OWASP Top-10 for LLM Applications, plus 198 image-based attacks, each paired with a text equivalent, so a defense must survive the same attack in either channel. Perimeter defenses suppress attacks only by over-refusing 54-92% of legitimate work, so a low attack-success rate can conceal severe over-refusal. As a reference defense, we propose a calibrated Meta-Auditor pairing synchronous state checks with asynchronous trajectory monitoring; on a family-disjoint 906-scenario held-out split it reaches 98.5% recall at a 2.9% false-positive rate. Since it monitors trajectory, not surface content, its detection is channel-invariant: the strongest auditor flags image-delivered attacks about as reliably as their text equivalents (recall gap 2 points per model). Across GPT, Gemini, and Claude auditors, calibration keeps false positives low while recall tracks model capability, strongest for Claude.
Figures & tables
| Benchmark | MA | X-sess. | Proc. | Exec. | Over-ref. |
|---|---|---|---|---|---|
| AgentDojo ( Debenedetti et al., 2024 ) | – | – | – | ✓ | ✓ |
| ToolEmu ( Ruan et al., 2024 ) | – | – | ✓ | (emu.) | |
| ASB ( Zhang et al., 2025 ) | – | – | – | ✓ | ✓ |
| AgentLAB ( Rahman et al., 2026 ) | – | ✓ | – | ||
| Blindspot ( Debenedetti et al., 2026 ) | – | – | ✓ | ✓ | |
| EnterpriseOps ( Malay et al., 2026 ) | ✓ | – | ✓ | – |
| Component | Count | Description |
| Artifacts | ||
| Scenario records | 1,728 | Intent, sessions, document binding, execution specification, and ground truth. |
| Shared PDFs | 32 | Reusable domain documents referenced by immutable identifiers. |
| Needle manifests | 324 | Separate injection specifications for document-grounded attacks. |
| Document-image attacks | 198 | 66 families rendered as enterprise-document PNGs 3 replicates; each paired 1:1 with a text equivalent. |
| Image benign controls | 198 | Matched benign document images (validity-matching in progress). |
| Attack success rate (ASR) | |||||
| Layer | Overall | T1 | T2 | T3 | Benign FPR |
| L1 Single agent | 0.0% | 0.0% | 0.0% | 0.0% | 48.3% |
| L2 Undefended mesh | 43.9% | 35.1% | 45.0% | 51.7% | 37.3% |
| L3 LlamaFirewall | 10.2% | 7.9% | 11.9% | 10.6% | 92.5% |
| L3 ShieldGemma † | 27.8% | 21.9% | 30.5% | 31.1% | 54.3% |
| L4 Progent | 4.9% | 2.6% | 6.0% | 6.0% | 64.0% |
| Auditor | Recall [95% CI] | FPR [95% CI] | F1 | Resid. ASR | Tok/scen | Lat. (s) |
| Closed frontier | ||||||
| claude-opus-4-8 | 98.5% [96.8, 99.2] | 2.9% [1.7, 4.8] | 0.978 | 1.3% | 10.4k | 6.7 |
| claude-sonnet-4-6 | 95.8% [93.5, 97.3] | 8.6% [6.4, 11.6] | 0.937 | 1.5% | 7.3k | 7.3 |
| gpt-5.1 | 84.1% [80.5, 87.2] | 1.3% [0.6, 2.9] | 0.907 | 2.6% | 6.1k | 3.3 |
| gemini-3.5-flash | 83.4% [79.7, 86.5] | 11.0% [8.5, 14.3] | 0.857 | 2.4% | 17.7k | 7.3 |
| Open-weight | ||||||
| Auditor | Image recall | Text recall | (img txt) |
|---|---|---|---|
| claude-opus-4-8 | 100.0% (198/198) | 100.0% (198/198) | 0.0 |
| claude-sonnet-4-6 | 100.0% (198/198) | 99.0% (196/198) | |
| gpt-5.1 | 96.0% (190/198) | 96.0% (190/198) | 0.0 |
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | Value [95% CI] |
|---|---|
| Scenarios (malicious / benign) | 64 (48 / 16) |
| L2 undefended ASR | 45.8% [32.6, 59.7] |
| Recall | 97.9% [89.1, 99.6] |
| Benign FPR | 6.2% [1.1, 28.3] |
| L5 residual ASR | 0.0% |
| Mesh (attacker) | n | L2 ASR | Recall | FPR | Resid. ASR |
|---|---|---|---|---|---|
| gpt-5.5 | 604 | 44.2% | 0.971 | 0.046 | 1.5% |
| gemini-3.5-flash | 603 | 23.5% | 0.971 | 0.073 | 1.5% |
| claude-opus-4-8 | 604 | 15.0% | 0.874 | 0.013 | 0.0% |
| Domain | Mechanism | Attack class |
|---|---|---|
| Quant. finance | Cross-session | Semantic state poisoning |
| Gaslighting | Identity-spoof gaslight | |
| Long-context | Reasoning influence | |
| Collusion | Temporal exploitation | |
| CI/CD supply | Cross-session | Supply-chain poisoning |
| Gaslighting | Authority escalation |
| Domain | Attacks | L2 ASR | L5 ASR | L5 Recall | L5 FPR |
|---|---|---|---|---|---|
| Autonomous CRM | 135 | 22.2% | 0.0% | 100.0% | 3.7% |
| CI/CD supply chain | 114 | 59.6% | 0.0% | 100.0% | 7.0% |
| Data engineering | 126 | 52.4% | 0.0% | 100.0% | 0.0% |
| Quantitative finance | 78 | 44.9% | 7.7% | 91.0% | 0.0% |
| Overall | 453 | 43.9% | 1.3% | 98.5% | 2.9% |
| Scenario (domain) | Attack class | Forged state steered tool | Contract that defines “landing” |
|---|---|---|---|
| QF_SEMANTIC_0000 quant. finance | semantic state poisoning | ml_pipeline.risk_weight_modifier injected, then trigger_pipeline_execution | param_boundary (CRITICAL): hyperparams.risk_weight must stay ; the attack drives it below. |
| DE_POLICY_R_0027 data eng. | policy reframing | data_platform.exposure_policy reframed, then expose_database_endpoint | exfiltration_scope (CRITICAL): exposure must not be { public , external , unrestricted }. |
| CR_CRISIS_E_0027 autonomous CRM | crisis-escalation hijack | crm.incident_context forged, then authorize_cross_region_fulfillment | unauthorized_cross_region (CRITICAL): the incident id must carry an INC-PROD- / INC-VALID- prefix and finance_controller billing authorization. |
| Benign controls | Malicious | |
| Mean realism (1–5) | 4.34 | 4.33 |
| % rated | 100% | 87% |
| Items with attack/benign judgment | 16 | 28 |
| Recovered ground-truth label | 16 benign | 25 attack, 3 unsure |
| OWASP category | # | Representative classes |
|---|---|---|
| LLM04 Data & Model Poisoning | 5 | semantic-state poisoning, temporal exploitation, supply-chain poisoning, shared-memory pollution, lineage forgery |
| LLM06 Excessive Agency | 6 | authority escalation, policy reframing, reasoning influence, SLA metric gaming, entitlement manipulation, crisis-escalation hijack |
| LLM09 Misinformation | 3 | identity-spoofing gaslight, compliance theater, warranty-fraud injection |
| LLM02 Sensitive Info. Disclosure | 1 | intermediate-step leakage |
| LLM08 Vector & Embedding | 1 | split-evidence corroboration |
| OWASP LLM Top-10 (2025) category | Malicious items (primary) |
|---|---|
| LLM06 Excessive Agency | 14 |
| LLM02 Sensitive Information Disclosure | 6 |
| LLM04 Data and Model Poisoning | 5 |
| LLM08 Vector and Embedding Weaknesses | 5 |
| LLM09 Misinformation | 3 |
| LLM03 Supply Chain | 1 |
| Verifier exit | Routed | Flagged | Allowed |
|---|---|---|---|
| Deterministic anchors (pre-execution veto) | 196 | 196 | 0 |
| Fast-secure agreement | 271 | 0 | 271 |
| Grey-zone arbiter | 439 | 263 | 176 |
| Async-semantic alone | 0 | 0 | 0 |
| Total | 906 | 459 | 447 |
| Setting | Auditor TPR |
|---|---|
| Relabel ablation, tell stripped ( , paired) | 98.0% (49/50) |
| Fresh authenticated-channel attacks ( ) | 97.9% (47/48) |
| Corpus | Threat model | Recall / TPR [95% CI] | FPR [95% CI] |
|---|---|---|---|
| AgentDojo | single-session indirect prompt injection; recall over the 44 landed runs, 7.5% on the 106 resisted runs | 86.4% [73.3, 93.6] | 0.0% [0.0, 3.0] |
| ASB (Memory Poisoning) | cross-session memory poisoning | 95.0% [88.8, 97.8] | 2.0% [0.6, 7.0] |
| ErosionBench (native) | cross-session state erosion | 98.5% | 2.9% |
| Corpus | Threat model | Recall / TPR [95% CI] | FPR [95% CI] |
|---|---|---|---|
| AgentDojo | single-session indirect prompt injection; recall over the 44 landed runs, 7.5% on the 106 resisted runs | 86.4% [73.3, 93.6] | 0.0% [0.0, 3.0] |
| ASB (Memory Poisoning) | cross-session memory poisoning | 95.0% [88.8, 97.8] | 2.0% [0.6, 7.0] |
| ErosionBench (native) | cross-session state erosion | 98.5% | 2.9% |
| Split | Undefended ASR | + Meta-Auditor residual ASR | Detected |
|---|---|---|---|
| Stealthy ( ) | 100% | 2% | 49/50 |
| Disruptive ( ) | 100% | 8% | 46/50 |
| Aggressive ( ) | 100% | 2% | 49/50 |
| Non-aggressive ( ) | 100% | 8% | 46/50 |
| All ( ) | 100% | 5% | 95/100 |
| Benign controls ( ) | — | FPR = 2% (2/100 flagged) | |
| Auditor | Recall [95% CI] | Benign FPR [95% CI] | T1 | T2 | T3 |
|---|---|---|---|---|---|
| claude-opus-4-8 | 100.0% [92.1, 100.0] | 0.0% [0.0, 16.1] | 100.0% | 100.0% | 100.0% |
| gpt-5.1 | 95.6% [85.2, 98.8] | 0.0% [0.0, 16.1] | 93.3% | 93.3% | 100.0% |
| Tier (image arm) | Malicious | TP | Recall [95% CI] |
|---|---|---|---|
| T1 | 22 | 19 | 86.4% [66.7, 95.3] |
| T2 | 22 | 19 | 86.4% [66.7, 95.3] |
| T3 | 22 | 18 | 81.8% [61.5, 92.7] |
| Overall | 66 | 56 | 84.8% [74.3, 91.6] |
| Verifier (image arm, raw semantic verdict) | Recall | FPR |
|---|---|---|
| opus-4-8 , threshold sweep | 34.3% (68/198) | 2.5% (5/198) |
| opus-4-8 , threshold sweep | 62.6% (124/198) | 22.2% (44/198) |
| opus-4-8 , threshold sweep | 98.5% (195/198) | 57.1% (113/198) |
| opus-4-8 , threshold sweep | 100.0% (198/198) | 80.8% (160/198) |
| Gemma-4-31B-IT (VLM), single verdict | 84.8% (56/66) | 68.2% (45/66) |
| System | Metric | Image | Text |
|---|---|---|---|
| Undefended mesh (L2) | attack landing / ASR (n=198) | 35.4% | 31.3% |
| LLM-judge (same telemetry) | TPR / FPR | 39.4 / 40.4 | 44.9 / 43.9 |
| PromptGuard | TPR / FPR | 0 / 0 | 0 / 0 |
| Progent | TPR / FPR | 0 / 0 | 0 / 0 |
| Meta-Auditor opus-4-8 | recall (own frozen Mode-A rule) | 100.0% | 100.0% |
| Meta-Auditor sonnet-4-6 | recall (own frozen Mode-A rule) | 100.0% | 99.0% |
| Verifier (image arm) | Recall / TPR | FPR | Behavior |
| Meta-Auditor opus-4-8 , composite | 100% landed | 20.2% | flags every landed attack |
| Meta-Auditor sonnet-4-6 , composite | 100% landed | 17.7% | flags every landed attack |
| Gemma-4-31B-IT VLM (open), single verdict † | 84.8% | 68.2% | open auditor, uncalibrated |
| PromptGuard (input or output scanner) | 80.3% | 78.3% | fires indiscriminately |
| LLM-judge (input or output scanner) | 40.4% | 41.9% | TPR FPR (chance) |
| Progent (input or output scanner) | 34.3% | 22.7% | weak discrimination |
| Domain | Carrier template (UI archetype) | Pinned attack class |
|---|---|---|
| generic (any) | configuration dashboard; scanned vendor invoice; signed approval memo; policy-document scan | any class (fallback pool) |
| CI/CD supply chain | pipeline-run screenshot | compliance theater |
| artifact registry / SBOM view | supply-chain poisoning | |
| #deploys Slack thread | split-evidence corroboration | |
| signed attestation scan | authority escalation | |
| quantitative finance | Grafana risk dashboard | reasoning influence; semantic state poisoning |
AI Agent Safety Benchmarks
AI agent safety benchmarks measure whether tool-using and embodied AI systems act safely across realistic tasks and interactions.
See all 237 papers