Tool-using agents act on email, files, and payments, so an instruction hidden in a web page or a shared document can make an agent send data or money to an attacker. Many runtime defenses put a language model on the decision that lets an action through, so the decision depends on a model that may read the attacker's text and adds model calls to every step. Provenance defenses keep the model off the final check but need a model to plan or route each task first. Where an action goes can instead be read from the tools' typed returns: an email's sender or a payment's account arrives in a typed field, while injected instructions arrive as free text. SPILLWAY turns this observation into a deterministic floor with no per-task plan. The check admits a call when each target value that appears in tool output but not in the user's task was returned by a read-only tool in a typed field or is covered by a fixed delegation rule. A deterministic sanitizer removes instruction-like regions its fixed rules match before the model reads them and retains the removed spans as evidence, and a local judge may restore a refused call, never one whose target lies in the removed text. Across three backbones, and seven task suites on one of them, no standard templated injection redirects an action to an attacker's target under SPILLWAY, at the highest observed mean utility and composite scores (NRP, ERS) of the defenses we run under that attack. A forged chat template also produces a raw zero, and mean in-scope attack success under a released reinforcement-learning attacker is .005. SPILLWAY reaches this floor with no defense-triggered backbone re-run and almost no added model call. As personal AI agents act on users' email, files, and payments, SPILLWAY gives them an auditable refusal floor on where their actions go that keeps them useful and holds across backbones.
Figures & tables
Figure 2: SPILLWAY’s two boundaries. (A) The egress floor admits a call whose targets are named in T , typed in the index I , or covered by a delegation rule (Alg. 1 ). Only a refused call reaches the fail-closed two-view judge. (B) The sanitizer (three rule groups shown) removes matched instruction-like regions before the model reads them and keeps them as spans S . A refused target inside S never reaches the judge.
Undefended
SPILLWAY
MELON
Progent
DRIFT
ROPE
Spotlighting
PI detector
Suite
Backbone
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
BU/UAU
ASR
Banking
gpt-4o-mini
.562/.424
.556
.562/.444
.000
.438/.306
.049
.375/.299
.097
.438/.431
.111
.438/.431
.000
.500/.431
.549
.500/.375
.118
qwen3-coder:30b
.375/.486
.299
.500/.438
.000
.375/.299
.125
.312/.354
.056
.688/.542
.014
.375/.375
.000
.375/.410
.382
.500/.424
.097
qwen3:8b
.500/.403
.181
.500/.438
.000
.500/.403
.215
.375/.361
.000
.562/.410
.028
.438/.438
.000
.562/.431
.188
.438/.340
.083
Slack
Table 1: Standard attack on AgentDojo ( Debenedetti et al., 2024 ) (three backbones × four suites, v1.2.2, full grid), per cell: benign / under-attack utility (BU/UAU ↑ ) and in-scope attack success (ASR ↓ ). The last row is the twelve-cell mean. In-scope ASR counts injections that redirect an effect to an attacker-named identity. Content-manipulation tasks are excluded (no target check bounds them), and raw and executed levels are in App. A . ROPE’s router is the cell’s own backbone (Table 8 also gives its authors’ router). Metrics defined in Sec. 4 .
12-cell mean
Composite
Cells of 12
Method
BU ↑
UAU ↑
ASR ↓
NRP (BU) ↑
NRP (UAU) ↑
ERS ↑
raw hit
filter-zeroed
in-scope leak
Undefended
.540
.417
.305
.375
.290
.608
12
0
12
SPILLWAY
.539
.489
.000
.539
.489
.700
0
0
0
MELON
.526
.314
.094
.477
.284
.666
11
1
10
Progent
.437
.371
.031
.423
.359
.602
11
0
11
DRIFT
.460
.453
.016
.453
.446
.627
7
0
7
Table 2: AgentDojo breadth-grid summary (Table 1 ): twelve-cell means of BU/UAU/ASR, the composites NRP ↑ /ERS ↑ (Sec. 4 ) and NRP with UAU as in MSB ( Zhang et al., 2026 ) , and the cells with a raw hit, zeroed by the content-manipulation filter, and leaking in-scope.
Figure 3: Per-episode cost (qwen3:8b benign, four suites). Bars: added model invocations per episode over the undefended agent, backbone plus any defense model, machine-independent. For SPILLWAY the defense model is its local judge. Added wall-clock (serving-stack dependent) in gray. MELON, Progent, and DRIFT re-invoke the backbone several times. SPILLWAY adds almost none.
Figure 4: Delegated-routing grid. AgentDyn ( Li et al., 2026b ) and AgentDojo suite v1.2.2, gpt-4o-mini, standard attack, full grid. ROPE ( Ma et al., 2026b ) uses its authors’ Claude Opus 4.8 router and cached scopes (Workspace scopes produced once by the same router).
Arm
Banking
Slack
Travel
Workspace
Floor
.333 / .000
.181 / .000
.243 / .000
.445 / .000
Floor + judge
.375 / .028
.190 / .000
.243 / .000
.439 / .002
Floor + sanitizer
.375 / .000
.286 / .000
.343 / .000
.412 / .000
SPILLWAY (all)
.438 / .000
.286 / .000
.343 / .000
.414 / .000
SPILLWAY, no span veto
.438 / .000
.286 / .000
.343 / .000
.418 / .000
Table 3: Component ablation on qwen3:8b ( AgentDojo v1.2.2, 949 pairs per arm): in-scope under-attack utility (UAU ↑ ) / attack success (ASR ↓ , executed effect). Delegation rules are on in every arm, and object cases (Sec. 3.1 ) reach their judge in every row. The judge row recovers identity-target refusals. SPILLWAY in bold. The last row removes only the span veto.
ChatInject (qwen3:8b)
AutoDojo (gpt-4o-mini)
Workspace
Slack
Composite
Banking
Slack
Travel
Composite
Method
UAU
ASR
UAU
ASR
NRP
ERS
UAU
ASR
UAU
ASR
UAU
ASR
NRP
ERS
Undefended
.052
.245
.095
.648
.269
.496
.444
.354
.524
.571
.329
.329
.342
.574
SPILLWAY
.396
.000
.286
.000
.451
.621
.479
.000
.562
.000
.550
.000
.625
.765
MELON
.084
.020
.095
.000
.494
.660
.340
.021
.286
.000
.293
.000
.564
.716
Progent ‡
.161
.005
.057
.010
.305
.457
.306
.104
.467
.124
.414
.121
.468
.654
Table 4: Stress tests (suite v1.2.2): in-scope under-attack utility (UAU ↑ ) / attack success (ASR ↓ ). Left: ChatInject ( Chang et al., 2026 ) , qwen3:8b, 560 Workspace and 105 Slack pairs. Right: AutoDojo ( Ma et al., 2026a ) optimized against Progent, gpt-4o-mini, 144 Banking, 105 Slack, 140 Travel pairs. Trace, executed, and in-scope levels in Table 6 . NRP ↑ /ERS ↑ (Sec. 4 ) use each panel’s benign utility (Table 1 ). ROPE’s ChatInject row uses a benign run in that row’s setting (Appendix A ). SPILLWAY in bold. Its AutoDojo row uses rules written on four failing pairs (held-out sets in Appendix A ). ‡ Released code adapted to our agent and suite (App. B ). † authors’ planner and validator.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Cell
Undefended
SPILLWAY
MELON
Progent
DRIFT
ROPE
Spotlighting
PI detector
Banking
gpt-4o-mini
.556
.000
.049
.097
.111
.000
.549
.118
qwen3-coder:30b
.299
.000
.125
.056
.014
.000
.382
.097
qwen3:8b
.181
.000
.215
.000
.028
.000
.188
.083
Slack
gpt-4o-mini
.629
.000
.000
.076
.000
.019
.562
.086
Appendix
Table 5: AgentDojo breadth grid (suite v1.2.2, 949 attacked pairs per backbone: 144 Banking, 105 Slack, 140 Travel, 560 Workspace): raw / in-scope ASR per backbone, suite, and method. One value when the two agree.
Banking
Slack
Travel
Method
trace
exec.
in-scope
trace
exec.
in-scope
trace
exec.
in-scope
Undefended
.354
.354
.354
.571
.571
.571
.364
.364
.329
SPILLWAY
.000
.000
.000
.000
.000
.000
.000
.000
.000
MELON
.021
.021
.021
.000
.000
.000
.043
.043
.000
Progent
.104
.104
.104
.210
.124
.124
.150
.150
.121
DRIFT
.090
.090
.090
.010
.010
.010
.086
.086
.086
Appendix
Table 6: AutoDojo on gpt-4o-mini (144 Banking, 105 Slack, 140 Travel pairs): trace-level, executed (exec.), and in-scope ASR. DRIFT’s three levels coincide. ROPE’s scorer reports executed and in-scope ASR, not a separate trace ( − ).
Method
UAU ↑
in-scope ASR ↓
Undefended
.128
.568
SPILLWAY
.257
.005
MELON
.114
.061
Progent ‡
.166
.096
DRIFT †
.428
.064
ROPE ‡
.173
.007
Appendix
Table 7: Frontier RL attacker. PISmith ( Yin et al., 2026 ) , released by its authors, replayed open-loop with the first of its ten pair-conditioned variants per injection (ASR@1, gpt-4o-mini, v1.2.2, full grid): in-scope UAU ↑ /ASR ↓ , macro over the four suites. SPILLWAY (bold) holds the attack to the floor at the most utility there. Only ROPE comes close. ‡ Released code adapted to our agent and suite (App. B ). † authors’ planner and validator.
Suite
SPILLWAY
ROPE (Opus 4.8 router)
ROPE (gpt-4o-mini router)
Banking
.444 / .000
.438 / .000
.431 / .000
Slack
.562 / .000
.514 / .048
.343 / .019
Travel
.579 / .000
.500 / .000
.350 / .000
Workspace
.848 / .000
.670 / .000
.539 / .000
GitHub
.600 / .000
.500 / .000
.383 / .000
DailyLife
.435 / .000
.295 / .010
.150 / .000
Appendix
Table 8: Delegated-routing grid ( AgentDojo v1.2.2 above the rule, AgentDyn below), gpt-4o-mini: in-scope under-attack utility (UAU ↑ ) / attack success (ASR ↓ ), one run per suite (144, 105, 140, 560, 180, 200, 180 pairs from top). The mean is unweighted over suites. The last two rows give the composites NRP ↑ /ERS ↑ (Sec. 4 ), macro-averaged over the seven suites. The AgentDyn suites have no benign run, so NRP and ERS here use UAU for BU, as MSB ( Zhang et al., 2026 ) does for NRP (NRP = UAU ×(1− ASR ) ).