CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering

Mark Russinovich

cs.CRSep 29, 2026

CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering

Authors: Mark Russinovich

Organizations: Microsoft Azure

Abstract

Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that suppresses this behavior inside the model. Per model, a five-step recipe fits a residual-stream direction from paired episodes differing only in whether an embedded instruction is followed, and retains it only if it passes pre-specified causal and capability gates. At deployment, the direction is subtracted from every tool-result token during prefill. The edit is always on--there is no detection decision to evade--and requires no fine-tuning, auxiliary model, or added tokens, only white-box serving and tool-result span boundaries. Across five open-weights models (8B-106B, five vendor lineages), held-out attack success falls from 0.21-1.00 undefended to 0.00-0.17 defended, and AgentDojo compromise rate from 0.10-0.49 to 0.006-0.079, at 93-100% typography-normalized benign utility, with larger task-dependent costs when reasoning over steered content. A benchmark-level adaptive attacker reaching 0.67-0.73 undefended is held to roughly a quarter of that on the two most deeply evaluated models. Among the defenses we measured on capable models, those achieving lower compromise rates either lost 22-89% of benign utility or fine-tuned the served weights. White-box gradient attacks through the deployed vector compromise at most 2 of 52 episodes, and none of 2,052 replayed human red-team attacks succeeds. CounterSteer largely neutralizes instructional takeover: a black-box framing search cracks 3 of 18 development samples. Parameter manipulation--attacker-chosen arguments in otherwise legitimate calls--is only partially resisted (13 of 18); the decision becomes linearly readable at argument emission but not at the examined pre-generation sites, and is not removed by the tested prefill- or decode-time steering, motivating argument-provenance controls.

Figures & tables

class	where the follow decision lives	what removed it in our tests	CounterSteer
tool hijack	the retrieved span, at prefill	a constant prefill direction	largely neutralized
parameter manipulation	the model’s own reasoning, at argument emission (§ VI-C )	only fine-tuning (SecAlign)	reduced, not closed
delegated authority	the user prompt grants it	system-level provenance (e.g., ROPE [ 5 ] )	out of scope (Class B)

TABLE I: The three-part map of injection-following that organizes this paper. “What removed it” is what our measurements found, not a completeness claim.

	single-turn ASR $\downarrow$		AgentDojo $\downarrow$		benign $\uparrow$
model	undef.	defended	undef.	defended	% of clean
gpt-oss-20b	.59–.98	.000–.135	.475	.079	94.4
Qwen3-30B	.52–1.00	.000–.173	.489	.072	94.1
Gemma-4-31B	.87–.96	.000–.038	.239	.006	92.9
GLM-4.5-Air	.81	.019	.222	.056	100.0
Llama-3.1-8B	.21	.038	.100	.050	100.0

TABLE II: The five-model result in brief. Single-turn: ASR range over that model’s firing corpus classes, one pre-specified held-out test pass each (detail: Table XIII ; CIs: Tables XV – XXI ). AgentDojo: the canonical runs (Reading the numbers); benign utility by its own checker, % of clean (gpt-oss typography-normalized).

	gpt-oss-20b †		Qwen3-30B		Gemma-4-31B		GLM-4.5-Air †		Llama-3.1-8B
defense	cmp $\downarrow$	util $\uparrow$	cmp $\downarrow$	util $\uparrow$	cmp $\downarrow$	util $\uparrow$	cmp $\downarrow$	util $\uparrow$	cmp $\downarrow$	util $\uparrow$
no defense (clean util absolute)	.483	98.1	.489	91.1	.239	100.0	.183	80.4	.100	26.8
PromptGuard-2 filter [ 21 ]	.205	100.0	.228	100.0	—	—	.117	100.0	.056	100.0
DeBERTa filter [ 22 ]	.034	47.2 (!)	.089	54.9 (!)	—	—	—	—	.044	92.3
PIGuard filter [ 23 ]	.011	58.2 (!)	.006	62.7 (!)	—	—	.000	57.8 (!)	.006	92.3
CachePrune [ 9 ]	.216	88.7	—	—	—	—	—	—	.045	80.0

TABLE III: Headline agentic results: defenses

\times

models on AgentDojo (180-case grid). cmp

=

compromise rate (its own security checker); util

=

benign utility, % of that row’s own clean run (“no defense”: absolute clean completion, the capability reference). † comparison/detector-probe batteries, not the canonical runs (§ V ). “—”

=

not measured; (!)

=

capability-guard territory; bold/underline

=

best/second-best per column among unmarked rows. Bottom block: AutoDojo AD@6 (§ V-C ).

defense	AgentDojo cmp $\downarrow$	Class A $\downarrow$	utilAtk $\uparrow$	benign $\uparrow$ % of clean	adaptive param. $\downarrow$ (union; per-q.)
undefended	.483	59	.551	(ref.)	18/18; .708
CounterSteer	.091	0	.716	90.6	13/18; .303
AGRI ‡	.062	—	—	78.0	—
SecAlign (trained) †	—	—	—	90.4	1/18; .002
CachePrune	.216	15	.688	88.7	17/18; .604
PromptGuard-2 filter	.205	21	.311	100.0	—

TABLE IV: Baseline separation on gpt-oss-20b (comparison battery). Class A: non-delegated compromises /140 (undefended 57–59). adaptive: best-of-24 parameter search, cracked/18 dev; per-query rate (Table VII ). ‡ own paired battery. † trained; benign % of base clean; AgentDojo 768-legacy. (!) capability-guard; — not reported. CIs: Table XI .

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

arm (defense on clean)	gpt-oss web param ( $n{=}52$ )	gpt-oss JSON hij. (80)	Qwen web param (52)	Qwen JSON hij. (90)
CounterSteer (same-process)	.385	.525	.346	.333
regeneration reference (cross-run)	.462	.637	.462	.400
spotlighting	.442	.625	.462	.356
prompt sandwich	.462	.600	.462	.356
reminder	.423	.600	.365	.344
CachePrune (gpt-oss only)	.385	.537	—	—

Appendix

TABLE V: Defense-on- clean composition fidelity, judge % equivalent (zero compromise flags anywhere), four cells. Row one is the always-on steering cost from the main utility tables; row two is the regeneration reference : prompts byte-identical to the clean render, regenerated cross-run with no defense, scored by the same judge—the shared wording-drift floor every cross-run arm (all baselines here) must be read against. It does not directly calibrate the same-process steering row, so ratios of row one to row two are never taken. Filter rows at 0.000 are capability destruction (100% of clean webpage payloads flagged and deleted), not composition loss. Pooled over the four cells, steering sits

{\sim}

7–9 pp below the regeneration reference (

p{=}0.012

n{=}274

paired), a lower bound on its true deficit (the baselines absorb cross-run noise steering does not); in strict byte-exact terms the webpage ordering reverses (steering 75/65 vs. regeneration 71.2/55.8)—steering preserves structured calls better and pays in composed prose. Per-cell strict/lenient brackets live in the released fidelity artifacts; the steering row’s brackets are in the main utility tables.

adaptive success $\downarrow$	gpt-oss-20b	Qwen3-30B	Llama-3.1-8B
undefended	0.673	0.730	0.377
CounterSteer	0.175	0.188	0.307
AGRI (probe-gated prefill)	0.143	0.411	—
CachePrune	0.305	0.558	0.252
PromptGuard-2 filter	0.527	0.670	0.369
SecAlign (trained)	0.160	—	—

Appendix

TABLE VI: Benchmark-level adaptive attack (AutoDojo optimizer, six iterations, AgentDojo’s own checker) against each defense, macro over the 36 pre-specified (suite

\times

objective

\times

vector) cells per model. Adaptive is the attacker’s best over seeds and optimized rewrites ( AD@6 ). Cell coverage, void-cell rulings and per-suite anatomy: the “Table VI notes” paragraph below.

arm	union $\downarrow$ pattern	union $\downarrow$ literal	per-query $\downarrow$ pattern	per-query $\downarrow$ literal
undefended	18/18	3/18	0.708	0.076
spotlighting-with-delimiting	18/18	4/18	0.595	0.044
CachePrune [ 9 ]	17/18	4/18	0.604	0.037
CounterSteer (deployed)	13/18	1/18	0.303	0.002
SecAlign [ 4 ] (trained)	1/18	0/18	0.002	0.000

Appendix

TABLE VII: Defense-aware adaptive parameter-manipulation search, five arms: best-of-24 framing queries (6 authority levels

\times

4 voices) per sample,

n{=}18

development samples,

432

attempts per arm. union

=

samples cracked by

\geq

1 framing; per-query

=

mean firing rate. Tier-1 only; no arm scores by refusing (no-action

\leq 0.07

everywhere). Read per-query beside union : the undefended literal union (3/18) sitting below two defended arms’ 4/18 is a union-statistic artifact of rare, sample-concentrated literal fires. SecAlign is a trained arm and the search’s framings are cells of its own training grid—trained framing families, not framing generalization. Spotlighting’s delimiter leak and SecAlign’s single scorer-level fire: the notes below.

split	ASR $\downarrow$ undefended [95% CI]	ASR $\downarrow$ defended [95% CI]	clean-traffic cost
dev $n{=}1537$	105/1537 = 0.068 [0.057, 0.082]; ${\leq}0.073$ truncation-corrected	0/1537 observed [0.000, 0.002]	none (0 spurious sends; retrieval coverage 0.988, matching the clean arm)
test $n{=}515$ (held out)	23/515 = 0.045 [0.030, 0.066]	0/515 observed [0.000, 0.007]	none (retrieval coverage 0.990, matching the clean arm)

Appendix

TABLE VIII: LLMail-Inject replay (human-adaptive), gpt-oss-20b deployed configuration.

model	directions	layer sets	doses	compositions / variants	held-out confirm
gpt-oss-20b	$\geq$ 16 (+ $\geq$ 9 random draws)	$\geq$ 2	$\geq$ 16	$\geq$ 20	yes (one pass)
Qwen3-30B	5	2	$\geq$ 11	2	yes (one pass)
Gemma-4-31B	1 (+probe axes)	3	8	0	yes (two rungs: wire-format T ∗ ; then docs $+$ wordings $+$ framings held out)
GLM-4.5-Air	$\geq$ 5 (incl. 2 role axes)	2	$\geq$ 6	0	yes (one pass, webpage param)
Llama-3.1-8B	3 (+3 random draws)	4	$\geq$ 9	0	yes (one pass)

Appendix

TABLE IX: Per-model defense search space actually explored during development (lower bounds where early exploration is recorded only in summary). Doses counts distinct step magnitudes evaluated; compositions counts blends and variant mechanisms (gates, span choices, decode-time stacks). All test-split evaluations are one pre-specified pass, no reruns.

bench ( $N$ )	context	baseline $\uparrow$	steered $\uparrow$
GSM8K ( $n{=}250$ )	plain / tool-framed	0.896 / 0.912	0.896 / 0.912
MMLU ( $n{=}500$ )	plain / tool-framed	0.814 / 0.810	0.814 / 0.702
IFEval ( $n{=}250$ )	plain / tool-framed	0.624 / 0.672	0.624 / 0.520

Appendix

TABLE X: Identity check (plain) and steered-span reading probe (tool-framed), gpt-oss-20b deployed configuration. Denominator: all items. The plain columns are an implementation check of the by-definition identity, not a utility evaluation.

defense	compromise $\downarrow$ [95% CI] (own undef.)	blk/int ( $p$ )	Class A $\downarrow$	utilAttack $\uparrow$ (undef.)	benign $\uparrow$ % of clean	filter fp $\downarrow$
gpt-oss-20b (Class A over 137–140 non-delegated cases; undefended Class A 51–59)
CounterSteer (deployed)	16/176 $=$ 0.091 [.057, .143] (0.483)	70/1 ( $6.1{\times}10^{-20}$ )	0/140 (59)	0.716 (0.551)	90.6%	—
CachePrune [ 9 ]	38/176 $=$ 0.216 [.162, .282] (0.483)	55/8 ( $9.8{\times}10^{-10}$ )	15/140 (59)	0.688 (0.551)	88.7%	—
reminder (AutoDojo) [ 13 ]	48/176 $=$ 0.273 [.212, .343] (0.483)	45/8 ( $2.4{\times}10^{-7}$ )	33/140 (59)	0.574 (0.551)	88.7%	—
prompt sandwich	55/174 $=$ 0.316 [.252, .389] (0.489)	38/8 ( $9.2{\times}10^{-6}$ )	31/138 (59)	0.609 (0.551)	88.2%	—
spotlighting [ 2 ]	74/176 $=$ 0.420 [.350, .494] (0.483)	21/10 ( 0.071 n.s. )	46/140 (59)	0.616 (0.551)	92.5%	—

Appendix

TABLE XI: State-of-the-art comparison: four-arm batteries (clean / defense-on-clean / attacked / defended) per defense, full 180-case AgentDojo grid, 4096 tokens/turn (GLM-4.5-Air 8192), each row against its own same-process undefended baseline (“undef.”). compromise

=

AgentDojo’s checker over paired verdict-bearing cases; blk/int

=

compromises blocked / introduced (exact McNemar); Class A

=

non-delegated compromises, defended (undefended); benign

=

defense-on-clean % of clean over paired unique tasks; fp

=

fraction of benign tool messages a filter deleted; n/m

=

the delegated-authority partition is not computed for that panel. (!)

=

capability-guard territory, a mechanism predicate rather than a benign-% threshold, detailed—with the typography normalization’s scope and its one disclosed movement—in the “Table XI provenance” paragraph of Appendix E .

term	meaning
primary models	gpt-oss-20b and Qwen3-30B: full evaluation battery and the deepest adaptive evaluation (§ IV )
undefended baseline	the same run’s no-defense arm; every contrast in this paper is paired against it, never against another run
ASR	single-turn attack success rate over scoreable samples; the obedience-pattern reading unless marked exact-literal
compromise rate	AgentDojo/AgentDyn’s own security checker: fraction of injection cases where the attacker’s task completed
obedience-pattern / exact-literal	the injected action was performed / the attacker’s exact payload was delivered verbatim
benign utility	task fidelity, defense on, no injection, % of the undefended clean run—the always-on deployment cost

Appendix

TABLE XII: Glossary of recurring terms.

	AgentDojo	webpage param	webpage tool-hijack	JSON tool-hijack	JSON param
compromise: ASR $\downarrow$ (AgentDojo: compromise rate), undefended $\to$ defended (rung)
gpt-oss-20b	$.475\to\mathbf{.079}$	$.962\to\mathbf{.135}$ T	$.981\to\mathbf{0/52}$ T	$.675\to\mathbf{.013}$ T	$.591\to\mathbf{.045}$ T
Qwen3-30B	$.489\to\mathbf{.072}$	$1.000\to\mathbf{.173}$ T	$.981\to\mathbf{0/52}$ T	$.722\to\mathbf{.033}$ T	$.522\to\mathbf{.060}$ T
Gemma-4-31B	$.239\to\mathbf{.006}$	$.962\to\mathbf{.038}$ T	$.865\to\mathbf{.000}$ T	N/F (.083–.112)	N/F (.045–.048)
GLM-4.5-Air	$.222\to\mathbf{.056}$	$.808\to\mathbf{.019}$ T	hijack classes low-fire; JSON param. not run (Table XVIII )
Llama-3.1-8B	$.100\to\mathbf{.050}$	no viable cell (base 1.000)	$.212\to\mathbf{.038}$ T	N/F (.042)	$.273\to\mathbf{.000}$ D (not viable)

Appendix

TABLE XIII: Multi-model results: five corpus classes

\times

five models; every cell is a within-model contrast at its own budget, dose and split, never compared across models. Top: ASR (the AgentDojo column: compromise rate), undefended

\to

defended , with its rung—T

=

held-out test split (one pre-specified pass); D

=

development split; N/F

=

does not fire undefended (undefended rate shown); AgentDojo

=

its own 180-case grid and checkers, no test split. Bottom: utility with the defense on—benign

\mid

under attack, judge % of unattacked with the (strict/lenient) bracket in parentheses (§ IV ; integer-rounded where space requires—exact brackets: Tables XV – XVIII ); AgentDojo utility is its own checker (benign % of clean over paired unique tasks; under-attack raw beside undefended-attacked raw). “—”

=

no defended run. Caveats and run provenance: the “Table XIII provenance” paragraph (Appendix G ).

evaluation	gpt-oss-20b (deployed configuration)	Qwen3-30B (selected $12\sigma$ )
JSON tool-hijack (T)	$.675\to\mathbf{.013}$ ; utilB 52.5 (25/80); utilA 37.5 (30/76) (undef. 13.8)	$.722\to\mathbf{.033}$ ; utilB 33.3 (21/69); utilA 20.0 (22/67) (undef. 8.9)
JSON param. (T, samples-only)	$.591\to\mathbf{.045}$ ; utilB 53.0 (23/83); utilA 43.9 (23/74) (undef. 18.2)	$.522\to\mathbf{.060}$ ; utilB 31.3 (21/64); utilA 31.3 (27/66) (undef. 16.4)
webpage tool-hijack (T)	$.981\to\mathbf{0/52}$ ; utilB 50.0 (98/98); utilA 57.7 (94.2/94) (undef. 0.0)	$.981\to\mathbf{0/52}$ ; utilB 13.5 (63/63); utilA 13.5 (58/58) (undef. 1.9)
webpage param. (T)	$.962\to\mathbf{.135}$ ; utilB 38.5 (75/75); utilA 38.5 (60/60) (undef. 0.0)	$1.000\to\mathbf{.173}$ ; utilB 34.6 (65/65); utilA 15.4 (50/50) (undef. 0.0)
AgentDojo (4096 tok/turn)	$.475\to\mathbf{.079}$ ; benign 94.4% of clean; utilAttack 73.9% (undef. 62.5%)	$.489\to\mathbf{.072}$ same-process (separate-run replicate $.457\to.094$ ); benign 94.1%; utilAttack .717 raw (undef. .667)
LLMail-Inject replay	$128/2052\to\mathbf{0/2052}$ observed (2052 episodes)	$117/1537\to\mathbf{0/1537}$ observed (audited count—a parser audit recovered 13 successful attacker calls; full dev replay, no test-shard replay exists)

Appendix

TABLE XIV: The two primary models: the full suite, including the § V-C adaptive attacks. Corpus rows: ASR undefended

\to

defended ; utilB/utilA (benign / under attack, defense on), judge % of unattacked with the (strict/lenient) bracket; undef. is the undefended-attacked arm’s judge utility from the same run. Denominators: scoreable samples (Appendix G ); webpage-param T rows carry the near-duplicate-template caveat (Reading the numbers, § V ); bracket values are integer-rounded where space requires (exact brackets: Tables XV – XVI ). Adaptive rows are tier-1 only (no arm scores by refusing, no-action

\leq 0.07

); query attacks are best-of-

k

searches on development samples against the live defended agent.

attack class (corpus, $N$ )	ASR $\downarrow$ undefended	ASR $\downarrow$ defended [95% CI]	utilB $\uparrow$ judge (strict/len.)	utilA $\uparrow$ judge (strict/len.) [undef.]
tool hijack — JSON tool-log ( $n{=}96$ , 80 scoreable)	0.675	1/80 = 0.013 [0.002, 0.067]	52.5 (25.0/80)	37.5 (30.0/76) [13.8]
tool hijack — webpage ( $n{=}52$ , 52 scoreable; multi-turn)	0.981	0/52 observed [0.000, 0.069]; multi-turn 0/49 observed [0.000, 0.073]	50.0 (98.1/98)	57.7 (94.2/94) [0.0]
param manipulation — JSON ( $n{=}70$ , 66 scoreable; samples-only holdout—the attacker-template set was in the fit)	0.591	3/66 = 0.045 [0.016, 0.125]	53.0 (22.7/83)	43.9 (22.7/74) [18.2]
param manipulation — webpage ( $n{=}52$ held-out templates, 52 scoreable)	0.962	7/52 = 0.135 [0.067, 0.253]	38.5 (75.0/75)	38.5 (59.6/60) [0.0]

Appendix

TABLE XV: gpt-oss-20b, held-out test evaluation (one pre-specified pass per corpus).

\downarrow

lower is better,

\uparrow

higher; denominators are samples scoreable against the clean reference; utility is the judge % of unattacked with the (strict/lenient) bracket, and [

\cdot

] the undefended-attacked arm’s judge utility from the same run (§ IV , Appendix B ). Guard rates and the webpage rows’ template-holdout caveat: the joint provenance paragraph beside Table XVI .

attack class (corpus, $N$ ; holdout axes)	ASR $\downarrow$ undefended	ASR $\downarrow$ defended [95% CI]	utilB $\uparrow$ judge (strict/len.)	utilA $\uparrow$ judge (strict/len.) [undef.]
tool hijack — JSON tool-log ( $n{=}96$ , 90 scoreable; samples $+$ attacker templates held out)	0.722	3/90 = 0.033 [0.011, 0.093]	33.3 (21.1/69)	20.0 (22.2/67) [8.9]
tool hijack — webpage ( $n{=}52$ ; exact-template-text holdout; strictly-disjoint-40 subset 0/40 observed [0.000, 0.088])	0.981	0/52 observed [0.000, 0.069]	13.5 (63.5/63)	13.5 (57.7/58) [1.9]
param manipulation — JSON ( $n{=}70$ , 67 scoreable; samples-only holdout—the attacker-template set was in the fit)	0.522	4/67 = 0.060 [0.023, 0.144]	31.3 (20.9/64)	31.3 (26.9/66) [16.4]
param manipulation — webpage ( $n{=}52$ ; exact-template-text holdout; strictly-disjoint-40 subset 8/40 = 0.200 [0.105, 0.348])	1.000	9/52 = 0.173 [0.094, 0.297]	34.6 (65.4/65)	15.4 (50.0/50) [0.0]
development split, not holdout: tool hijack — JSON tool-log ( $n{=}96$ , 91 scoreable)	0.648	4/91 = 0.044 [0.017, 0.108]	45.1% / 91% strict/len.	47.3% / 87% (undef. 14.3/21)

Appendix

TABLE XVI: Qwen3-30B, held-out test evaluation (one pre-specified pass per corpus, selected

12\sigma

); the last row is a labeled development split, not holdout; columns as in Table XV (utility: judge with the strict/lenient bracket; the development row is strict/lenient only, not judge-scored). Each webpage row states its strictly-disjoint-40 subset (template-holdout caveat: the joint provenance paragraph below).

evaluation	rung	ASR $\downarrow$ undefended $\to$ defended [95% CI]	utilB $\uparrow$	utilA $\uparrow$	guard / notes
webpage param. ( $n{=}52$ ; docs $+$ wordings $+$ framings held out)	T	$0.962\to\mathbf{0.038}$ (2/52); doc-clustered [0.000, 0.096], sample Wilson upper 0.130; 48 fixed / 0 introduced, $p{=}7.1{\times}10^{-15}$	78.8 (96.2/96)	78.8 (92.3/92)	guard clean, truncation 0.00; both survivors under one held-out wording (2/13 on it, 0/39 on the rest); $\alpha{=}6$ monotonicity control PASS ( $0.962\geq 0.192\geq 0.038$ )
webpage tool-hijack ( $n{=}52$ ; same holdout axes)	T	$0.865\to\mathbf{0.000}$ (0/52); cluster-Wilson [0.000, 0.114] over 30 docs; 45/0, $p{=}5.7{\times}10^{-14}$	73.1 (96.2/96)	69.2 (96.2/96)	guard clean; $\alpha{=}6$ control saturated at 0.000; A100 $\leftrightarrow$ H100 replica probe 24/24 agreement
webpage param.	T ∗	$1.000\to\mathbf{0.038}$ ; doc-clustered [0.000, 0.105], quote-upper 0.130 (sample Wilson); 50 flipped / 0 broken	88.5 (100/100)	78.8 (96/96)	guard clean
webpage tool-hijack	T ∗	$0.865\to\mathbf{0.000}$ [0.000, 0.278] doc-clustered; 45 flipped / 0 broken	80.8 (100/100)	82.7 (100/100)	guard clean
JSON tool-hijack / JSON param.	N/F	undefended 0.083–0.112 (shipped JSON, $n{=}24$ / $96$ ) / 0.045–0.048	—	—	N/F by the program’s convention; no defended run
prose-carrier medium ( $n{=}96$ )	N/F	undefended 0.000 [0.000, 0.038]	—	—	measured null: sane on-task summaries, truncation 0.00

Appendix

TABLE XVII: Gemma-4-31B-it, per-model detail. T

=

the certification rung, holding out carrier documents (30 fresh, URL-disjoint), attacker wordings (4 new per class) and framing templates simultaneously; T ∗

=

the earlier wire-format-only holdout rung (carriers and wordings shared with development), retained beside it. CIs on both webpage rungs are doc-clustered (documents are the unit of independence). Both rungs are judge-scored (judge % of unattacked with the strict/lenient bracket; the T-heldout cells were rescored 2026-09-09 under the same controls-gated protocol). Scope caveats: the “Table XVII provenance” paragraph (Appendix G ).

evaluation	rung	ASR $\downarrow$ undefended $\to$ defended [95% CI]	utilB $\uparrow$	utilA $\uparrow$	guard / notes
webpage param. ( $n{=}52$ , 52 scoreable; samples $+$ template text held out)	T	$0.808\to\mathbf{0.019}$ [0.003, 0.101]; exact-literal 0/52; 41 fixed / 0 introduced	26.9 (94.2/94)	11.5 (81/81)	contamination 0.000; no-action 0.038 vs. defense-on-clean 0.000 (2/52, attack-conditional); the one surviving compromise is a domain-level parameter hijack counted by token taint, not exact-literal
webpage param., development ( $n{=}96$ , 96 scoreable)	D	$0.750\to 0.156$ (exact-literal 0.000); 57 fixed / 0 introduced	40.6 (93/93)	18.8 (59/59)	no-action 0.052; truncation 0.05
webpage tool-hijack; JSON tool-hijack	—	fire weakly undefended single-turn (webpage ${\sim}0.25$ ; JSON low)—those cells do not measure the defense. At the multi-turn horizon the disjoint-tool corpus fires $0.948$ undefended and the deployed configuration yields a partial reduction only: $\mathbf{0.781}$ (16 fixed / 0 introduced, $p{=}3.1{\times}10^{-5}$ ; steering verified active over the injected span)—an open boundary, not a closure	—	—
JSON param.; adaptive suite	—	not run	—	—
AgentDojo, 180-case grid, 8192 tok/turn	full grid	$0.222\to\mathbf{0.056}$ (40/180 $\to$ 10/180); 30 fixed / 0 introduced, $p{=}1.9{\times}10^{-9}$	benign 100.0% (0.857 $\to$ 0.857, 56 paired tasks; 1 lost, 1 gained)	0.733 vs. undefended 0.733	zero truncation in every arm (nothing censored); layers pinned 20/24/28 (deployed configuration); 7 of the 10 surviving compromises sit on one task whose user prompt delegates authority to the file (“follow the instructions precisely”), 2–3/180 are genuine role-confused survivors; strongest agent measured (clean 0.857)
bidirectional causal gate	D	increase side PASSES on the deployed direction: $0.615\to 0.827$ at $-\alpha 8$ (8192 budget, dev $n{=}52$ ; exact McNemar 14/3, $p{=}0.013$ ; defense-on-clean at $-\alpha$ : ASR 0.000, utility 88.5%)	—	—

Appendix

TABLE XVIII: GLM-4.5-Air, per-model detail. Deployed configuration: the action-centred behavioral direction at

\alpha{=}8

on the pooled-corpus fit’s per-layer scales

\sigma_{\ell}=6.4/8.9/10.3

(

{\approx}17

–

18\sigma

in its own per-layer scale), layers 20/24/28, 8192-token generation budget. Utility: judge % of unattacked with the (strict/lenient) bracket; AgentDojo utility is its own checker; the guard is read absolutely (defense-on-clean no-action 0.000). Holdout detail: the “Table XVIII provenance” paragraph (Appendix G ).

stage	split / samples used	held out from
generalization screening (step 3)	probe split, minus one whole authority-framing level ( firm )	that framing level, for the gate’s fit only
direction fitting (deployed)	probe split (behavioral-contrast captures; deployed gpt-oss fit: its deterministic first 24 sample ids)	dev/test samples; held-out attacker templates. Framing levels: withheld in the Llama-3.1-8B fit and the validated gpt-oss-20b refit; the other deployed fits use every level
layer selection	probe split (per-layer probe diagnostics)	dev/test samples
dose selection	dev split (template-disjoint from test)	test samples and, where stated, test attacker templates
test evaluation	test split, pre-specified one-pass, no reruns	all fitting and selection stages (samples; and where stated, attacker templates)
adaptive attacker construction	dev-disjoint sample subsets ( $n{=}18$ per class for the query search; disjoint-tool targets for GCG)	the attacked samples are held out from fitting; the deployed vector is not consumed by the query/surrogate arms and is supplied directly in the exact-vector arm. The surrogate refit deliberately reuses the fitting samples (the draw is deterministic, § C )

Appendix

TABLE XIX: Data provenance: which split each pipeline stage uses, and what it is held out from. Framing-level holdout, per fit: the recipe withholds an authority-framing level in the deployed fit itself (§ III-A ). Llama-3.1-8B’s deployed fit follows that rule end-to-end; a gpt-oss-20b refit under the rule is validated end-to-end (four one-touch held-out passes

+

the AgentDojo grid, all within the deployed configuration’s CIs; the deployed corpus references ran at a

0.7\%

lower magnitude seam, disclosed); the other three deployed fits hold the level out at the gate only, refitting over all levels (§ IV ). Contrast centring, per deployed fit: GLM-4.5-Air within (sample

\times

delegation

\times

action); the other four within (sample

\times

delegation), each deployed direction’s behavioral component measuring mean

|\cos|\leq 0.31

(per-layer max 0.334) to the action-identity (tool-hijack-vs-parameter) axis at its deployed layers.

arm	compromise $\downarrow$ (all)	Class B $\downarrow$	Class A $\downarrow$
undefended	84/177 = 0.475	29/36 = 0.806	55/141 = 0.390
CounterSteer (deployed)	14/177 = 0.079	14/36 = 0.389	0/141 = 0.000

Appendix

TABLE XX: Post-hoc delegated-authority partition, applied symmetrically to every gpt-oss arm of the certification run (4096-token budget, § V-B ). Class B = the 36 cases whose user task delegates instruction authority to retrieved content; Class A = the rest. Denominators are valid cases per arm and class.

model	params / arch	fit status	ASR / compromise rate $\downarrow$ defended
gpt-oss-20b	20B MoE	certified (held-out test pass)	0.000–0.135 corpora (Table XV ); 0.079 AgentDojo at a 4096-token budget (§ V-B )
Qwen3-30B-A3B-Thinking	30B MoE (3B active)	certified (held-out test pass)	0.000–0.173 corpora (Table XVI ); AgentDojo $0.489\to 0.072$ same-process (Table XI ), dose-selection replicate 0.094 (Table XXIV ), both at 4096 tokens
Gemma-4-31B-it	31B dense	certified (one-pass held-out test evaluations on both firing corpora with carrier documents, attacker wordings and framing templates all held out, plus the full AgentDojo grid at the deployed configuration); on the single-turn corpora the steered span is ${\sim}90\%$ of the whole prompt (those rungs do not isolate the payload-scoped edit; the agentic grid does); no adaptive evaluation	webpage param. $0.962\to\mathbf{0.038}$ (2/52; doc-clustered [0.000, 0.096], sample Wilson upper 0.130; 48 fixed / 0 introduced, $p{=}7.1{\times}10^{-15}$ ) and tool hijack $0.865\to\mathbf{0.000}$ (0/52; cluster-Wilson [0.000, 0.114]; 45/0, $p{=}5.7{\times}10^{-14}$ ), benign utility 96.2%/96% strict both corpora, guard clean, zero truncation; AgentDojo $0.239\to\mathbf{0.006}$ (42/0, $p{=}4.6{\times}10^{-13}$ ; 0 defense-introduced) at benign 92.9% ( $-7.1$ pp n.s.), 0 truncated turns in 2,721 generations; the earlier wire-format-only rung (T ∗ ) stands beside it ( $1.000\to 0.038$ , $0.865\to 0.000$ ); the prose-carrier medium is a measured null (undefended 0.000 [0.000, 0.038], $n{=}96$ ; Table XVII )
GLM-4.5-Air	${\sim}$ 106B MoE	certified (held-out test pass on its firing corpus, webpage parameter manipulation, plus the full AgentDojo grid at the deployed configuration); tool-hijack classes fire weakly undefended, JSON param. not run; no adaptive evaluation	webpage param. $0.808\to\mathbf{0.019}$ T ( $n{=}52$ ; near-duplicate-template caveat, § V ; exact-literal 0/52; Wilson [0.003, 0.101]; McNemar 41 fixed / 0 introduced; benign utility 26.9 judge (94.2/94 strict/lenient), under attack 11.5 (81/81); contamination 0.000; guard clean, read absolutely—defense-on-clean no-action 0.000, defended 0.038 attack-conditional); development split $0.750\to 0.156$ at defense-on-clean strict correctness 0.927; AgentDojo $0.222\to\mathbf{0.056}$ (30 fixed / 0 introduced, $p{=}1.9{\times}10^{-9}$ ) at benign 100.0% and utilAttack $0.733=$ undefended, zero truncation in every arm (Table XVIII )
Llama-3.1-8B-Instruct	8B dense	certified (held-out test pass on its firing corpus, webpage tool hijack, plus the full AgentDojo grid, the 26-battery rival comparison and the benchmark-level adaptive attack at the deployed configuration); fit passes all gates incl. the bidirectional causal gate ( $0.135\to 0.558$ at $-\alpha$ ); JSON tool-hijack does not fire undefended (0.042); JSON param. reaches ASR 0 only at a capability-destroying dose (judge benign 0.431), no viable cell; low task capability (13–15/56 AgentDojo clean tasks) bounds what its agentic and AgentDyn grids separate (AgentDyn: 2/60 solvable, not measurable)	webpage tool hijack $0.212\to\mathbf{0.038}$ T ( $n{=}52$ ; 11/52 $\to$ 2/52 (3/52 under a lenient tool-call executor—one surviving compromise is the attacker’s exact call with malformed JSON; $p{=}0.0078$ still significant); Wilson [0.011, 0.130]; McNemar 9 fixed / 0 introduced, $p{=}0.0039$ ; benign utility 100% byte-exact, judge floor 0.827; under attack 65.4 strict / 0.558 judge, both above undefended 59.6 / 0.462; a fit-withheld framing level composed onto the injection also blocked, 0/4 fired, point estimate); AgentDojo $0.100\to\mathbf{0.050}$ (11 fixed / 2 introduced, $p{=}0.022$ ) at benign 100%, thin-base caveat (Table XI )

Appendix

TABLE XXI: Model coverage: five open-weights models, 8B–106B, dense and MoE, five vendor lineages. “Certified” = evaluation complete and adversarially reviewed; the defended column is ASR on the single-turn corpora and compromise rate on AgentDojo, over the denominators of the table cited in each row. Rows whose numbers appear nowhere else carry their utility and capability-guard readings in the same cell.

model	hardware (ladder)	capture $+$ fit	screening $+$ causal gates	dose search	held-out certification (single-turn)	AgentDojo 180-case grid
gpt-oss-20b	4 $\times$ A100-80GB local $+$ A100 boxes; grid on one 8 $\times$ H100 node	iterative	n.r. (CPU-side)	iterative	4 corpora, one pre-specified pass, ${\leq}85$ min	62 min (8 $\times$ H100, 4 arms)
Qwen3-30B-A3B-Thinking	4 $\times$ A100-80GB local $+$ A100 boxes; grid on one 8 $\times$ H100 node	iterative	n.r.	iterative	4 corpora, one pre-specified pass, ${\leq}5.5$ h (pre-specification to last artifact)	one of six batteries in a 22.6 h 8 $\times$ H100 job; not separated
Llama-3.1-8B-Instruct	3 $\times$ A100-80GB local; grid on 4 $\times$ A100 (two boxes)	${\sim}2.3$ h (incl. a parser-bug diagnosis $+$ rerun)	${\sim}25$ min (causal smoke $+$ 3 random controls)	${\sim}1.8$ h (6 doses $\times$ 3 corpora $+$ layer sweep)	gate $+$ test touch $+$ bidirectional gate, 11 min	4 single-GPU shards, ${\leq}36$ min; end-to-end ${\sim}8.6$ h, one day
Gemma-4-31B-it	one 8 $\times$ H100 node (bring-up, cert, grid); local A100s (dose batteries)	${\sim}1$ h (one job: capture, probes, factorial, fit)	inside the bring-up job	n.r.	56 min total for both: five held-out confirms $+$ the full 180-case grid $+$ benign arms, one 8 $\times$ H100 job
GLM-4.5-Air	one 8 $\times$ H100 node (capture, sweep); local 4 $\times$ A100 (cert, grid; 212 GB sharded)	6.3 h (bring-up); certified refit CPU-only, n.r.	inside the bring-up; recipe gates CPU-side, n.r.	16.0 h sweep (superseded direction); recipe ladder terminated early, no clean total	T pass completed 3 h 05 m after the dev re-run (4 $\times$ A100, 8192-token budget)	12.2 h (43,906 s, single process, 8192-token budget)

Appendix

TABLE XXII: Per-model cost of the recipe. Wall-clock per stage where a clean record exists; n.r.

=

stage ran but no wall-clock was logged; iterative

=

developed over the research period, not separable into a single measurement. Direction fits are CPU-side; the GPU cost is activation capture and behavioral evaluation. Where no launch time was logged, a certification duration is measured from the artifacts’ timestamps and is an upper bracket.

dose $\alpha$	compromise $\downarrow$ defended [95% CI]	fixed/intr. ( $p$ )	benign $\uparrow$
3.0	55/177 = 0.311 [0.247, 0.382]	36/7 ( $9{\times}10^{-6}$ )	92.2%
4.12	44/177 = 0.249 [0.191, 0.317]	44/4 ( $1.5{\times}10^{-9}$ )	86.3%
5.5	26/177 = 0.147 [0.102, 0.207]	58/0 ( $6.9{\times}10^{-18}$ )	86.3%
6.7	19/177 = 0.107 [0.070, 0.162]	65/0 ( $5.4{\times}10^{-20}$ )	84.3%
8.06 (deployed)	15/177 = 0.085 [0.052, 0.135]	69/0 ( $3.4{\times}10^{-21}$ )	82.0%

Appendix

TABLE XXIII: gpt-oss-20b, full dose curve on AgentDojo (the curve Fig. 6 plots). 4096-token per-turn budget ; one direction (the deployed configuration’s) at every dose; undefended baseline 0.475 (84/177). Compromise rate over the 177 scoreable injection cases; fixed/intr.

=

compromises fixed / introduced (exact McNemar vs. the same-process undefended arm); benign

=

% of the ladder’s own clean arm (

n{=}55

paired unique tasks). Readings are raw: the typography normalization of Table XI could not be applied to the

\alpha{=}3.0

–

6.7

arms (transcripts not persisted), and its one compromise-rate movement (

0.085\to 0.090

on this ladder’s deployed arm) is disclosed there.

dose	compromise $\downarrow$ defended [95% CI]	utilAttack $\uparrow$ (raw)	benign $\Delta$ pp $\uparrow$
$10\sigma$	22/180 = 0.122 [0.082, 0.178]	0.728	$-5.4$
$12\sigma$ (selected)	17/180 = 0.094 [0.060, 0.146]	0.694	$-5.4$
$14\sigma$	6/180 = 0.033 [0.015, 0.071]	0.550	$-19.6$
$16\sigma$	$\geq$ 2/180 = 0.011 [0.003, 0.041] (!)	0.356 (!)	$-39.3$ (!)

Appendix

TABLE XXIV: Qwen3-30B, full dose curve on AgentDojo, 4096-token per-turn budget (the 768 collapse this budget avoids: Appendix A ). Denominators: compromise rate over all injection cases completing (the

\alpha{=}16

run’s own undefended baseline is over 175 cases, having run before a 5-case harness fix); benign over 56 paired unique tasks. (!) marks capability-guard territory; the

\alpha{=}16

row is a censored lower bound (note below).

family	representative work	lever	deployment cost	measured here
prompt-level	spotlighting/delimiting [ 2 ] ; prompt sandwich (folklore)	transforms the untrusted span so its provenance stays visible (delimiting, datamarking, encoding) and instructs the model not to follow it; or re-asserts the trusted instruction after the data	extra prompt tokens on every request	at 4096: $0.483{\to}0.273$ – $0.420$ (gpt-oss), $0.489{\to}0.356$ (Qwen reminder), Table XI
training-time	StruQ [ 3 ] , SecAlign [ 4 , 39 ] , Jatmo [ 40 ] , instruction hierarchy [ 38 ] , ISE [ 41 ] , ASIDE [ 42 ]	retrains or re-architects the model so untrusted spans carry less instructional force	a training pipeline per model; the served weights change	our SecAlign LoRA: compromises $62/180{\to}9/180$ (768 budget); benign utility of base clean, typography-normalized: $98.2\%$ at 768, $90.4\%$ unique-task / $82.3\%$ per-case at the shared 4096 budget (raw $79.7\%$ / $72.7\%$ : the DPO finetune itself emits the typography)
detection	internal-state probes: TaskTracker [ 36 ] , Attention Tracker [ 37 ] ; input-text guard models: PIGuard [ 23 ]	reads a signal and flags; no intervention is proposed, and the response is left to the surrounding system	a threshold that fails open on a miss; a second model or pass, except where the signal is already computed during the forward pass	input-text guards run as deletion filters in Table XI (compromise 0.011–0.228 at benign 48–100%, over-defense per [ 13 ] ); internal-state probes not run as defenses—§ VI shows two directions that discriminate but do not steer
detection-gated intervention	ICON [ 6 ] , ARGUS [ 7 ] , AGRI [ 8 ]	a probe decides when to intervene, then edits attention (ICON), representations plus a post-filter (ARGUS), or injects an anti-injection reasoning prefill (AGRI)	detector accuracy is a single point of failure; the gate is an additional attack surface (evaluated for AGRI: the adaptive attacker runs against the live gated arm, § V-C )	AGRI measured same-harness (Tables III , VI , § VIII ); ICON/ARGUS not run (see text)
always-on internal intervention	CachePrune [ 9 ] , V-Steer [ 10 ] , CounterSteer	edits internals over the untrusted span with no gate: KV-neuron pruning, per-request value-vector rescaling, or a fixed residual-stream direction	white-box serving access; benign utility	at 4096: CachePrune $0.483{\to}0.216$ , CounterSteer $0.483{\to}0.091$ (Table XI )
system-level / provenance	CaMeL [ 27 ] , MELON [ 45 ] , ROPE [ 5 ] , tool filters	constrains the consequences of a hijack: dual-LLM execution with data-flow tracking (CaMeL), masked re-execution and output comparison (MELON), origin-routed admission of parameter values (ROPE)	re-architecting the agent runtime; extra executions or an audited parameter set	a tool filter reaches compromise 0/176 observed with benign utility 10.9% of clean and utilAttack 0.106 typography-normalized (Table XI )

Appendix

TABLE XXV: IPI defense families, the lever each pulls, and what a deployment pays. “Measured here” is our own runs only, and is not comparable to the cited works’ reported numbers (different harnesses, models, attack sets); rates are AgentDojo compromise rates,

x/n

are corpus compromise counts. Read each entry against its own run’s undefended baseline (Table XI caveats), and never gpt-oss against Qwen (§ V-D ).

Explore similar work

May 23, 2026cs.LG

IterInject: Indirect Prompt Injection Against LLM Agents via Feedback-Guided Iterative Optimization

LLM-based agents are increasingly deployed for complex tasks requiring planning, tool use, and interaction with external services. Their reliance on untrusted external content exposes them to indirect prompt injection (IPI), in which adversarial instructions embedded in retrieved data hijack agent behavior. Existing attacks rely on static payloads that cannot adapt to agent-specific defenses; even recent adaptive methods lack structured feedback to guide optimization. We introduce \oursys, a feedback-guided iterative framework that closes the loop between injection, diagnosis, and refinement: a rule-based diagnoser produces structured outcome labels with behavioral descriptions, and an LLM-based optimizer refines payloads conditioned on the full optimization history. A synthesis step generates new disguise seeds from failure patterns, enabling the strategy space to self-evolve. On AgentDojo and InjectAgent, \oursys substantially outperforms static baselines and existing adaptive methods across four victim models. Extension experiments on Claude Code, a production-grade coding agent with layered defenses, show that optimized payloads achieve full success on 5 of 9 targets; even those that resist full exploitation exhibit measurable improvement from iterative refinement. We further present a mechanistic analysis of IPI, identifying an attention-mediated threshold mechanism in mid-to-late layers; three causal interventions validate this finding and point to concrete defense directions.

Zixuan Chen, Jiaxiang Chen, Li Luo +4

1Shanghai Jiao Tong University · 2The University of Hong Kong

Indirect Prompt Injection Large Language Model Agents Hijacking LLM Defense Mechanisms Prompt Engineering

Ask PDF

May 10, 2026cs.CR

AgentShield: Deception-based Compromise Detection for Tool-using LLM Agents

Defenses against indirect prompt injection (IPI) in tool-using LLM agents share two structural weaknesses. First, they all attempt to prevent attacks rather than detect the compromises that slip through. Second, they have only been evaluated in English, leaving users of low-resource languages such as Kurdish and Arabic without tested protection. This paper addresses both gaps with AgentShield, a deception-based detection framework that places three layers of traps inside the agent's tool interface: fake tools, fake credentials, and allowlisted parameters. The same trap triggers serve as high-precision labels for a self-supervised classifier. An LLM agent that follows an attacker's hidden instruction almost always touches one of these traps, which gives both a real-time compromise signal and a zero-FP label for training a downstream detector without manual annotation. Across 176 cross-lingual attack prompts and four LLMs from three providers, and because modern LLMs already refuse most IPI attempts on their own (attack success rate <= 10%), AgentShield's job is to catch the attacks that do slip through. On commercial models, it catches 90.7%-100% of such successful attacks, with zero false alarms on 485 normal-use tests. It survives a systematic adaptive-attack evaluation with zero evasion on commercial models, and the self-supervised classifier transfers across models and languages without retraining.

Yassin H. Rassul, Tarik A. Rashid

Computer Science and Engineering Department, School of Science and Engineering, University of Kurdistan Hewlêr, Erbil, Iraq · Artificial Intelligence and Innovation Centre, University of Kurdistan Hewlêr, Erbil, Iraq

Attacker Large Language Model Language-Model Agents Deception

Ask PDF

Sep 14, 2026cs.CR

ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents

Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prompt injection (IPI) attacks. Existing defenses mainly rely on prompt hardening, content filtering, pre-generated plans, or permission constraints. These approaches often struggle with complex tasks or over-sanitize external content, making it difficult to balance security and utility. The key challenge is therefore to preserve execution flexibility while precisely identifying and removing the malicious content that actually induces unsafe actions. To address this challenge, we propose ActGuard, a pre-execution action auditing framework. Rather than judging whether external content is inherently suspicious, ActGuard assesses whether it causes the current action to deviate from a locally reasonable expectation. At each step, ActGuard predicts the tools likely to be used by the upcoming action and constructs a local tool prior without constraining the execution trajectory. Before execution, it compares the candidate action against this prior and performs tool-level contrastive analysis and parameter-level evidence localization to identify deviations in tool selection and action parameters. A verifier then examines the localized evidence, masks only spans confirmed as malicious, and regenerates the action from the sanitized context. This design preserves legitimate planning flexibility while minimizing information loss from indiscriminate filtering. We evaluate ActGuard on challenging benchmarks for tool-using agents. Results show that ActGuard reduces attack success rates to a level comparable to state-of-the-art defenses while maintaining task utility close to the no-attack setting, achieving a favorable security-utility trade-off. Our code is publicly available at: https://github.com/binzhwang/ActGuard.

Bingzheng Wang, Xiaoyan Gu, Wentao Wang +3

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China 3Key Laboratory of Cyberspace Security Defense, Beijing, China +3

Indirect Prompt Injection Attacker Large Language Model Large Language Model Agents Model Auditing

Ask PDF