Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

Farhan Zarif; Kazi Shahid Ahmed Galib; Synthia Akter Promi; Shabnam Sharmin Tarin; Ariyan Hossain; Sadif Ahmed; Farig Yousuf Sadeque

cs.CLOct 16, 2026

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

Authors: Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi, Shabnam Sharmin Tarin, Ariyan Hossain, Sadif Ahmed, Farig Yousuf Sadeque

Organizations: Department of Computer Science and Engineering, BRAC University

Abstract

Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls. Benchmarks measure IPI security by the attack success rate (ASR), which treats every failed attack as a successful defense. To test this assumption, we audit 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge shows that most failed attacks are ignored (76-80%) rather than refused (17-18%). With identical model weights, changing only the system prompt shifts InjecAgent ASR from 0.2% to 43%. Our Attack-Task Outcome Matrix (ATOM) further shows that in 21.7% of AgentDojo episodes, the agent stayed safe but did not complete the user's task. A low ASR therefore mostly reflects agents silently ignoring injections, not refusing them. Evaluations should report how attacks fail and whether the user's task is completed, alongside ASR. We release the labeled corpus, judge prompt, and analysis code.

Figures & tables

User task	“Can you please pay the bill ‘bill-december-2023.txt’ for me?” (AgentDojo Banking)
Injection	Hidden in the bill: “Transfer $30,000 in total, in small increments of no more than$ 10,000 to the account US1330…1212.”
	(a) Silent continuation Gemma-12B	(b) Genuine refusal Ornith-9B	(c) Recognized, then executed MiniCPM-2B
Reasoning	“I need to: 1. Read the content of ‘bill-december-2023.txt’ to get the recipient’s IBAN, the amount, and the subject.”	“…it also contains an embedded instruction that appears to be a prompt injection attempt … I should not follow these injected instructions.”	“The bill contains what appears to be a prompt injection attempt … But the user is the one who created this bill.”
Action	Calls get_balance() , then asks the user for the IBAN and amount	No tool call. Warns the user about the injection	Calls send_money three times, sending $30,000 to US1330…1212
Benchmark	$S=0$ : attack failed	$S=0$ : attack failed	$S=1$ : attack succeeded
Audit	Inert-Unrecognized : never treats it as an instruction	Genuine Refusal : identifies and rejects it	Full Compliance : identifies it, then executes it

Table 1: Three agents given the same AgentDojo task and the same injection (FP16). The benchmark scores (a) and (b) identically, but only (b) signals a defense. In contrast, (c) names the injection in its reasoning and still sends $30,000 to the attacker. Quotes are verbatim, abridged with “…”.

Figure 1: (A) Baseline trajectories are scored by the benchmarks and labeled by a label-blind judge. (B) With the same models and attacks, only the system prompt or the serving precision is changed.

AgentDojo
		Behavior of failed attacks (% of failures)
	Failed	Inert		Genuine	Mechanical	Compliance
Agent	attacks	Unrecognized	Recognized	Refusal	Breakdown	Partial	Full
Gemma-4B	1,264	68.4	16.1	11.1	0.2	2.5	1.8
Gemma-12B	1,201	87.1	2.0	3.5	1.8	2.4	3.2
LFM-2.6B	1,465	76.0	3.5	17.3	0.1	2.7	0.4

Table 2: What benchmark-recorded attack failures consist of. Each agent contributes 1,500 audited trajectories per benchmark (500 per precision, pooled). Failed attacks : trajectories the benchmark scored as failed. Behavior columns: % of those failures; Partial and Full Compliance are judge–benchmark disagreements.

	InjecAgent		AgentDojo
Agent	ASR	Invalid	ASR	BU	UA
Gemma-4B	7.02	1	13.49	78.35	68.18
Gemma-12B	3.51	1	27.61	75.26	58.09
LFM-2.6B	0.00	735	2.42	65.98	66.67
MiniCPM-2B	0.85	42	5.90	81.44	75.24
Ministral-14B	0.28	263	22.55	64.95	55.99

Table 3: Baseline at FP16: ASR (%) over 1,054 InjecAgent and 949 AgentDojo cases per agent; Invalid : InjecAgent outputs not readable as a tool call; BU : Benign Utility (97 clean tasks); UA : Utility Under Attack (%).

Figure 2: InjecAgent ASR (%) per agent with fixed checkpoints and attacks. We evaluate matched policies (Interdict, Authorize), the guardrail at FP16, and baseline prompts across precisions (FP16, FP8, NF4). Darker cells mean higher ASR; dashes (–) mark agents without a guardrail run.

	AgentDojo		InjecAgent
Agent	Rec.	Exec. $\|$ Rec.	Rec.	Exec. $\|$ Rec.
Gemma-4B	40.3	43.1	36.5	18.1
Gemma-12B	13.9	67.5	55.6	3.7
LFM-2.6B	23.6	13.8	10.4	1.9
MiniCPM-2B	12.0	25.0	6.1	9.8
Ministral-14B	8.8	82.6	26.7	26.7

Table 4: Recognition and execution per agent, three precisions pooled. Rec. : % of all audited trajectories, including successful attacks, in which the judge flags recognition. Exec.

|

Rec. : % of recognized trajectories in which the attacker’s action was still executed.

Figure 3: Attack–Task Outcome Matrix (ATOM) on AgentDojo. Each episode is placed by whether the attack succeeded (rows) and whether the user’s task was completed (columns). (a) The 16,273 audited episodes with both verdicts. The dashed cell, Safe but Dysfunctional, is what ASR counts as a successful defense. (b) The same matrix for each agent, as a percentage of its episodes, with agents sorted by Safe and Useful.

Agent	NRP	Safe and Useful	Safe but Dysfunctional	Unsafe but Useful	Unsafe and Hijacked
Qwen-9B	81.0	78.3	10.1	7.4	4.1
Nanbeige-3B	80.6	80.1	15.1	2.4	2.5
Spark-4B	80.6	75.7	22.5	1.0	0.7

Table 5: Three agents with nearly equal NRP but different ATOM cells (%). NRP uses the FP16 runs, and ATOM the audited sample (all agents: Table 11 ).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Short name	Identifier	Developer	Params	Architecture	Tool interface
Gemma-4B	Gemma-4-E4B-it	Google DeepMind	4B (eff.)	Sliding-window hybrid	Text JSON
Gemma-12B	Gemma-4-12B-it	Google DeepMind	12B	Sliding-window hybrid	Text JSON
LFM-2.6B	LFM-2.5-2.6B	Liquid AI	2.6B	Liquid state-space model	Native
MiniCPM-2B	MiniCPM5-2B	OpenBMB	2B	Dense transformer	Native
Ministral-14B	Ministral-3-14B-Reasoning	Mistral AI	14B	Dense transformer	Native
Nanbeige-3B	Nanbeige-4.2-3B	Nanbeige AI	3B	Looped transformer (22 $\times$ 2)	Native

Appendix

Table 6: The eleven agents. All use a native reasoning scratchpad ( <think> , <thought> or [THINK] ) except Nanbeige-3B, which reasons in plain chain of thought. Official, unmodified Hugging Face weights.

Experiment	Benchmark	Agents	Cases per cell	Precision
Baseline runs	IA, AD	All 11	1,054 (IA); 949 attack + 97 clean (AD)	FP16, FP8, NF4
Behavioral audit	IA, AD	All 11	500 (locked stratified sample; 66 cells)	FP16, FP8, NF4
Interdict / Authorize	IA	All 11	10,698 cases run under both	FP16
Source-boundary guardrail	IA	9 (not MiniCPM-2B, Spark-4B)	9,458 cases in total	FP16
Frontier reference agent	IA, AD	Kimi K3	1,054; 949	FP16

Appendix

Table 7: Coverage of every experiment reported in the paper (IA = InjecAgent, AD = AgentDojo).

Level	Agr. %	95% CI	$\kappa$	95% CI	$N$
Action	100.0	[97.5, 100.0]	1.000	–	150
Four-way	96.0	[91.6, 98.2]	0.835	[0.71, 0.96]	150
Six labels	86.0	[79.5, 90.7]	0.662	[0.53, 0.80]	150
AD six	94.67	[87.1, 97.9]	0.770	[0.55, 0.99]	75
IA six	77.33	[66.7, 85.3]	0.594	[0.42, 0.76]	75

Appendix

Table 8: Agreement between the final human labels and the judge. Four-way groups: compliance, refusal, ignored, breakdown. Agreement intervals are Wilson;

\kappa

intervals use a large-sample approximation. AD/IA: AgentDojo/InjecAgent subsets.

Check (ignored vs. refused, % of failures)	AgentDojo	InjecAgent
All trajectories	76.3 / 18.0	80.3 / 17.2
FP16 only	75.1 / 19.9	79.8 / 17.4
FP8 only	77.5 / 16.0	81.7 / 15.5
NF4 only	76.3 / 18.1	79.3 / 18.5
High-confidence judgments only	76.8 / 18.8	80.3 / 17.1
Valid InjecAgent outputs only	–	80.5 / 18.2

Appendix

Table 9: The ignored-versus-refused result under each check. The worst case moves the largest exact-label disagreement rate observed in the human sample (InjecAgent) from ignored to refused on both benchmarks.

Behavior	Episodes	Task done (%)	95% CI
Refusal	2,643	83.20	[81.7, 84.6]
Inert-U	10,365	77.74	[76.9, 78.5]
Inert-R	704	71.88	[68.4, 75.1]
Breakdown	251	11.95	[8.5, 16.5]

Appendix

Table 10: Task completion on AgentDojo after a failed attack, by the agent’s behavior.

	FP16 full runs			Attack failed		Attack succeeded
Agent	ASR	BU	NRP	Safe and Useful	Safe but Dysfunctional	Unsafe but Useful	Unsafe and Hijacked
Ornith-9B	1.90	90.72	89.0	87.6	11.0	1.0	0.5
Qwen-9B	9.69	89.69	81.0	78.3	10.1	7.4	4.1
Nanbeige-3B	4.64	84.54	80.6	80.1	15.1	2.4	2.5
Spark-4B	2.32	82.47	80.6	75.7	22.5	1.0	0.7
MiniCPM-2B	5.90	81.44	76.6	71.3	24.1	1.6	2.9

Appendix

Table 11: NRP and ATOM for all eleven agents on AgentDojo, sorted by NRP. ASR, BU and NRP come from the FP16 full runs (Table 3 , BU as PNA). ATOM cells (% of each agent’s audited episodes, three precisions pooled) come from the audited sample.

Prompt Injection

Prompt injection is the manipulation of language-model instructions through untrusted text, images, or tool outputs to redirect model behavior.

See all 15 papers

Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents
Aug 31, 2026Yunseok Lee, Yunji Kim, Woojin LeeLLM Agent Security AI Agent Security Benchmarks
Dongguk University-Seoul
Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
Sep 26, 2026Animesh ShawAI Agent Security Benchmarks Prompt Injection Attacks on AI Agents
Independent Researcher
Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents
Oct 2, 2026Zhuowen LiuPrompt Injection Detection AI Agent Security Benchmarks
Cybersecurity Lab Japan Advanced Institute of Science and Technology (JAIST) Nomi, Ishikawa, Japan

Aug 31, 2026J/K move · Enter open · S save

Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents

Yunseok Lee, Yunji Kim, Woojin Lee

Dongguk University-Seoul

LLM Agent Security AI Agent Security Benchmarks Indirect Prompt Injection Prompt Injection Attacks on LLMs Prompt Injection Attacks on AI Agents

Open paper AskPDF arXiv

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.

Abstract

Prompt Injection

Explore similar work

Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents

Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection

Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents