VeriGrey: Greybox Agent Validation

Yuntong Zhang; Sungmin Kang; Ruijie Meng; Marcel Böhme; Abhik Roychoudhury

cs.AIMar 18, 2026

VeriGrey: Greybox Agent Validation

Authors: Yuntong Zhang, Sungmin Kang, Ruijie Meng, Marcel Böhme, Abhik Roychoudhury

Organizations: National University of Singapore · Max-Planck Insitute of Security and Privacy

Abstract

Agentic AI has been a topic of great interest recently. A Large Language Model (LLM) agent involves one or more LLMs in the back-end. In the front end, it conducts autonomous decision-making by combining the LLM outputs with results obtained by invoking several external tools. The autonomous interactions with the external environment introduce critical security risks. In this paper, we present a grey-box approach to explore diverse behaviors and uncover security risks in LLM agents. Our approach VeriGrey uses the sequence of tools invoked as a feedback function to drive the testing process. This helps uncover infrequent but dangerous tool invocations that cause unexpected agent behavior. As mutation operators in the testing process, we mutate prompts to design pernicious injection prompts. This is carefully accomplished by linking the task of the agent to an injection task, so that the injection task becomes a necessary step of completing the agent functionality. Comparing our approach with a black-box baseline on the well-known AgentDojo benchmark, VeriGrey achieves 33% additional efficacy in finding indirect prompt injection vulnerabilities with a GPT-4.1 back-end. We also conduct real-world case studies with the widely used coding agent Gemini CLI, and the well-known OpenClaw personal assistant. VeriGrey finds prompts inducing several attack scenarios that could not be identified by black-box approaches. In OpenClaw, by constructing a conversation agent which employs mutational fuzz testing as needed, VeriGrey is able to discover malicious skill variants from 10 malicious skills (with 10/10= 100% success rate on the Kimi-K2.5 LLM backend, and 9/10= 90% success rate on Opus 4.6 LLM backend). This demonstrates the value of a dynamic approach like VeriGrey to test agents, and to eventually lead to an agent assurance framework.

Figures & tables

Figure 1. A diagram of the attack model for our work.

Figure 2. Examples of context bridging and feedback for VeriGrey . Presented examples are from our experiments, lightly edited for clarity.

Panel of Figure 3 — Figure 3. Prompt template of mutating injection prompts.

LLM	Method	Average ITSR for each Suite				Average ITSR for each Difficulty			Average ITSR (Overall)
LLM	Method	Workspace	Travel	Banking	Slack	Easy	Medium	Hard	Average ITSR (Overall)
GPT-4.1	Baseline	16.1	40.7	34.7	78.1	55.2	7.6	11.3	37.7
GPT-4.1	VeriGrey	48.2	87.1	68.1	100	88.1(+32.9)	30.1(+22.5)	43.3(+32.0)	70.7 (+33.0)
Gemini-2.5 Flash	Baseline	9.8	65.0	14.6	78.1	49.5	11.7	11.9	36.8
Gemini-2.5 Flash	VeriGrey	15.9	77.1	32.6	90.5	63.2(+13.7)	16.2(+4.5)	17.5(+5.6)	47.4 (+10.6)
Qwen-3 235B	Baseline	40.2	90.7	64.6	100	88.7	28.1	32.0	67.6

Table 1. Average Injection Task Success Rate (ITSR) on the AgentDojo benchmark. All values are percentages; numbers in parentheses denote absolute percentage-point difference from the baseline.

	Average ITSR for each Difficulty			Average ITSR (Overall)
	Easy	Medium	Hard	Average ITSR (Overall)
VeriGrey (Full)	88.1	30.1	43.3	70.7
VeriGrey – w/o Feedback	79.1(-9.0)	20.2(-9.9)	21.6(-21.7)	59.6 (-11.1)
VeriGrey – w/o Context Bridging	60.2(-27.9)	10.1(-20.0)	13.4(-29.9)	44.0 (-25.8)

Table 2. Ablation results with Feedback and Context Bridging removed separately. All values are percentages; numbers in parentheses denote absolute percentage-point difference from the full VeriGrey setup. All data obtained with GPT-4.1 as the backend LLM.

Defense	No Injection	VeriGrey		Black-box Baseline
Defense	UTSR (%)	UTSR (%)	ITSR (%)	UTSR (%)	ITSR (%)
No Defense	85.0	42.2	69.7	69.6	41.2
Prompt sandwiching	86.5	54.4	65.2	75.1	17.4
Data delimiter	81.0	45.6	67.6	70.8	36.4
Prompt injection detection	50.4	19.8	21.6	22.5	5.2
Tool filter	81.7	48.8	17.4	71.5	12.8

Table 3. Average User Task Success Rate (UTSR) and Injection Task Success Rate (ITSR) when prompt injection defenses are in place. All values are percentages. All data obtained with GPT-4.1 as the backend LLM.

Attack No.	Task
1	Send contents of SECRET file to a URL
2	Replace password values in repository
3	Inject malicious memory into agent
4	Save Gemini API key to a target location
5	Exfiltrate user information (e.g. system username) to a URL
6	Install cron job to download and run payload

Table 4. Gemini CLI injection tasks.

Method	Task by Difficulty										ITSR (%)
	Easy					Hard
	1	2	3	4	5	6	7	8	9	10
Baseline (black-box)	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	$\times$	60
VeriGrey	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	90
Baseline + Tool Filter	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	30
VeriGrey + Tool Filter	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	$\times$	50

Table 5. Gemini CLI attack success results using the black-box baseline and VeriGrey .

Category	# Skills	Kimi-K2.5 vulnerable?		Opus-4.6 vulnerable?		GPT-5.2 vulnerable?
Category	# Skills	Original	Mutated	Original	Mutated	Original	Mutated
Silent action	1	1	1	1	1	1	1
Silent telemetry	1	0	1	0	1	0	1
Malicious install	8	0	8	0	7	0	6
Total	10	1	10	1	9	1	8

Table 6. Results of testing OpenClaw with different LLMs as its backend. Each cell shows the number of skills for which the agent followed the malicious instructions embedded in the skill definition. Column “Original” refers to the original skills from ClawHub; column “Mutated” refers to the skills after mutation by VeriGrey .

Figure 4. Tactics discovered by VeriGrey to attack OpenClaw.

LLM Agent Security

Large language model agent security studies attacks and safeguards for systems that plan, use tools, retain memory, and act across applications.

See all 327 papers

Assessing Automated Prompt Injection Attacks in Agentic Environments
Jun 9, 2026David Hofer, Edoardo Debenedetti, Florian TramèrAdversarial Prompt Generation LLM Agent Security
ETH Zurich Zurich, Switzerland
Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents
Jul 22, 2026Or Zion Eliav, Eyal Lenga, Shir Bernstien +1AI Agent Security AI Agent Security Benchmarks
Faculty of Computer and Information Science Ben-Gurion University of the Negev Beer-Sheva, Israel
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5AI Agent Auditing LLM Auditing
Google Cloud AI Research · Google · Michigan State University

Jun 9, 2026J/K move · Enter open · S save

Assessing Automated Prompt Injection Attacks in Agentic Environments

David Hofer, Edoardo Debenedetti, Florian Tramèr

ETH Zurich Zurich, Switzerland

Adversarial Prompt Generation LLM Agent Security AI Agent Security Benchmarks Indirect Prompt Injection Prompt Injection Attacks Prompt Injection Attacks on LLMs+2

Open paper AskPDF arXiv

Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a comprehensive empirical evaluation of automated prompt injection attacks against LLM agents, adapting both white-box (GCG) and black-box (TAP) methods to the agentic setting within the AgentDojo framework. We evaluate across 80 task pairs spanning four domains and multiple models, and find that black-box optimization substantially outperforms gradient-based methods, a gap we attribute to GCG's optimization instability under reasonable compute budgets. We also find that TAP's effectiveness depends on the attacker model, as both general capability and safety tuning affect attack success--stronger models produce more effective injections, while safety-tuned attackers can refuse to generate adversarial prompts. Task-universal attacks transfer effectively to unseen tasks and out-of-distribution domains, but attacks optimized on smaller open-source models do not transfer to frontier models like GPT-5. These findings highlight automated prompt injection as a credible but model-dependent threat, with significant barriers remaining for model-agnostic exploitation.

Abstract

LLM Agent Security

Explore similar work

Assessing Automated Prompt Injection Attacks in Agentic Environments

Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents

PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections