ContextLeak: Exfiltrating LLM Agent Context via Malicious Tools

Yuqi Jia; Ruiqi Wang; Patrick Li; Yuepeng Hu; Peinian Li; Neil Gong

cs.CRAug 28, 2026

ContextLeak: Exfiltrating LLM Agent Context via Malicious Tools

Authors: Yuqi Jia, Ruiqi Wang, Patrick Li, Yuepeng Hu, Peinian Li, Neil Gong

Organizations: Duke University · Stanford University

Abstract

Exfiltrating an LLM agent's runtime context -- such as the user prompt, execution trajectory, and tool list -- poses severe security and privacy risks to users. Such attacks can be carried out via malicious tools and typically require three conditions: (1) the agent selects the malicious tool for task execution, (2) the agent passes its runtime context as input arguments to the tool, and (3) the tool's implementation transmits these inputs to an attacker-controlled endpoint. Existing work primarily focuses on conditions (1) and (3), leaving condition (2) largely unexplored, despite its critical role in enabling successful context exfiltration. In this work, we bridge this gap by developing ContextLeak, a malicious tool attack that induces the agent to both select the tool and disclose its context as input arguments. We realize this attack by carefully crafting the tool's name and description using reinforcement learning. Specifically, ContextLeak employs an LLM, referred to as the attack LLM, to automatically generate the malicious tool's name and description. To improve attack effectiveness, we fine-tune the attack LLM via reinforcement learning on a set of shadow users with diverse, simulated agent contexts. Our key technical contribution is the design of novel reward functions tailored to the context exfiltration objective, enabling effective reinforcement-learning-based fine-tuning of the attack LLM. Extensive evaluation demonstrates that our attack remains highly effective even when the shadow users' contexts differ substantially from those of the victim users. Moreover, ContextLeak significantly outperforms existing malicious tool attacks when adapted to this setting.

Figures & tables

Figure 1: Illustration of the context exfiltration attack pipeline.

Figure 2: Illustration of fine-tuning the attack LLM in ContextLeak.

Method	User Prompt			Conversation History			Tool List
Method	MTSR	EDS	ES	MTSR	EDS	ES	MTSR	Prec.	Rec.	F1
Combined Attack	0.14	0.84	0.96	0.15	0.57	0.85	0.11	0.59	0.39	0.45
ObliInjection	0.80	0.77	0.93	0.82	0.58	0.85	0.71	0.71	0.51	0.55
PAIR	0.20	0.84	0.96	0.10	0.34	0.71	0.05	0.40	0.20	0.26
TAP	0.20	0.70	0.92	0.17	0.58	0.80	0.10	0.40	0.26	0.30
JudgeDeceiver	0.65	0.75	0.93	0.62	0.59	0.85	0.51	0.80	0.59	0.65

Table 1: Performance of baselines and ContextLeak in exfiltrating different types of context.

LLM	User Prompt			Conversation History			Tool List
LLM	MTSR	EDS	ES	MTSR	EDS	ES	MTSR	Prec.	Rec.	F1
GPT-OSS-20B	0.63	0.94	0.98	0.66	0.71	0.86	0.60	0.92	0.73	0.80
Gemma-4-E4B-it	0.71	0.99	1.00	0.70	0.96	0.99	0.60	0.85	0.78	0.80
Qwen-3.5-9B	0.82	1.00	1.00	0.77	0.83	0.97	0.70	0.98	0.83	0.87
GPT-4.1	0.60	0.98	1.00	0.65	0.90	0.96	0.56	0.99	0.87	0.91
GPT-5-mini	0.81	0.91	0.97	0.65	0.94	0.99	0.66	0.94	0.92	0.92

Table 2: Transferability of ContextLeak across different backend LLMs.

Victim User Type	User Prompt			Conversation History			Tool List
Victim User Type	MTSR	EDS	ES	MTSR	EDS	ES	MTSR	Prec.	Rec.	F1
IDID	0.92	0.99	1.00	0.89	0.85	0.96	0.86	0.76	0.65	0.68
IDOD	0.82	1.00	1.00	0.82	0.81	0.94	0.76	0.62	0.54	0.57
ODID	0.87	0.99	1.00	0.90	0.90	0.96	0.88	0.87	0.82	0.84
ODOD	0.83	0.98	0.99	0.74	0.88	0.96	0.70	0.93	0.87	0.89

Table 3: Transferability of ContextLeak across victim users from different domains and datasets. IDID, IDOD, ODID, and ODOD represent in-dataset-in-domain, in-dataset-out-domain, out-dataset-in-domain, and out-dataset-out-domain, respectively.

Panel of Figure 3 — Figure 3: Impact of (a) user prompt length, (b) number of conversation history rounds, and (c) number of benign tools on the attack performance of ContextLeak for each corresponding context type. (d) Sensitivity of ContextLeak to the reward-balancing weight $\lambda$ on the conversation history context.

Defense	Utility		User Prompt			Conversation History			Tool List
Defense	CTSR	IAQ	MTSR	EDS	ES	MTSR	EDS	ES	MTSR	Prec.	Rec.	F1
Llama-3-8B-Instruct	0.18	3.55	0.43	0.69	0.90	0.67	0.30	0.53	0.34	0.60	0.38	0.44
StruQ	0.15	2.83	0.08	0.75	0.95	0.06	0.19	0.37	0.09	0.17	0.08	0.10
SecAlign	0.17	2.57	0.10	0.13	0.28	0.06	0.27	0.45	0.09	0.43	0.21	0.27
Meta-SecAlign	0.17	3.30	0.42	0.68	0.91	0.68	0.30	0.52	0.33	0.78	0.37	0.49

Table 4: Performance of ContextLeak against prevention-based defenses.

Detection	User Prompt		Conv. History		Tool List
Detection	FPR	FNR	FPR	FNR	FPR	FNR
PromptGuard	0.02	1.00	0.01	0.99	0.02	1.00
DataSentinel	0.01	1.00	0.00	1.00	0.00	1.00
PromptArmor	0.00	1.00	0.00	0.97	0.00	0.98
Ant MCPScan	0.01	1.00	0.01	1.00	0.01	1.00

Table 5: FPR and FNR of detection-based defenses against ContextLeak.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Evolution of average argument reward $R_{\mathrm{arg}}$ on victim users during fine-tuning of the attack LLM across different types of context.

Panel of Figure 5 — Figure 5: Our system prompt used for guiding the attack LLM to generate tool name and description.

Panel of Figure 6 — Figure 6: Prompt used for the strategy-extraction LLM.

Table 6: Performance of ContextLeak on victim users for each domain under in-dataset-in-domain and in-dataset-out-domain settings.

Panel of Figure 7 — Figure 7: Prompt used for evaluating input argument quality (IAQ).

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 508 papers

Apr 7, 2026

Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use

Tool-use large language model (LLM) agents are increasingly deployed to support sensitive workflows, relying on tool calls for retrieval, external API access, and session memory management. While prior research has examined various threats, the risk of systematic data exfiltration by backdoored agents remains underexplored. In this work, we present Back-Reveal, a data exfiltration attack that embeds semantic triggers into fine-tuned LLM agents. When triggered, the backdoored agent invokes memory-access tool calls to retrieve stored user context and exfiltrates it via disguised retrieval tool calls. We further demonstrate that multi-turn interaction amplifies the impact of data exfiltration, as attacker-controlled retrieval responses can subtly steer subsequent agent behavior and user interactions, enabling sustained and cumulative information leakage over time. Our experimental results expose a critical vulnerability in LLM agents with tool access and highlight the need for defenses against exfiltration-oriented backdoors.

Wuyang Zhang, Shichao Pei

University of Massachusetts Boston

LLM Agent Security LLM Backdoor Attacks Memory-Augmented Language Model Agents

Ask PDF

Oct 26, 2026

Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol

LLM agents increasingly access external tools through the Model Context Protocol (MCP), but the natural-language metadata used to describe those tools creates a prompt-injection surface. Within this established tool-poisoning surface, we study Log-To-Leak, an additive post-hoc logging attack that induces a separate attacker-controlled call after the final legitimate call. Its payload targets the user query, MCP server name, server response, and agent response. We operationalize the injected directive through Trigger, Tool Binding, Justification, and Pressure, and separately measure call success, payload fidelity, and preservation of legitimate behavior. We evaluate five MCP servers and four LLM agents using five metrics spanning effectiveness, utility, and efficiency. Across 18 model-server pairs with valid tool execution, Log-To-Leak achieves an attack success rate (ASR) of 62.64-100% and conditional logging accuracy (LA) of 76.55-95.64%; 16 pairs have absolute target-task completion changes of at most 6.63 percentage points. Three evaluated defenses leave residual ASR above 55% on GitHub MCP with GPT-4o, while an instruction detector flags 85.53% of benign descriptions. These results show that successful task completion can coexist with unauthorized post-hoc disclosure and motivate host-controlled authorization for secondary tool calls.

Yuepeng Hu, Chongyu Fan, Sirat Samyoun +1

Michigan State University East Lansing, MI, USA · TikTok San Jose, CA, USA

Prompt Injection Attacks on AI Agents LLM Agent Safety MCP Security Indirect Prompt Injection

Ask PDF

Aug 20, 2026

Inadvertent Context Leakage in Language Models

For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's context window introduces hidden correlations into the model's benign outputs, allowing reconstruction even when the model correctly refuses direct extraction. We further study whether an adversary can actively engineer prompts that amplify this effect, using the model as a covert carrier to transmit secrets through seemingly innocuous text. In both cases, this limited leakage is exploited using a novel adaptive attack that assumes black-box access to the underlying model. In controlled experiments across eight proprietary models, we find that 2-digit in-context secrets are reconstructed with near-perfect accuracy and 4-digit secrets at 82% exact match, all from outputs the model produces in response to ordinary, non-adversarial requests. We observe that more capable models leak more: stronger instruction-following amplifies sensitivity to in-context secrets, suggesting leakage is a byproduct of capability as opposed to a patchable bug. We show this leakage enables two practical attacks: (1) a trained classifier that infers semantic predicates about user memories (e.g., health conditions, financial events) from routine natural-language outputs, and (2) an RL-trained adversary that extracts full Social Security Numbers from a production-style agent.

Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri +2

FAIR, Meta Superintelligence Labs · University of California, Berkeley · Google DeepMind +1

Privacy Leakage in Language Models LLM Security

Ask PDF

Abstract

AI Agent Security

Explore similar work

Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use

Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol

Inadvertent Context Leakage in Language Models