COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layers

Nahom Birhan; Mehrdad Rostamzadeh; Sidhant Narula; Mahmoud Nazzal; Mohammad Ghasemigol; Daniel Takabi

cs.CROct 3, 2026

COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layers

Authors: Nahom Birhan, Mehrdad Rostamzadeh, Sidhant Narula, Mahmoud Nazzal, Mohammad Ghasemigol, Daniel Takabi

Organizations: Old Dominion University

Abstract

Large language models increasingly mediate tool use in Model Context Protocol (MCP) systems, where adversarial influence may enter through user instructions, tool schemas, tool outputs, or protocol messages. Existing benchmarks often evaluate deployed agents, conflating model susceptibility with guardrails, orchestration, and general task capability. We introduce COPEX (COntext Provider EXploitation), a controlled benchmark that isolates the model as an MCP client by fixing the surrounding agent stack and varying only the tool-selecting model. COPEX covers 25 attack types instantiated as 125 scenarios across four entry surfaces: model/agent, client, server/tool, and transport. Across nine models and 3,375 trials, the mean attack success rate is 64.4%, with surface-level means ranging from 58.3% to 71.4%. Some client- and transport-level attacks succeed partly outside the model's observation or control, separating system exposure from model susceptibility. Combined input and context scanning reduces mean attack success by 49.6% on an eight-attack defense subset relative to the undefended setting. The benchmark is available at https://github.com/inspire-center/copex.

Figures & tables

Figure 1: End-to-end remote-access attack. A malicious server reports success while opening a reverse shell, which is detected from the execution trace.

Feature	MCPTox	MSB	MCPSecBench	MCP-SafetyBench	COPEX(ours)
Primary target	Tool poisoning	Agent workflow	agent platforms	Agent workflow	Model behavior
Isolates model behavior	✗	✗	✗	✗	✓
Attack types	3	12	17	20	25
Transport attacks	✗	✗	✓	✗	✓
Outcome oracle	Tool calls	Execution	Semantic	Execution	Trace + semantic

Table 1: Comparison with recent MCP security benchmarks, ordered by attack-type coverage. COPEX fixes the surrounding agent stack and evaluates the LLM acting through the MCP client.

Surface	Adversary-controlled components
Model/agent ( ma )	User task and upstream content in $p$
Client ( cl )	Client-side parsing, metadata, and context assembly
Server/tool ( st )	Tool schemas, responses, and server-side behavior
Transport ( tr )	In-transit data, endpoint resolution, and connection state

Table 2: The four attack-entry surfaces considered by COPEX , with their labels in Figure 2 .

Figure 2: COPEX execution environment. A YAML manifest configures the scenario, model, servers, defenses, and outcome oracle. The orchestrator executes a bounded agent loop and records the trajectory; red dashed arrows denote the model/agent ( ma ), client ( cl ), server/tool ( st ), and transport ( tr ) attack-entry surfaces.

Model	Opus-4.7	Sonnet-4.6	Qwen-2.5	Llama-3.2	GPT-5.5	GPT-5 Mini	GPT-4.1	Gemini-3.1	Grok-4.3	Mean
Avg.	44.3	45.1	61.1	64.5	67.5	68.0	73.1	74.1	81.6	64.4
ma	33.3	38.3	56.6	53.3	75.0	76.7	55.0	61.7	75.0	58.3
cl	38.1	37.1	58.1	60.0	66.7	69.5	86.7	95.2	98.1	67.7
st	44.2	47.9	64.9	70.3	61.8	65.5	69.1	64.8	73.9	62.5
tr	73.3	62.2	60.0	68.9	80.0	62.2	80.0	75.6	80.0	71.4

Table 3: Main benchmark ASR (%). Avg. is the macro average over the 25 attack types. Mean is the macro average over the nine models. The rows ma , cl , st , and tr report macro averages for the model/agent, client, server/tool, and transport entry surfaces.

Figure 3: ASR (%) across models. The panels contain different attack sets and therefore do not isolate the effect of the outcome oracle.

Table 7

Model	Sonnet-4.6	GPT-5.5	Gemini-3.1	Grok-4.3	Mean
ReAct	71.4	84.6	96.2	92.6	86.2
Reflexion	61.5	79.3	96.2	93.1	82.5
$\boldsymbol{\Delta}$	$-9.9$	$-5.3$	$0.0$	$+0.5$	$-3.7$

Table 6: ASR (%) under ReAct and Reflexion on the evaluated attack subset; the last column is the macro mean over the four models.

\Delta=\mathrm{Reflexion}-\mathrm{ReAct}

; negative values indicate lower ASR under Reflexion.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Surface	Attack	Mechanism	ASR (%)
ma	Prompt injection	Direct instructions bypass safety constraints and expose credentials.	37.8
	Indirect injection	Malicious directives are embedded in model-consumed files.	53.3
	Tool misuse	Social engineering redefines tool purposes and induces misuse.	90.4
	Dependency injection	Poisoned tool descriptions induce unnecessary prerequisite calls.	51.9
cl	Command overlap	Client commands conflict with server tools and leak sensitive data.	88.1
	Schema inconsistency	Tool schemas do not match their implementations.	57.8

Appendix

Table 7: COPEX attack taxonomy and mean ASR by attack. The primary entry surfaces are model/agent ( ma ), client ( cl ), server/tool ( st ), and transport ( tr ). Each mean covers nine models, five scenario variants, and three runs per variant.

Layer	Attack	Eval	Sonnet-4.6	Opus-4.7	Gemini-3.1	Grok-4.3	Llama-3.2	Qwen-2.5	GPT-4.1	GPT-5 Mini	GPT-5.5
Model	Prompt injection	J	0.0	0.0	46.7	20.0	80.0	73.3	60.0	40.0	20.0
	Indirect prompt injection	J	0.0	0.0	93.3	100.0	33.3	53.3	0.0	100.0	100.0
	Tool misuse via confused AI	J	93.3	93.3	26.7	100.0	100.0	100.0	100.0	100.0	100.0
	Function dependency injection	D	60.0	40.0	80.0	80.0	0.0	0.0	60.0	66.7	80.0
MCP Client	Slash command overlap	J	33.3	60.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	Schema inconsistencies	D	66.7	80.0	100.0	93.3	20.0	0.0	100.0	6.7	53.3

Appendix

Table 8: Per-attack ASR (%) across 9 models. J = Claude Haiku 4.5 judge; D = deterministic trace check. Each attack has 5 variants

\times

3 runs.

Attack	N	I	C	B	N	I	C	B	N	I	C	B	N	I	C	B
	Sonnet-4.6				GPT-5.5				Gemini-3.1				Grok-4.3
Function Dep. Injection	60	60	60	60	80	80	80	60	80	80	80	80	80	80	80	80
Schema Inconsistencies	80	0	0	0	80	0	0	0	100	0	0	0	80	0	0	0
Parameter Poisoning	20	20	20	20	20	60	40	60	100	100	80	100	80	100	60	60
Pkg Squatting (Tool)	60	40	60	40	60	40	60	40	60	40	60	40	60	40	60	40
Pkg Squatting (Server)	100	20	100	20	100	20	100	20	60	20	80	20	100	20	100	20

Appendix

Table 9: Per-attack ASR (%) for scanner-based defense evaluation.

Attack	R	Ref	R	Ref	R	Ref	R	Ref
	Sonnet-4.6		GPT-5.5		Gemini-3.1		Grok-4.3
Function Dep. Injection	60	60	80	80	80	80	80	80
Schema Inconsistencies	80	60	100	80	100	100	100	100
Parameter Poisoning	20	20	20	20	100	100	80	80
Data Exfiltration	60	20	40	80	40	20	40	80
Configuration Drift	80	60	100	100	80	100	100	100

Appendix

Table 10: Per-attack ASR (%) for architecture comparison. R = ReAct, Ref = Reflexion.

	Sonnet-4.6		GPT-5.5		Gemini-3.1		Grok-4.3
Attack	Det	Judge	Det	Judge	Det	Judge	Det	Judge
Parameter Poisoning	20.0	20.0	50.0	40.0	100.0	100.0	80.0	100.0
Schema Inconsistencies	80.0	80.0	80.0	100.0	100.0	100.0	60.0	60.0
Excessive Permissions	100.0	100.0	80.0	100.0	80.0	60.0	100.0	100.0
Function Dependency Injection	60.0	60.0	80.0	80.0	80.0	80.0	80.0	100.0
Pkg Squatting (Tool)	60.0	100.0	60.0	100.0	60.0	100.0	60.0	100.0

Appendix

Table 11: Deterministic vs. judge ASR (%) by attack and model for the six-attack matched comparison summarized in Table 5 .

Figure 4: Provider-reported prefill and decode tokens per trial on a logarithmic scale. Right-side labels report the API cost of each 375-trial hosted-model evaluation. Local models are omitted because local compute is not assigned an API price.

Panel of Figure 5 — Figure 5: System prompt configuration for the independent LLM judge evaluator. The model processes execution traces and assigns a standardized success or detection verdict using structured JSON outputs to preserve rubric consistency across heterogenous agent trials.

Panel of Figure 6 — Figure 6: YAML manifest specification for a judge-mode Configuration Drift scenario evaluating tool surface stability over sequential protocol discovery steps.

Panel of Figure 7 — Figure 7: Serialized execution log of a Package Name Squatting trial. The deterministic evaluation engine flags a vulnerability compromise verdict by scanning the runtime tool execution sequences for unauthorized execution hashes.

Panel of Figure 8 — Figure 8: Prompt used by the Reflexion critic to review the completed ReAct trajectory before returning the final answer.

Panel of Figure 9 — Figure 9: System prompt configuration for the InputScanner guardrail layer, designed to filter adversarial intents and indirect prompt injections before core engine consumption.

Panel of Figure 10 — Figure 10: System prompt configuration for the ContextScanner tool schema audit layer, inspecting dynamic MCP manifests for description-embedded injections, parameter coercion traps, and illicit capability registration before exposure to the host agent.

Panel of Figure 11 — Figure 11: System prompt configuration for the ContextScanner validation layer, intercepting tool payloads to detect dynamic semantic drift, rug pull vectors, and late-stage context hijacking.

Explore similar work

Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search
Mar 25, 2026Yulin Shen, Xudong Pan, Geng Hong +1Prompt Injection Attacks on AI Agents MCP Security
School of Computer Science, Fudan University · Shanghai Innovation Institute
MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use
Dec 31, 2025Zixiang Liu, Wenrui Liu, Elsie Dai +5LLM Agent Evaluation AI Agent Benchmarks
Peking University · ZTE
Game-Theoretic Multi-Agent Control for Robust Contextual Reasoning in LLMs
Jun 9, 2026Saeid Jamshidi, Amin Nikanjam, Arghavan Moradi Dakhel +2Adversarial Attacks on LLMs Multi-Agent Control
SWAT Laboratory, Polytechnique Montréal, Montréal, QC, Canada · Huawei Distributed Scheduling and Data Engine Lab, Montr´eal, QC, Canada

Mar 25, 2026J/K move · Enter open · S save

Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search

Yulin Shen, Xudong Pan, Geng Hong, Min Yang

School of Computer Science, Fudan University · Shanghai Innovation Institute

Prompt Injection Attacks on AI Agents MCP Security Indirect Prompt Injection Black-Box Adversarial Attacks

Open paper AskPDF arXiv

Recent advances in the Model Context Protocol (MCP) have enabled large language models (LLMs) to invoke external tools with unprecedented ease. This creates a new class of powerful and tool augmented agents. Unfortunately, this capability also introduces an under explored attack surface, specifically the malicious manipulation of tool responses. Existing techniques for indirect prompt injection that target MCP suffer from high deployment costs, weak semantic coherence, or heavy white box requirements. Furthermore, they are often easily detected by recently proposed defenses. In this paper, we propose Tree structured Injection for Payloads (TIP), a novel black-box attack which generates natural payloads to reliably seize control of MCP enabled agents even under defense. Technically, We cast payload generation as a tree structured search problem and guide the search with an attacker LLM operating under our proposed coarse-to-fine optimization framework. To stabilize learning and avoid local optima, we introduce a path-aware feedback mechanism that surfaces only high quality historical trajectories to the attacker model. The framework is further hardened against defensive transformations by explicitly conditioning the search on observable defense signals and dynamically reallocating the exploration budget. Extensive experiments on four mainstream LLMs show that TIP attains over 95% attack success in undefended settings while requiring an order of magnitude fewer queries than prior adaptive attacks. Against four representative defense approaches, TIP preserves more than 50% effectiveness and significantly outperforms the state-of-the-art attacks. By implementing the attack on real world MCP systems, our results expose an invisible but practical threat vector in MCP deployments. We also discuss potential mitigation approaches to address this critical security gap.

Abstract

Explore similar work

Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search

MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use

Game-Theoretic Multi-Agent Control for Robust Contextual Reasoning in LLMs