cs.CRMar 25, 2026

Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search

Authors: Yulin Shen, Xudong Pan, Geng Hong, Min Yang

Organizations: School of Computer Science, Fudan University · Shanghai Innovation Institute

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent advances in the Model Context Protocol (MCP) have enabled large language models (LLMs) to invoke external tools with unprecedented ease. This creates a new class of powerful and tool augmented agents. Unfortunately, this capability also introduces an under explored attack surface, specifically the malicious manipulation of tool responses. Existing techniques for indirect prompt injection that target MCP suffer from high deployment costs, weak semantic coherence, or heavy white box requirements. Furthermore, they are often easily detected by recently proposed defenses. In this paper, we propose Tree structured Injection for Payloads (TIP), a novel black-box attack which generates natural payloads to reliably seize control of MCP enabled agents even under defense. Technically, We cast payload generation as a tree structured search problem and guide the search with an attacker LLM operating under our proposed coarse-to-fine optimization framework. To stabilize learning and avoid local optima, we introduce a path-aware feedback mechanism that surfaces only high quality historical trajectories to the attacker model. The framework is further hardened against defensive transformations by explicitly conditioning the search on observable defense signals and dynamically reallocating the exploration budget. Extensive experiments on four mainstream LLMs show that TIP attains over 95% attack success in undefended settings while requiring an order of magnitude fewer queries than prior adaptive attacks. Against four representative defense approaches, TIP preserves more than 50% effectiveness and significantly outperforms the state-of-the-art attacks. By implementing the attack on real world MCP systems, our results expose an invisible but practical threat vector in MCP deployments. We also discuss potential mitigation approaches to address this critical security gap.

Figures & tables

Fig. 1: Attack scenario overview. A client relies on a third-party Model Context Protocol (MCP) server to provide structured inputs to its locally deployed LLM-based tool. An adversary compromising the MCP server delivers a stealthy update containing a semantically coherent prompt injection embedded within legitimate response fields. The local LLM interprets this adversarial context as trusted input, executing malicious instructions while bypassing client-side defenses due to the payload’s syntactic and semantic plausibility.
Fig. 2: Overview of the TIP framework for generating stealthy, transferable JSON payloads through a tree-based search. It includes three main stages: Branch , where tool response simulation and a dual coarse-to-fine strategy refine payload intent and structure; Prune , which evaluates candidates via transferability checks (e.g., Monte Carlo scoring) across diverse instructions and models; and Feedback & Iteration , optimizing payloads using path-aware feedback from historical high-scoring nodes.
DefenseMethodMetricFraudData Steal
   ProductWeatherBookShip
No DefenseFixedASR16.0%45.0%10.2%0.0%
Query----
TAPASR91.0%90.0%0.0%0.0%
Query2580248025802580
TIP (Ours)ASR100.0%100.0%100.0%95.0%
TABLE I: Attack Success Rates (ASR) and Query Counts across different attack methods. TIP consistently outperforms baselines in both success rate and query efficiency across all defense settings.
DefenseModelGetProductGetWeatherExpediaBookingShipManager
  FixedTAPOursFixedTAPOursFixedTAPOursFixedTAPOurs
No DefenseLlama3.1-8B-Instruct3.3%9.0%97.1%1.1%0.0%97.7%85.9%54.7%100.0%8.2%2.0%96.0%
Qwen2.5-72B-Instruct31.0%98.0%100.0%64.0%92.0%100.0%43.0%0.0%100.0%8.1%0.0%100.0%
Llama3.3-70B-Instruct2.0%0.0%98.0%2.0%39.0%100.0%0.0%0.0%100.0%52.0%0.0%100.0%
Instruction PreventionLlama3.1-8B-Instruct8.2%13.6%70.1%4.3%6.8%81.5%59.6%4.8%53.1%0.0%0.0%15.5%
Qwen2.5-72B-Instruct21.0%49.0%95.0%52.0%94.0%92.0%1.0%0.0%91.4%0.0%0.0%42.0%
TABLE II: Transferability of attack effectiveness across different victim models. Payloads were trained on a small model ensemble (Qwen2.5/InternLM/GLM) and tested against unseen target models . TIP demonstrates superior generalization compared to baselines.
Fig. 3: Cosine similarity between generated payloads and legitimate responses. TIP payloads exhibit higher semantic similarity to benign data, reducing the likelihood of detection.
ModelVariant #1Variant #2Variant #3Full Design
Qwen2.5-7B-Instruct0.0%37.7%73.0%95.0%
Llama3.1-8B-Instruct0.0%63.2%83.4%96.0%
Qwen2.5-72B-Instruct0.0%1.0%97.0%100.0%
Llama3.3-70B-Instruct0.0%45.0%61.0%100.0%
TABLE III: Impact of Removing Individual Components from TIP on Attack Success Rates: Variant #1 ( w/o. Defense Awareness ), Variant #2 ( w/o. Strategy ) and Variant #3 ( w/o. Path Feedback ). Results indicate that removing any single component significantly degrades performance or transferability.
Fig. 4: Comparison of training optimization curves between the baseline TAP and our proposed TIP. TAP suffers from local convergence, i.e., performance drops after initial peak, while our proposed TIP maintains a stable upward trend due to path-aware feedback.
Fig. 5: Tree-Structured optimization process for attack payload generation. The figure illustrates the iterative expansion of the search tree where each node represents a candidate payload with an associated score ranging from 0 to 1. Nodes are color coded based on their scores with red indicating low scores and yellow indicating high scores. The optimization process explores diverse payloads through branching and pruning which aims to converge towards high scoring nodes that maximize attack success rates.
Fig. 6: Attack effectiveness in a real-world LM Studio with Qwen2.5-7B-Instruct-1M. The agent retrieves the weather data and, prompted by the TIP-generated payload, proactively cites the attacker’s phishing URL as the official source.
Fig. 7: Attack effectiveness in a real-world Visual Studio Code IDE with GPT-4o. Despite the safety alignment of GPT-4o, the model trusts the MCP tool context and embeds the malicious link directly into the developer’s workspace.

Explore similar work

CardsList
Oct 3, 2026

COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layers

Large language models increasingly mediate tool use in Model Context Protocol (MCP) systems, where adversarial influence may enter through user instructions, tool schemas, tool outputs, or protocol messages. Existing benchmarks often evaluate deployed agents, conflating model susceptibility with guardrails, orchestration, and general task capability. We introduce COPEX (COntext Provider EXploitation), a controlled benchmark that isolates the model as an MCP client by fixing the surrounding agent stack and varying only the tool-selecting model. COPEX covers 25 attack types instantiated as 125 scenarios across four entry surfaces: model/agent, client, server/tool, and transport. Across nine models and 3,375 trials, the mean attack success rate is 64.4%, with surface-level means ranging from 58.3% to 71.4%. Some client- and transport-level attacks succeed partly outside the model's observation or control, separating system exposure from model susceptibility. Combined input and context scanning reduces mean attack success by 49.6% on an eight-attack defense subset relative to the undefended setting. The benchmark is available at https://github.com/inspire-center/copex.

Nahom Birhan, Mehrdad Rostamzadeh, Sidhant Narula +3

Old Dominion University

LLM SecurityLanguage Model RobustnessLLM Safety BenchmarksMCP SecurityAdversarial Attacks on LLMs
AskPDF
Sep 22, 2026

A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem

Agents using the Model Context Protocol (MCP) rely on semantic matching to select tools from third-party servers, exposing a semantic supply-chain risk through attacker-controlled metadata and outputs. We introduce A2M (Attraction-to-Manipulation), a two-stage black-box framework for hijacking MCP agents. The Attraction phase optimizes tool metadata to increase invocation probability; the Manipulation phase uses execution traces to refine adversarial tool returns that steer agents toward attacker-desired outcomes. On LiveMCPBench, direct attacks optimized and evaluated on GLM-4.6 achieve a macro-average malicious tool invocation rate of 93.6% across four scenarios, increase weighted token costs to 32.4×\times the benign baseline under Cognitive Denial of Service, and attain a mean attack success rate of 74.4% across Information Exfiltration, Environment Integrity Compromise, and Reasoning Derailment. Transfer to four other models without re-optimization yields corresponding macro-averages of 63.6%, 2.7×\times, and 24.5%. These findings motivate stronger tool vetting and runtime isolation in MCP ecosystems. Code is publicly available at https://github.com/Lilaizhen/A2M.

Laizhen Li, Xuan Wang, Peicheng Zhao +4

Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Nanyang Technological University +3

MCP SecurityAI Agent SecurityBlack-Box Adversarial Attacks
AskPDF
May 22, 2026

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents

The rise of tool-using Large Language Model (LLM) agents, standardized by protocols like the Model Context Protocol (MCP), has unlocked unprecedented autonomous execution capabilities for LLM Agents by integrating external open-domain knowledge and tools. However, this interoperability introduces a covert attack surface targeting the agent's cognitive planning layer. This paper systematically investigates Tool Description Poisoning (TDP), a novel semantic attack. In TDP, malicious instructions are not embedded in a tool's executable code, but rather covertly injected into its descriptive metadata, the very "manual" an agent relies on for secure planning and decision-making. To rigorously and systematically evaluate this emerging threat, we introduce the MCP-TDP Security Benchmark. This high-fidelity sandbox environment comprises 32 realistic, real-world test cases spanning 6 distinct risk categories. Our evaluation of 8 mainstream LLMs reveals severe vulnerabilities, with leading models like GPT-4o exhibiting a nearly 100% Attack Success Rate (ASR) in six high-risk scenarios. Furthermore, our findings demonstrate that common prompt-guardrail defenses are largely ineffective and can, counterintuitively, even be counterproductive (a phenomenon which we term the "Firewall Fallacy"). Crucially, we also propose a defense mechanism: "Reactive Self-Correction," where an agent autonomously detects and reverts its own malicious actions post-execution. This work provides the first specialized security benchmark tailored for TDP, offering essential insights for securing the cognitive and planning layers of advanced agentic systems.

Shi Liu, Xuehai Tang, Xikang Yang +4

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

MCP SecurityLLM Agent SecurityAI Agent Security Benchmarks
AskPDF