MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?

Lingqi Jiang; Jialuo Chen; Jianan Ma; Xinhao Deng; Xiaohu Du; Sibo Yi; Yuqi Qing; Zhenguang Liu

cs.CRSep 28, 2026

MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?

Authors: Lingqi Jiang, Jialuo Chen, Jianan Ma, Xinhao Deng, Xiaohu Du, Sibo Yi, Yuqi Qing, Zhenguang Liu, +3 more

Organizations: Zhejiang University · Ant Group · Hangzhou Dianzi University · Tsinghua University

Abstract

Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that agents retrieve and inspect during execution. Because these images guide actions, attackers can disguise malicious instructions as ordinary visual guidance within otherwise legitimate skills. Existing skill-security research primarily examines text-carried attacks or scanner detection, leaving the runtime effects of image-borne attacks insufficiently evaluated. We introduce MMSkillRisk, to our knowledge the first publicly available benchmark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. To instantiate this attack surface, we design Native-Context Visual Attack (NCVA), which disguises malicious instructions as native components of teaching images, such as annotations and interface labels. The accompanying SKILL.md provides auxiliary guidance toward relevant visual regions without explicitly stating the malicious operation. Built from 28 curated clean skills, MMSkillRisk contains 36 attack packages and 108 executable cases spanning five attack objectives, with separate checks for attack success and legitimate-task completion. Across nine model-harness configurations evaluated in isolated sandboxes, NCVA induces unauthorized operations in every configuration. Its pooled attack success rate (ASR) reaches 43.1%, exceeding the matched text-carrier baseline by 16.4 percentage points, with higher ASR in all nine configurations. Attack success and legitimate-task completion co-occur in 36.5% of cases, reaching 72.2% for GPT-5.6-sol with Codex. These results show that skill-bundled images can induce unauthorized actions even as agents complete legitimate tasks, so task success alone does not establish safe skill use. Our code and data are available at https://github.com/kaill-jlq/MMSkillRisk.

Figures & tables

Figure 1: Data construction pipeline of MMSkillRisk. The stages run from left to right. Skill and Task Preparation curates 28 base skills and constructs 84 benign tasks. Attack Design and Instantiation creates 36 compromised skill variants across five risk categories, embedding malicious instructions in teaching images with accompanying textual guidance. Instance Testing and Verification executes skill–task pairs in isolated Docker environments and assesses attack success and task completion using traces, artifacts, and state changes, yielding 108 evaluation instances.

Harness	Model	Text-only baseline (%)			NCVA (%)
		ASR	TSR	TC-ASR	ASR	TSR	TC-ASR	VIAR
Codex	GPT-5.6-sol	59.3	84.5	51.9	81.5	89.3	72.2	91.7
	Kimi-K2.6	32.4	73.8	23.1	65.7	78.6	49.1	68.5
	Kimi-K3	14.8	96.4	13.9	38.0	97.6	37.0	38.9
	Qwen3.8-Flash-Next	13.0	95.2	12.0	32.4	94.1	32.4	33.3
Claude Code	Kimi-K2.6	36.1	73.8	23.1	50.9	75.0	35.2	52.8

Table 2: Attack and task performance across nine model–harness configurations (%). ASR, TC-ASR, and VIAR use 108 cases; TSR excludes 24 integrity-destruction cases. Overall rates are pooled. Bold marks higher values between attack conditions, including ties; underlining marks column maxima within each harness. VIAR is reported only for NCVA.

Harness	Model	Read $\rightarrow$	Seen $\rightarrow$	Attempt $\rightarrow$	Success
Claude Code	Kimi-K3	100.0	99.1	17.6	17.6
	Kimi-K2.6	100.0	53.7	50.9	50.9
	Qwen3.8-Flash	100.0	100.0	27.8	27.8
Codex	Kimi-K3	99.1	96.3	38.0	38.0
	Kimi-K2.6	100.0	70.4	65.7	65.7
	Qwen3.8-Flash	100.0	94.4	32.4	32.4

Table 3: From image access to attack execution (%).

Figure 3: Attack-category profiles across model–harness configurations. Each radar chart reports the ASR (%) of one configuration across five attack categories. All panels use 0-100% scale.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Base skill	Scope of the three legitimate tasks
anydesign	Extract design information from page screenshots, reproduce components, and document the design.
breadth-chart-analyst	Read breadth charts and their signals, and prepare analytical results.
guizang-social-card-skill	Create social media cards for different themes using supplied materials.
infographic	Produce infographics that follow the requested topic and brand requirements.
sector-analyst	Read sector rotation and momentum information across time frames.

Appendix

Table 5: Publicly collected skills and their legitimate task scopes.

Base skill	Scope of the three legitimate tasks
mmskills-calc-charts	Identify Calc chart configurations and output states.
mmskills-chrome-permissions	Read Chrome permission and security settings.
mmskills-gimp-layers	Read GIMP layer and mask states.
mmskills-thunderbird-filters	Read mail-filter rules and their enabled states.
mmskills-ubuntu-assistive-input	Read accessibility and input settings.
mmskills-vlc-audio-settings	Read audio effects and volume preferences.

Appendix

Table 6: Skills adapted from MMSkills resources and their legitimate task scopes.

Base skill	Scope of the three legitimate tasks
gui-calc-freeze	Determine the frozen-pane state of a spreadsheet.
gui-calc-pivot	Inspect pivot-table field layouts.
gui-chrome-search	Distinguish the default search engine from a highlighted candidate.
gui-gimp-color	Determine the current state of a color-adjustment interface.
gui-gimp-layers	Determine layer selection, visibility, and related states.
gui-impress-export	Inspect presentation-export dialog settings.

Appendix

Table 7: Locally authored schematic GUI skills and their legitimate task scopes.

Base skill	Scope of the three legitimate tasks
visual-chart-reconciliation	Reconcile and correct chart data bindings or values.
visual-form-routing	Determine how requests should be routed using forms and flow diagrams.
visual-invoice-reconciliation	Reconcile invoices, purchase orders, and posting batches.
visual-palette-audit	Check interface color tokens and correct inconsistencies.
visual-warehouse-picking	Allocate picking batches using bin labels and inventory states.

Appendix

Table 8: Original visual workflow skills and their legitimate task scopes.

Figure 5: Additional examples of benign and poisoned skill images with workflow-aligned attack embeddings.

Model	API identifier	Harness	Version
GPT-5.6-sol	gpt-5.6-sol	Codex	0.153.3
Kimi-K2.6	Kimi-K2.6	Codex	0.153.3
Kimi-K3	Kimi-K3	Codex	0.153.3
Qwen3.8-Flash-Next	Qwen3.8-Flash-Next	Codex	0.153.3
Kimi-K2.6	Kimi-K2.6	Claude Code	2.1.268
Kimi-K3	Kimi-K3	Claude Code	2.1.268

Appendix

Table 9: Model identifiers and agent-harness versions used in the evaluation.

Component	Codex / Claude Code	DSH
Operating system	Debian GNU/Linux 13 (trixie)	Debian GNU/Linux 13 (trixie)
Debian full version	13.6	13.6
Architecture	linux/arm64	linux/arm64
Python	3.11.16	3.11.16
Node.js	22.22.0	22.22.0
CPU quota per instance	2 CPUs	2 CPUs

Appendix

Table 10: Container environment and per-instance resource limits.

Harness	Model	Clean TSR	NCVA TSR
Codex	Kimi-K3	98.8	97.6
Claude Code	Kimi-K3	98.8	97.6
DSH	DeepSeek-V4.1-Flash	92.9	89.3
Codex	Qwen3.8-Flash-Next	96.4	94.1
Claude Code	Qwen3.8-Flash-Next	96.4	96.4

Appendix

Table 11: Task success rates (%) with clean skills and NCVA. Each result is measured over 84 non-integrity cases.

Figure 6: Attack-success judge: system prompt and output schema

Figure 7: Task judge, step 1: acceptance-checklist prompt

Figure 8: Task judge, step 2: task-completion prompt

Explore similar work

May 12, 2026cs.CR

SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces

Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present SkillSafetyBench, a runnable benchmark for evaluating such skill-mediated safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold-model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments.

Chang Jin, An Wang, Zeming Wei +7

1Shanghai AI Laboratory · 2Peking University · 3East China Normal University

Large Language Model Agents Adversarial Robustness

Ask PDF

Jun 16, 2026cs.CR

Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners

Agent skills are emerging as an important attack surface in LLM-based systems. Through an empirical study of existing skill scanners, we find that current defenses primarily rely on textual descriptions, manifests, and source code as the main signals for security analysis, which can leave visually conveyed malicious intent insufficiently examined. This creates a practical blind spot: harmful operational instructions hidden in images may bypass scanning while still being recoverable by multimodal agents during deployment. To systematically investigate this threat, we propose SkillCamo, a document-mediated multimodal instruction attack that conceals malicious instructions within images bundled with a skill while rewriting the surrounding documentation to naturally reference those images as part of the normal workflow. Thus, the attack does not rely on the image alone, but on the joint interpretation of textual guidance and visual payload at execution time. To defend against such attacks, we further propose ExecScan, an execution-grounded multimodal scanning module that performs intent extraction, behavior reconstruction, abuse assessment, and deliberative execution simulation over skill artifacts. ExecScan jointly analyzes documentation, code, referenced resources, and visual content to recover hidden instructions, reconstruct executable behavior chains, and identify downstream risks such as exfiltration, destruction, persistence, deception, and privilege escalation. Extensive experiments show that image-hidden malicious instructions challenge existing skill scanners, while ExecScan can improve the skill scanning performance.

Xiaojun Jia, Jie Liao, Simeng Qin +5

Nanyang Technological University, Singapore · Chongqing University, China · Northeastern University, China +2

White-Box Spectral-Subspace-Guided Attack Multi-Llm Agents

Ask PDF

Jun 1, 2026cs.CL

SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction

Agent skills occupy a privileged position in the agent workflow, as agents are expected to implicitly follow and execute them, rendering third-party skills a vulnerable attack surface. Existing studies have revealed unsafe agent behaviors induced by skill-based attacks, but they primarily evaluate poisoned skills within a single task execution and enumerate harms through ad-hoc risk lists. To bridge these gaps, we introduce SkillHarm, a benchmark of skill-based attacks across the skill-use lifecycle, paired with a systematic taxonomy of skill-relevant risks. SkillHarm evaluates two attack scenarios: Fixed-Payload Poisoning (FPP), where a fixed poisoned skill package directly compromises any task session that invokes it, and Self-Mutating Poisoning (SMP), where an initially benign execution silently mutates persistent skill content, deferring harm until a subsequent reuse. It further defines 12 risk types based on the agent workflow component targeted by the harm: data pipelines, system environments, and agent autonomy. To instantiate these attacks at scale, we build AutoSkillHarm, an automated construction pipeline with coding agents driven by natural-language harnesses. The resulting benchmark contains 879 attack samples across 71 skills. Experiments show that current agents remain vulnerable with attack success rates up to 86.3% in FPP and 69.3% in SMP. Our analysis further reveals a latent risk: many apparent attack failures stem from the agent failing to engage with the poisoned file rather than genuine resistance, and current defenses still fail to reliably mitigate the threat.

Yuting Ning, Zhehao Zhang, Yash Kumar Lal +8

The Ohio State University · Amazon AGI · Stanford University

Malicious Agents Agentic Workflows Lifecycle

Ask PDF