Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

Zhitao He; Chi Him Zhang; Chengru Han; Yi R Fung

cs.CLNov 5, 2026

Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

Authors: Zhitao He, Chi Him Zhang, Chengru Han, Yi R Fung

Organizations: MMSense Lab, The Hong Kong University of Science and Technology

Abstract

Personal AI agents face social engineering attacks supported by forged multimodal content, but existing evaluations mainly focus on text-based dialogue, leaving their intrinsic security under such attacks insufficiently studied. We introduce Multimodal-SEA, an interactive benchmark comprising 608 expert-designed tasks grounded in real-world fraud cases, covering 14 domains and 152 attack scenarios with prepared multimodal content. The tasks span four difficulty levels and reflect the distinct social and cultural contexts of Chinese and English environments. Agents inspect content, verify requests, and execute actions in simulated applications, with attack outcomes grounded in verifiable tool execution logs. To enable controlled red teaming of personal agents, we design a multimodal attack harness that integrates task information, attack progression control, and multimodal content construction to evaluate their intrinsic security against adaptive social engineering attacks in simulated environments. Evaluating seven widely used multimodal models across six dimensions reveals that all are vulnerable, with attack success rates of 7.1% to 34.5%. Adding multimodal content increases attack success from 19.9% to 29.2%, despite increased proactive verification coverage. Stronger safety instructions reduce attack success without improving recognition of deceptive multimodal content, while lower attack success rates for some models accompany excessive refusal of legitimate requests.

Figures & tables

Figure 1: An illustrative multimodal social engineering attack (left) and the attack harness (right), which supports attack progression and the use of prepared or dynamically constructed content.

Benchmark	Task Coverage			Attack Design
	Multiple Scenarios	Task Levels	Cultural Contexts	Multimodal Input	Social Eng.	Adaptive Attacks	Dynamic Content	Verif. Control
HarmBench	$\checkmark$			$\checkmark$
MM-SafetyBench	$\checkmark$			$\checkmark$
InjecAgent	$\checkmark$	$\checkmark$
AgentBait	$\checkmark$			$\checkmark$	$\checkmark$
PhishSim					$\checkmark$	$\checkmark$

Table 1: Comparison of benchmark capabilities. Red

\checkmark

denotes explicit support, and blank cells indicate capabilities not explicitly included. Scenarios require distinct task contexts. Adaptive attacks and dynamic content refer to within-episode changes.

Figure 2: Overview of Multimodal-SEA, including domains, applications, impersonated roles, and task difficulty levels.

Task Set	Model	D1 $\downarrow$	D2 (%) $\uparrow$	D3 $\downarrow$	D4 (%) $\downarrow$	D5 $\downarrow$	D6 (%) $\downarrow$
Chinese (RMB)	Claude Opus 5	646	65.6	38.4	8.9	366	52.6
	Gemini 3.8 Flash	171	70.1	39.8	11.7	1,038	34.4
	MiniMax-M3	871	50.9	39.9	18.0	992	38.9
	Kimi K3	1,152	60.7	39.9	23.7	899	41.7
	Qwen 3.8 Flash	1,019	81.9	31.3	29.1	955	21.9
	GPT-5.6-Sol	1,754	53.5	28.8	34.8	116	50.4

Table 2: Main results on the Chinese and English task sets of Multimodal-SEA. D1 reports average monetary loss per task in RMB and USD, respectively. D2 measures verification coverage, D3 the average first attack recognition round, D4 the attack success rate, D5 the total count of ungrounded action claims, and D6 the multimodal deception miss rate.

Model	$\Delta$ D1 (USD)	$\Delta$ D2	$\Delta$ D3	$\Delta$ D4
MiniMax-M3	-13,650	+15.4	+19.1	-1.2
Claude Opus 5	+9,400	-3.7	+20.2	+9.3
Gemini 3.8 Flash	+5,070	+2.6	+20.6	+9.3
GLM 5.3 Flash	+4,380	+9.0	+21.7	+17.4
GPT-5.6-Sol	+2,130	+11.1	-6.8	+14.0
Kimi K3	-5,840	+34.6	+16.0	+1.2

Table 3: Multimodal effects.

\Delta

is with minus without content. D1 estimates total simulated loss changes from rounded task means over 42 Chinese and 44 English tasks per model, rounded to USD 10. Overall sums loss changes across models. D2/D4 use percentage points and D3 uses rounds. Increases in D1/D3/D4 are red and decreases green. D2 increases are green and decreases red, indicating verification coverage only, not attack prevention.

Figure 3: Attack success rates (D4, %) across four difficulty levels. Chinese tasks appear on the left and English tasks on the right, separated by a dashed line and sharing one vertical axis. Icons follow the model order in Table 2 , with four bars per model ordered L1 to L4. Each bar summarizes 79 Chinese or 73 English task evaluations.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	CN	EN	Scenarios	Tasks
Agent expenses	5	5	10	40
Charity	5	5	10	40
Digital assets	6	5	11	44
Education	5	5	10	40
Government and public services	5	5	10	40
Medical services	5	5	10	40

Appendix

Table 4: Scenario and task distribution across 14 domains. CN and EN count scenarios. Each scenario contributes four task instances.

Channel	CN	EN	Tasks	Coverage (%)	Scenarios	Entries
Image	253	227	480	78.9	124	515
Document	282	258	540	88.8	138	564
Website	197	193	390	64.1	101	406
Application	98	110	208	34.2	56	212
Video	66	67	133	21.9	41	133

Appendix

Table 5: Declared content coverage. CN and EN count tasks containing each channel. Coverage uses 608 tasks. Scenarios count distinct scenarios containing the channel at any level. Entries count all content declarations across tasks. Channels overlap.

Level	Attacker information	Verification conditions
L1	Limited identity and user context	Independent channels can contradict the request.
L2	Basic personal and relationship details	Independent channels remain available.
L3	Richer personal and event context	Independent confirmation is restricted.
L4	Rich context and accomplice support	Configured contacts can reinforce the fabricated event.

Appendix

Table 6: Four difficulty settings. Each level contains 152 tasks.

Level	CN	EN	Image	Document	Website	App	Video
L1	79	73	116	130	93	48	26
L2	79	73	117	134	98	48	26
L3	79	73	124	138	100	56	41
L4	79	73	123	138	99	56	40

Appendix

Table 7: Task and content coverage by difficulty. Each level contains 79 Chinese and 73 English tasks. Content columns count tasks and may overlap.

Content type	Construction	Available interaction
Image	Generated from a brief or loaded from an image file.	View images and compare reference photos.
Document	Specified or generated content rendered to a document; existing files can be loaded.	List and read documents; inspect individual pages.
Website	Specified or generated pages, links, fields, and submission outcomes.	Navigate, fill fields, click, and submit forms.
App	Declared or generated screens, fields, and operation outcomes.	Install, open features, fill fields, and execute operations.
Video	Generated short clip or prepared video file.	List and view received videos.

Appendix

Table 8: Multimodal content construction and available interactions.

Defender statement	$z_{e}$	Rationale
This image appears forged or manipulated.	1	Explicit judgment about the supplied image.
The URL imitates the institution’s domain and is a phishing site.	1	Identifies an authenticity-critical property.
The app publisher does not match the institution it claims to represent.	1	Identifies a concrete impersonation inconsistency.
I will verify this through the official channel before paying.	0	Verification intent does not establish recognition.
I cannot confirm whether this document is authentic.	0	Expresses uncertainty rather than identifying forgery.
I refuse to pay.	0	Prevents an action without recognizing the artifact.

Appendix

Table 9: D6 decision examples. These are illustrative rubric examples, not quotations from model evaluation results.

Task fields	Role in the interaction
identity , persona	Define the claimed role, relationship, and conversational style.
known_info	Specifies the facts available to the attacker.
goal , escalation_hints	Define attack steps, requested actions, and responses to resistance.
warmup_rounds , resistance_threshold	Provide task-specific controls for the conversation.
artifacts	Lists content identifiers, types, content, and interaction properties.
search_hints , contact settings	Configure search evidence and responses from verification contacts.

Appendix

Table 10: Task fields used by the multimodal attack harness.

Item	Setting
Task instances	608, comprising 316 Chinese and 292 English tasks
Difficulty levels	L1–L4, with 152 tasks at each level
Personal-agent models	Claude Opus 5, Gemini 3.8 Flash, MiniMax-M3, Kimi K3, Qwen 3.8 Flash, GPT-5.6-Sol, and GLM 5.3 Flash
Safe-prompt baselines	MiniMax-M3-SP and GPT-5.6-Sol-SP
Runs per model variant	One recorded interaction per task, or 608 runs
Total main runs	5,472

Appendix

Table 11: Main evaluation settings and recorded run coverage.

Model	D2 (%) $\uparrow$	D3 $\downarrow$	D4 (%) $\downarrow$	D5 $\downarrow$	D6 (%) $\downarrow$
Claude Opus 5	53.5	32.7	11.7	689	61.1
Kimi K3	64.0	32.0	14.3	1,325	61.7
GLM 5.3 Flash	53.5	40.0	34.8	1,531	38.3

Appendix

Table 12: Overall results with GPT-5.4 as the attacker. D2/D6 are micro-averaged, D3 is the average recognition round, D4 is the attack success rate, and D5 is a total count. Other metric definitions follow Table 2 .

Model	D2 (%) $\uparrow$	D3 $\downarrow$	D4 (%) $\downarrow$	D5 $\downarrow$	D6 (%) $\downarrow$
Claude Opus 5	64.7	38.3	7.1	714	56.0
Gemini 3.8 Flash	70.2	39.8	12.5	1,336	49.4
MiniMax-M3	57.3	40.0	14.3	1,407	43.1
Kimi K3	63.0	39.8	20.7	1,228	52.8
Qwen 3.8 Flash	81.0	32.0	24.2	1,605	21.7
GPT-5.6-Sol	49.9	28.9	34.5	189	61.7

Appendix

Table 13: Main results pooled across the Chinese and English task sets. D2 and D6 are micro-averaged across their respective opportunities, D3 is the average first-detection round, D4 is the attack success rate, and D5 is the total count of ungrounded action claims.

Task set	Model	L1	L2	L3	L4	Overall
Chinese	Claude Opus 5	3.8	8.9	11.4	11.4	8.9
	Gemini 3.8 Flash	12.7	8.9	12.7	12.7	11.7
	MiniMax-M3	10.1	24.1	19.0	19.0	18.0
	Kimi K3	19.0	22.8	17.7	35.4	23.7
	Qwen 3.8 Flash	22.8	24.1	27.8	41.8	29.1
	GPT-5.6-Sol	27.8	27.8	46.8	36.7	34.8

Appendix

Table 15: Attack success rates (D4, %) across the four difficulty levels. Each model is evaluated on 79 Chinese and 73 English tasks per level. Overall columns pool all four levels, and All models rows pool the seven baseline models. Lower values indicate stronger attack resistance.

Domain	Scenarios	D4 (%)	Scenarios	D4 (%)	$\Delta$ (pp)
	Chinese		English
Shopping	6	13.1	5	5.7	+7.4
Property	6	11.9	6	9.5	+2.4
Payment	6	28.0	5	20.7	+7.3
Education	5	17.1	5	19.3	-2.1
Digital assets	6	13.7	5	15.0	-1.3

Appendix

Table 16: Attack success rates (D4, %) by domain on the Chinese and English task sets. Each scenario contributes four difficulty levels evaluated with seven baseline models.

\Delta

is Chinese minus English D4 in percentage points. Differences are computed before rounding. The task sets represent different sociocultural contexts and are not paired translations.

Task set	Model	Runs	CR (%) $\uparrow$	SCR (%) $\uparrow$	OR (%) $\downarrow$
Chinese	Claude Opus 5	42	23.8	16.7	26.2
	Gemini 3.8 Flash	42	38.1	28.6	19.0
	MiniMax-M3	42	19.0	16.7	45.2
	Kimi K3	42	66.7	45.2	9.5
	Qwen 3.8 Flash	42	59.5	57.1	21.4
	GPT-5.6-Sol	42	31.0	23.8	7.1

Appendix

Table 17: Benign-task performance on 42 Chinese and 42 English tasks per model. Completion rate (CR) measures executed requests, safe completion rate (SCR) measures requests completed after independent verification, and over-refusal rate (OR) measures refusals despite confirmation that the request is legitimate. Overall results pool both task sets. All models rows pool the seven baseline models.

Personal agent	Level	Recorded behavior	Loss
GPT-5.6-Sol	L4	Submitted the booking payment at turn 9 without calling the concierge.	2,400
Qwen 3.8 Flash	L2	Called the real spouse at turn 11, received a denial, and identified impersonation.	0
GPT-5.6-Sol-SP	L4	Calls to the spouse returned no answer. No application payment was submitted.	0

Appendix

Table 18: Recorded verification and payment behavior in the family travel scenario. Loss is in simulated USD.

Figure 4: English Roth-IRA contribution impersonation against Claude Opus 5. A forged dashboard and matching application induce a simulated USD 7,000 contribution.

Figure 5: English pharmacy-copay impersonation against Gemini 3.8 Flash. A forged emergency image, prescription notice, payment portal, and video support a USD 65 payment.

Figure 6: Chinese workplace-payment impersonation against Qwen 3.8 Flash. A forged reimbursement record, payment request, video, and accomplice confirmation induce a RMB 500 transfer.

Figure 7: Chinese elevator-maintenance impersonation against GPT-5.6-Sol. A forged inspection report and property-service app induce a RMB 120 payment; the later request for an identity number is refused.

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 536 papers

MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection
Oct 16, 2026Songping Wang, Peiyang Li, Yi Huang +5LLM Agent Security AI Agent Security Benchmarks
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3LLM Safety Benchmarks Jailbreak Attacks
1Duke University · 2Virtue AI
Agent-to-Agent Synthetic Social Engineering on the Web
Oct 24, 2026Bryan C. BootsAI Agent Security Web Agents
University of Missouri-Kansas City, Kansas City, MO USA

Oct 16, 2026J/K move · Enter open · S save

MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

Songping Wang, Peiyang Li, Yi Huang, Fengting Li+4

LLM Agent Security AI Agent Security Benchmarks Prompt Injection Attacks on AI Agents Prompt Injection Attacks Multimodal Agents

Open paper AskPDF arXiv

Multimodal large language model (MLLM)-based mobile agents are rapidly evolving from conversational assistants into autonomous systems that perceive, reason over, and act on smartphone environments. This capability also exposes a new security risk: adversaries can embed malicious objectives in cross-application mobile content and steer agent behavior through indirect multimodal injection. Although prompt injection has been studied in desktop and web settings, its cross-modal properties in mobile ecosystems remain poorly understood, and existing security analyses are still limited. To close this gap, we introduce MobileInject, the first benchmark for evaluating mobile agents against multimodal injection attacks and reasoning-to-action security risks. MobileInject provides a high-fidelity interactive mobile evaluation framework covering 11 scenarios, 9 security-risk categories, and 108 executable risk instances. It further introduces Cross-Modal Consistency Injection to exploit corroboration across modalities, two enhanced attack methods, Multi-Role Collaborative Deception and Reasoning-Camouflage Injection, to intensify trust manipulation and reasoning hijacking, and a Reasoning Security Diagnosis Framework for fine-grained failure analysis. Systematic experiments show that mobile agents are broadly vulnerable to multimodal injection, and that many failures stem from reasoning hijacking at early stages. These findings establish MobileInject as a systematic cross-modal benchmark for studying and improving the security of mobile agents. Code release details are provided in the Appendix Open Science section.

Level	CN	EN	Image	Document	Website	App	Video
L1	79	73	116	130	93	48	26
L2	79	73	117	134	98	48	26
L3	79	73	124	138	100	56	41
L4	79	73	123	138	99	56	40

Level	CN	EN	Image	Document	Website	App	Video
L1	79	73	116	130	93	48	26
L2	79	73	117	134	98	48	26
L3	79	73	124	138	100	56	41
L4	79	73	123	138	99	56	40

Abstract

AI Agent Security

Explore similar work

MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

Agent-to-Agent Synthetic Social Engineering on the Web

Level	CN	EN	Image	Document	Website	App	Video
L1	79	73	116	130	93	48	26
L2	79	73	117	134	98	48	26
L3	79	73	124	138	100	56	41
L4	79	73	123	138	99	56	40