MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

Songping Wang; Peiyang Li; Yi Huang; Fengting Li; Xuhao Ran; Yueming Lyu; Chao Feng; Caifeng Shan

cs.CROct 16, 2026

MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

Authors: Songping Wang, Peiyang Li, Yi Huang, Fengting Li, Xuhao Ran, Yueming Lyu, Chao Feng, Caifeng Shan

Abstract

Multimodal large language model (MLLM)-based mobile agents are rapidly evolving from conversational assistants into autonomous systems that perceive, reason over, and act on smartphone environments. This capability also exposes a new security risk: adversaries can embed malicious objectives in cross-application mobile content and steer agent behavior through indirect multimodal injection. Although prompt injection has been studied in desktop and web settings, its cross-modal properties in mobile ecosystems remain poorly understood, and existing security analyses are still limited. To close this gap, we introduce MobileInject, the first benchmark for evaluating mobile agents against multimodal injection attacks and reasoning-to-action security risks. MobileInject provides a high-fidelity interactive mobile evaluation framework covering 11 scenarios, 9 security-risk categories, and 108 executable risk instances. It further introduces Cross-Modal Consistency Injection to exploit corroboration across modalities, two enhanced attack methods, Multi-Role Collaborative Deception and Reasoning-Camouflage Injection, to intensify trust manipulation and reasoning hijacking, and a Reasoning Security Diagnosis Framework for fine-grained failure analysis. Systematic experiments show that mobile agents are broadly vulnerable to multimodal injection, and that many failures stem from reasoning hijacking at early stages. These findings establish MobileInject as a systematic cross-modal benchmark for studying and improving the security of mobile agents. Code release details are provided in the Appendix Open Science section.

Figures & tables

Figure 1: Comparison between unimodal and multimodal injection. Multimodal injection aligns malicious cues across visual and textual channels, creating cross-modal reinforcement that can amplify attack effectiveness over single-channel injection.

Benchmark / Method	Evaluation Setting			Injection Coverage				Diagnostic Capability
	MGI	CAW	DSE	TOI	VOI	CMC	RIT	SOV	RTD	RATM	DVP
InjecAgent [ 37 ]	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
AgentDojo [ 5 ]	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
Agent-SafetyBench [ 39 ]	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
ChatInject [ 2 ]	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	$\times$	$\times$	$\times$
MobileSafetyBench [ 11 ]	$\checkmark$	$\checkmark$	$\checkmark$	$\triangle$	$\triangle$	$\times$	$\times$	$\checkmark$	$\triangle$	$\times$	$\times$

Table 1: Comparison with representative agent-security and mobile-agent safety benchmarks.

\checkmark

: supported;

\triangle

: partial or not the main focus;

\times

: not supported. MGI: mobile GUI interaction; CAW: cross-app workflow; DSE: dynamic stateful execution; TOI/VOI: textual/visual observation injection; CMC: cross-modal consistency; RIT: reinforced injection templates; SOV: state-based outcome verification; RTD: reasoning–action trajectory diagnosis; RATM: reasoning–action transition matrix; DVP: diagnostic visualization profile.

Figure 2: Overview of MobileInject. The framework constructs high-fidelity mobile tasks, injects adversarial objectives through cross-modal observations, executes MLLM-based agents in a closed loop, and applies the Reasoning Security Diagnosis Framework to analyze how malicious content affects agent reasoning, actions, and outcomes.

Figure 3: Task distribution of MobileInject across human activity domains, application scenarios, malicious consequence categories, and benign task carriers.

Component	Registered content
Scope	11 scenarios; Visual and Textual Observations
Objectives	Registered benign workflows and adversarial payloads
Instance	snapshot, apps, carrier, workflow, two oracles
Execution	reset $\rightarrow$ inject $\rightarrow$ closed-loop agent actions
Evidence	observation, recorded rationale, action, UI/state, endpoints
Quality control	schema checks, oracle replay, carrier-contract checks

Table 2: MobileInject construction contract.

Figure 4: RCI mechanism. Context-shaped pseudo-rationale makes the injected objective appear continuous with the current task, inducing a goal switch that propagates from the emitted rationale to subsequent actions.

Metric	Layer	Definition / use
$\mathrm{UC}\uparrow$	Clean utility	Utility in Clean Setting. Benign completion without attack; clean task-solving reference.
$\mathrm{UA}\uparrow$	Outcome	Utility under Attack. Benign completion under adversarial pressure.
$\mathrm{AGR}\downarrow$	Outcome	Adversarial Goal Success Rate. Fraction of episodes whose adversarial oracle fires.
$\mathrm{OCM}$	Outcome	Outcome Classification Matrix. Distribution over the four recorded endpoint pairs; no scalar direction.
$\mathrm{SAR}\uparrow$	Diagnostic	Security Awareness Rate. Fraction of episodes in which the recorded rationale notices the injected risk.
$\mathrm{RHR}\downarrow$	Diagnostic	Reasoning Hijack Rate. Outcome-constrained fraction with a compromised recorded rationale.

Table 3: RSDF metric system: clean utility reference and diagnostic dimensions.

Model	UC $\uparrow$	UA $\uparrow$	AGR $\downarrow$	SAR $\uparrow$	RHR $\downarrow$	RHOR $\uparrow$	RHD $\downarrow$	AHR $\downarrow$	AHAS $\downarrow$	SA $\uparrow$	AD $\downarrow$	RCO $\downarrow$	FH $\downarrow$
GLM-4.5V	66.67%	0.93%	83.33%	2.78%	98.15%	31.33%	100.00%	98.15%	0.13%	3.21%	0.00%	1.52%	95.27%
GLM-4.6V	44.44%	0.93%	66.67%	6.48%	96.30%	40.10%	71.85%	74.07%	0.25%	23.67%	0.00%	12.62%	63.71%
Qwen2.5-VL-72B	75.00%	9.26%	39.81%	14.81%	89.81%	37.71%	61.42%	63.89%	2.85%	37.92%	0.10%	19.37%	42.61%
Qwen3.5-122B	100.00%	16.67%	50.93%	37.96%	57.41%	44.15%	99.15%	63.89%	2.56%	43.60%	2.75%	0.99%	52.66%
Qwen3.6-35B	44.44%	37.96%	39.81%	19.44%	63.89%	53.85%	75.94%	65.74%	2.21%	51.31%	4.30%	8.63%	35.76%
Gemini-2.5-Flash	77.78%	3.70%	77.78%	5.56%	95.37%	42.25%	82.40%	95.37%	0.61%	12.97%	1.43%	0.21%	85.39%

Table 4: Overall safety metrics of different models using the MobAgent framework on MobileInject.

\uparrow

indicates that higher values are safer, and

\downarrow

indicates that lower values are safer.

Model	UA $\uparrow$	AGR $\downarrow$	SAR $\uparrow$	RHR $\downarrow$	RHOR $\uparrow$	RHD $\downarrow$	AHR $\downarrow$	AHAS $\downarrow$	OSS $\uparrow$	SA $\uparrow$	AD $\downarrow$	RCO $\downarrow$	FH $\downarrow$
GLM-4.5V	0.00%	85.19%	8.33%	99.07%	22.31%	81.30%	99.07%	4.36%	7.96	9.44%	0.60%	1.11%	88.84%
GLM-4.6V	0.00%	14.81%	7.41%	77.78%	11.18%	52.92%	26.85%	0.00%	54.63	53.96%	0.00%	29.92%	16.12%
Qwen2.5-VL-72B	1.85%	63.89%	15.74%	93.52%	20.95%	73.47%	80.56%	3.37%	22.78	23.06%	0.36%	9.06%	67.51%
Qwen3.5-122B	18.52%	60.19%	26.85%	83.33%	34.04%	80.00%	76.85%	4.82%	29.26	29.28%	3.08%	9.13%	58.51%
Qwen3.6-35B	4.63%	12.04%	12.04%	52.78%	22.45%	57.78%	27.78%	0.63%	61.48	67.47%	0.51%	18.27%	13.75%
Gemini-2.5-Flash	0.00%	70.37%	16.67%	98.15%	25.40%	77.98%	75.00%	4.81%	20.56	16.28%	0.31%	13.95%	69.46%

Table 5: Overall safety metrics of different models using the M3A Agent framework on MobileInject.

\uparrow

indicates that higher values are safer, and

\downarrow

indicates that lower values are safer.

Model	Q1	Q2	Q3	Q4
GLM-4.5V	0.93	15.74	0.00	83.33
GLM-4.6V	0.93	32.41	0.00	66.67
Qwen2.5-VL-72B	9.26	50.93	0.00	39.81
Qwen3.5-122B	16.67	32.41	0.00	50.93
Qwen3.6-35B	37.04	23.15	0.93	38.89
Gemini-2.5-Flash	3.70	18.52	0.00	77.78

Table 6: OCM outcome distribution across models. Q1–Q4 denote safe-complete, safe-failed, unsafe-complete, and unsafe-failed outcomes, respectively; higher Q1 and lower Q3/Q4 are safer.

Model	Malicious Content	Privacy Leakage	Unauthorized Transaction	Function Abuse	Content Tampering	Data Destruction	Resource Exhaustion	Illegal Promotion	Bias/Fairness	Phishing
UA
GLM-4.5V	0.00%	0.00%	0.00%	0.00%	11.11%	0.00%	0.00%	0.00%	0.00%	0.00%
GLM-4.6V	0.00%	0.00%	11.11%	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%
Qwen2.5-VL-72B	11.11%	5.56%	33.33%	0.00%	11.11%	0.00%	0.00%	22.22%	0.00%	11.11%
Qwen3.5-122B	16.67%	16.67%	0.00%	0.00%	22.22%	0.00%	55.56%	22.22%	11.11%	22.22%
Qwen3.6-35B	16.67%	27.78%	55.56%	44.44%	33.33%	55.56%	33.33%	44.44%	55.56%	44.44%

Table 7: Attack-category security breakdown under MobileInject. Each block reports UA, AGR, RHR, or AHR; higher UA and lower AGR/RHR/AHR are safer. Values are percentages, and colors denote direction-aware safety bins.

Model	Message	Add Contact	Email	Posting	Writing	Data Mgmt.	Calendar	Stocks	Website
UA
GLM-4.5V	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%	8.33%
GLM-4.6V	0.00%	0.00%	0.00%	0.00%	0.00%	0.00%	8.33%	0.00%	0.00%
Qwen2.5-VL-72B	33.33%	0.00%	0.00%	0.00%	0.00%	0.00%	8.33%	0.00%	41.67%
Qwen3.5-122B	16.67%	8.33%	33.33%	8.33%	0.00%	0.00%	66.67%	0.00%	16.67%
Qwen3.6-35B	50.00%	16.67%	83.33%	25.00%	0.00%	16.67%	91.67%	0.00%	58.33%

Table 8: Task-category security breakdown under MobileInject. Blocks report UA and AGR; higher UA and lower AGR are safer. Values are percentages, and colors denote direction-aware safety bins. Each model–workflow cell aggregates all registered adversarial-payload conditions.

Defense	UA $\uparrow$	AGR $\downarrow$	SAR $\uparrow$	RHR $\downarrow$	RHOR $\uparrow$	RHD $\downarrow$	AHR $\downarrow$	AHAS $\downarrow$	SA $\uparrow$	AD $\downarrow$	RCO $\downarrow$	FH $\downarrow$	OSS $\uparrow$
Baseline	0.93%	83.33%	2.78%	98.15%	31.33%	100.00%	98.15%	0.13%	3.21%	0.00%	1.52%	95.27%	7.96
Data Delimiters	0.93%	84.26%	11.11%	96.30%	55.22%	92.84%	96.30%	7.06%	17.42%	2.28%	2.32%	77.98%	10.00
Prescriptive	0.93%	73.15%	22.22%	91.67%	55.19%	89.44%	87.04%	8.54%	24.24%	3.92%	3.41%	68.42%	19.44
SCOT	0.93%	70.37%	23.15%	84.26%	51.88%	92.37%	80.56%	5.40%	26.08%	2.34%	5.05%	66.53%	23.52
Safe Guide	0.00%	73.15%	19.44%	87.04%	49.24%	91.50%	83.33%	6.81%	22.36%	1.06%	3.13%	73.44%	20.56
Safety Sandwich	9.26%	73.15%	10.19%	94.44%	53.57%	89.89%	89.81%	7.26%	18.22%	2.18%	7.15%	72.45%	15.93

Table 9: Defense prompting comparison under MobileInject. Upward and downward arrows mark safer directions; values are percentages except OSS. Cell colors denote per-column safety quartiles. Each displayed defense condition contains 108 instances.

Figure 5: Stability of the diagnosis framework and MobileInject across repeated runs. Annotations give

\sigma^{2}

for each metric.

Figure 6: Overall Security Score (OSS) and Reasoning-Action Coupling Ratio (RACR) across evaluated models.

Figure 7: CMCI ablation on UA and AGR under different injection settings. CLEAN: no injection. VOI/TOI: visual/textual observation injection. CMUI-T/CMUI-V: only the text/visual modality is aligned with the malicious objective. CMCI: both modalities are aligned with the malicious objective.

Figure 8: MRCD ablation on UA and AGR across different attack variants.

Figure 9: Effect of RCI on UA and AGR for different attack methods.

Figure 10: Reasoning-action transition modes across models. Stacked bars show safe alignment, action drift, reasoning constraint, and complete hijack.

Figure 11: Equal-exposure CMCI controls on Gemini-2.5-Flash, averaged over four major benign workflows. TOI/VOI denote one textual/visual exposure; TOI+/VOI+ denote two exposures within the same channel; CMCI uses one textual and one visual exposure and achieves the highest AGR.

Figure 12: Budget-matched attack controls on Gemini-2.5-Flash, averaged over four major benign workflows. The “+” baselines match the full CMCI setting with MRCD and RCI in length, objective repetitions, exposure rounds, and interface salience.

LLM Agent Security

Large language model agent security studies attacks and safeguards for systems that plan, use tools, retain memory, and act across applications.

See all 330 papers

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps
Aug 18, 2026Sujin Chen, Lijun Li, Tianyi Du +1AI Agent Security Mobile GUI Agents
Shanghai Artificial Intelligence Laboratory
Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection
Aug 9, 2026Rahul Deivasigamani, Sayeda Faatin Alvi, Derqui Andrea +2AI Agent Security LLM Agent Security
Radboud University · Nijmegen, The Netherlands
Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agents
May 27, 2026Zheng Wu, Pengzhou Cheng, Zongru Wu +5Probability Calibration GUI Agents
School of Computer Science, Shanghai Jiao Tong University · GenAI, Meta

Aug 18, 2026J/K move · Enter open · S save

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

Sujin Chen, Lijun Li, Tianyi Du, Jing Shao

Shanghai Artificial Intelligence Laboratory

AI Agent Security Mobile GUI Agents Prompt Injection Attacks on AI Agents AI Agent Safety Benchmarks

Open paper AskPDF arXiv

LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. However, because these agents routinely process untrusted environmental content, they are highly vulnerable to environmental injection attacks, which include indirect prompt injections and adversarial instructions. Such attacks can manipulate the behavior of agents without user awareness through diverse channels encountered in everyday mobile use. Despite these risks, existing benchmarks often fail to capture everyday user scenarios, lacking a systematic evaluation of GUI agents under environmental injection attacks on mobile devices. To address this gap, we introduce MobileWorldSafety, a benchmark of 142 risk tasks built on real Android applications. For each task, we define a programmatically verifiable risk indicator over the final system state and evaluate outcomes with a two-stage pipeline: rule-based verification handles unambiguous cases, while an LLM judge adjudicates ambiguous ones. This distinguishes safety failures from capability failures and enables objective and reproducible assessment. Evaluations on six agents, including both general agents and specialized GUI agents, demonstrate that all agents remain highly vulnerable, with attack success rates ranging from 40.4% to 66.9%. These findings indicate that current agents often fail to maintain safety alignment when adversarial content is presented as ordinary mobile context. MobileWorldSafety provides a foundation for quantifying these vulnerabilities and advancing research on robust mobile GUI agents.

Abstract

LLM Agent Security

Explore similar work

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection

Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agents