AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents

Yutao Luo; Haotian Zhu; Shuchao Pang; Zhigang Lu; Tian Dong; Yongbin Zhou; Minhui Xue

cs.CRMar 24, 2026

AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents

Authors: Yutao Luo, Haotian Zhu, Shuchao Pang, Zhigang Lu, Tian Dong, Yongbin Zhou, Minhui Xue

Organizations: Nanjing University of Science and Technology, China · Macquarie University, Australia · Western Sydney University, Australia · the University of Hong Kong, China ·

Abstract

The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.

Figures & tables

Fig. 1 : Overview of our backdoor framework. The framework leverages a compromised MLLM to execute adversary-intended actions when triggered by benign-looking notifications. By leveraging contact details exposed in recent data breaches (e.g., the Optus incident [ 1 ] ), adversaries can easily target specific users to deliver these notification triggers.

Fig. 2 : Overview of AgentRAE: First, we build poisoned data along task navigation in two phases. Then, we apply two-phase training: Phase 1 separates trigger representations, while Phase 2 injects multi-target backdoors with utility preserved. Finally, the notification trigger activates the adversary-intended action from the action space supported by existing mobile GUI agents.

Actions	Argument(s)	Functionality
COMPLETE	–	Mark that the instruction is completed.
CLICK	(x,y)	Click the on-screen position (x,y) .
TYPE	text	Type the given text .
SCROLL	direction	Scroll the screen towards the given direction .
HOME	–	Go to the home screen.
BACK	–	Go to the previous screen.

TABLE I : Arguments and functionality of actions in the GUI agent action space.

Fig. 3 : Examples of poisoned sample construction in two phases. Phase 1: Only poisoned samples with different notification triggers for contrastive learning. Phase 2: Mix clean, poisoned, and benign icon samples for backdoor fine-tuning.

		Clean Task		ASR (%)
Model	Methods	AMS	SR	Avg.	COMPL.	HOME	CLK.
SeeClick- aitw	Clean	62.14	11.97	0.00	0.00	0.00	0.00
	AEIA	62.08	11.52	2.18	1.85	0.05	4.63
	pop-ups	62.08	11.52	2.25	1.92	0.08	4.76
	Scenario-BadNets	61.87	10.77	72.09	63.96	69.84	83.12
	AgentRAE	59.54	10.94	95.59	96.82	95.70	94.23

TABLE II : Comparison of clean task performance (AMS, SR) and multi-target backdoor ASR across three triggers. ASR columns report the average and per-target rates ( COMPL. : COMPLETE, HOME , CLK. : CLICK).

Model	Methods	AMS	$\mathrm{ASR}_{\text{avg}}$	FPR	ASR by action type (%)
Model	Methods	AMS	$\mathrm{ASR}_{\text{avg}}$	FPR	COMPLETE	HOME	CLICK	TYPE	SCROLL	BACK	IMPOSSIBLE	RECENT	LONG PRESS	ENTER
SeeClick-aitw	Clean	62.14	0.00	0.00	0	0	0	0	0	0	0	0	0	0
	Scenario-BadNets	61.92	92.24	88.00	97.64	85.44	98.15	66.30	92.90	96.62	none	none	none	99.13
	AgentRAE (Ours)	58.17	99.58	11.00	100.00	99.20	100.00	98.32	99.78	99.17	none	none	none	100.00
OdysseyAgent-app	Clean	57.16	0.00	0.00	0	0	0	0	0	0	0	0	0	none
	Scenario-BadNets	56.12	19.60	18.95	16.10	36.88	46.62	12.93	8.89	13.96	24.85	21.26	18.28	none

TABLE III: Per-action ASR (%) under nine notification triggers.

\mathrm{ASR}_{\text{avg}}

is the mean over the nine action types; AMS measures clean-task utility; FPR denotes the false positive rate on clean samples with benign notifications.

Model	Methods	#Trigger=3		#Trigger=9
Model	Methods	AMS	ASR	AMS	ASR
SeeClick-aitw	Clean	62.14	/	62.14	/
SeeClick-aitw	AgentRAE	59.54	95.59	58.17	99.58
OdysseyAgent-random	Clean	74.88	/	74.88	/
OdysseyAgent-random	AgentRAE	71.25	90.19	70.04	89.25
OdysseyAgent-app	Clean	57.16	/	57.16	/

TABLE IV : Impact of increasing trigger numbers on AMS and ASR.

Method	OdysseyAgent-Random		OdysseyAgent-App
Method	AMS	ASR	AMS	ASR
Two-phase	71.25	90.19	56.61	95.87
Phase 2 Only	72.51	82.09 (-8.10)	60.17	86.01 (-9.86)
Method	OdysseyAgent-Task		Average
Method	AMS	ASR	AMS	ASR
Two-phase	56.00	91.93	61.29	92.00

TABLE V : Comparison of Two-phase Poisoning and Phase 2 Only. Metrics are AMS / ASR (%). Average is calculated over the three tasks.

$\mathcal{S}_{sil}$	$\mathcal{P}_{c}$	$\mathcal{P}_{a}$
0.03 (w/o Phase 1)	74.88	82.09
0.35	73.40	73.53
0.72	72.96	77.67
0.98	73.06	90.19

TABLE VI : Impact of trigger separation degree in Phase 1 on final attack performance.

\mathcal{S}_{sil}

: separation degree (Silhouette Score);

\mathcal{P}_{c}

: clean task performance (AMS) after Phase 1;

\mathcal{P}_{a}

: attack success rate (ASR) after Phase 2.

† Requires enumerating all target labels, infeasible for open-ended agent outputs. $\mathcal{T}$ : one 7B MLLM fine-tuning pass ( $\sim$ 2 GPU-hours on A800).
Defense Mechanism	NC	MOTH	ABS	MMBD	MNTD	Fine-Tuning	Fine-Pruning	NAD
No Class Enumeration	$\times$	$\times$	$\times$	$\times$	$\surd$	$\surd$	$\surd$	$\surd$
Applicability to MLLM	$\times$	$\times$	$\times$	$\times$	$\times$	$\surd$	$\surd$	$\surd$
Computational Cost	N/A †	N/A †	N/A †	N/A †	$>$ 1000 $\mathcal{T}$	1 $\mathcal{T}$	2 $\mathcal{T}$	3 $\mathcal{T}$
Multi-Target Defense	$\times$	$\times$	$\times$	$\times$	$\times$	$\surd$	$\surd$	$\surd$

TABLE VII : Comparison of existing defense methods for vision-language models.

Method	AMS (%)	ASR (%)
Clean	74.88	0
Backdoored	71.25	90.19
Fine-Pruning	70.00	88.87
Fine-Tuning	73.91	86.42
NAD	73.40	89.20

TABLE VIII : Performance of different defense strategies against backdoor attacks.

Fig. 4 : t - SNE visualization of trigger representations before and after Phase 1 contrastive learning.

Panel of Figure 13 — Fig. 5 : Backdoor robustness on OdysseyAgent-app: poison size and resolution.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 6 : Motivating Example 1: Notification-triggered task redirection on a mobile GUI agent.

Fig. 7 : Motivating Example 2: Notification-Triggered Denial-of-Service (DoS) on a Mobile GUI Agent.

Yutao Luo is currently pursuing the M.S. degree with the School of Cyber Science and Engineering, Nanjing University of Science and Technology. He received the bachelor’s degree in Computer Science and Technology from Ningbo University. His research interests include AI application security.

Appendix

Table 16

Haotian Zhu is currently pursuing the Ph.D. degree with the School of Cyber Science and Engineering, Nanjing University of Science and Technology. He received the bachelor’s degree from the School of Computing and Artificial Intelligence, Southwest Jiaotong University. His research interests include AI application security and privacy protection.

Appendix

Table 17

Shuchao Pang is currently a full professor at Nanjing University of Science and Technology, Nanjing, China, and also an Honorary Research Fellow at Macquarie University. Prior to that, he was a Research Associate with the University of New South Wales, Sydney, and a Postdoctoral Researcher with the Ingham Institute for Applied Medical Research, Sydney. He received his Ph.D. degree in computer science from Macquarie University, Sydney, NSW, Australia. His research interests include AI Security, Data Security and Privacy Protection, AI Agent Security, and Machine Learning.

Appendix

Table 18

Zhigang Lu is a Senior Lecturer with Western Sydney University, Australia. He received the BEng degree from Xidian University, the MPhil and PhD degrees from the University of Adelaide, all in computer science. Zhigang is a Consulting Area Editor for IEEE TIFS, an Associate Editor for IEEE TDSC and a PC member/meta reviewer/regular reviewer for many renowned international journals/conferences. With research interests in data privacy and machine learning, he has published over twenty papers in international journals/conferences, including IEEE TIFS, IEEE TDSC, IEEE TNNLS, ACM CCS, Usenix Security, NDSS, PETS, RAID, NeurIPS and AAAI.

Appendix

Table 19

Tian Dong is currently a postdoctoral fellow at the University of Hong Kong. He received the B.A., M.E. and Ph.D. degree from Shanghai Jiao Tong University, China, in 2019, 2022, and 2025. His research interests include the intersection of security, privacy, and machine learning.

Appendix

Table 20

Yongbin Zhou is currently a full Professor with the Nanjing University of Science and Technology, Nanjing, China, and also a Visiting Research Fellow with the State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, China. Prior to that, he was a full professor of the State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences and a professor with the School of Cyber Security, University of Chinese Academy of Sciences. His main research interests include theories and technologies of network and information security.

Appendix

Table 21

Minhui Xue is a Senior Research Scientist and lead of the AI Security team at CSIRO’s Data61, Australia. His research focuses on AI security and privacy, system and software security, and Internet measurement. He is a recipient of the IEEE TCSC Award for Excellence (Middle Career Researcher) and the Science Excellence Award at CSIRO’s Data61. He has received distinguished paper awards at NDSS 2025, USENIX Security 2024, FSE 2023, CCS 2021, and ASE 2018. He has also been recognized as a distinguished reviewer by NDSS 2024 and FSE 2023, and his work has been featured in the mainstream press including The New York Times and The Australian Financial Review. He is an Associate Editor for IEEE TIFS and TDSC.

Appendix

Table 22

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
May 27, 2026Ruoqi Guo, Yi Liu, Gelei Deng +7Mobile GUI Automation GUI Agents
1Griffith University · 2Quantstamp · 3Nanyang Technological University +4
Mind the Gap: Action Rebinding Attacks against Android GUI Agents
Jan 18, 2026Yi Qian, Kunwei Qian, Xingbang He +7Mobile GUI Automation Adversarial Attacks
State Key Laboratory for Novel Software Technology, Nanjing University · Hornor Device Co., Ltd · Institute of Dataspace, Hefei Comprehensive National Science Center China
UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents
May 28, 2026Yuxiang Chai, Han Xiao, Xinyu Fu +3Mobile GUI Automation GUI Agents
1CUHK MMLab · 2Huawei Research · 3Shenzhen Loop Area Institute +1

May 27, 2026J/K move · Enter open · S save

MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content

Ruoqi Guo, Yi Liu, Gelei Deng, Yiheng Xiong+6

1Griffith University · 2Quantstamp · 3Nanyang Technological University+4

Mobile GUI Automation GUI Agents AI Agent Security Benchmarks Mobile GUI Agents Prompt Injection Attacks Prompt Injection Attacks on AI Agents

Open paper AskPDF arXiv

Mobile graphical user interface (GUI) agents driven by vision-language models (VLMs) perceive the screen as rendered pixels and choose actions from what they see, so they cannot reliably separate trusted interface elements from user-generated content. We present MIRAGE (Mobile Injection of Realistic Adversarial GUI Examples), a pipeline that turns benign mobile screenshots into prompt-injection samples by placing attacker-controlled text into ordinary user-generated content regions, without modifying the agent, the application, or the operating system. MIRAGE operates in three stages: a Localizer identifies user-controllable regions on the screenshot, a Generator synthesises context-aware payloads and renders them in the application's native style, and a Curator moderates realism and balances the samples across applications, region types, and attack intents. A key challenge is that an injected screenshot must stay visually indistinguishable from genuine user content while still diverting the agent; we address this by separating the stages that control reach, realism, and distributional balance. On a 1,111-sample benchmark spanning ten applications and eleven attack intents, all five evaluated VLM agents are vulnerable, with attack success rates of 23%-30%, and MIRAGE scores higher on human realism ratings than the strongest prior attack (3.02 versus 2.52 out of 5). We further find that per-sample realism and attack success are uncorrelated, so visual-quality filtering alone cannot reliably defend against this threat.

Abstract

Backdoor Attacks

Explore similar work

MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content

Mind the Gap: Action Rebinding Attacks against Android GUI Agents

UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents