cs.AIApr 13, 2026

Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization

Authors: Zhixin Lin, Jungang Li, Dongliang Xu, Shidong Pan, Yibo Shi, Yuchi Liu, Yuecong Min, Yue Yao

Organizations: Shandong University · The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · New York University · 

Abstract

Mobile GUI agents powered by Multimodal Large Language Models (MLLMs) can execute complex tasks on mobile devices. Despite this progress, most existing systems still optimize task success or efficiency, neglecting users' privacy personalization. In this paper, we study the often-overlooked problem of agent personalization. We observe that personalization can induce systematic structural heterogeneity in execution trajectories. For example, privacy-first users often prefer protective actions, e.g., refusing permissions, logging out, and minimizing exposure, leading to logically different execution trajectories from utility-first users. Such variable-length and structurally different trajectories make standard preference optimization unstable and less informative. To address this issue, we propose Trajectory Induced Preference Optimization (TIPO), which uses preference-intensity weighting to emphasize key privacy-related steps and padding gating to suppress alignment noise. Results on our Privacy Preference Dataset show that TIPO improves persona alignment and distinction while preserving strong task executability, achieving 65.60% SR, 46.22 Compliance, and 66.67% PD, outperforming existing optimization methods across various GUI tasks. The code and dataset will be publicly released at https://github.com/Zhixin-L/TIPO.

Figures & tables

Explore similar work

CardsList
  1. PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent

    Mar 31, 2026Hongyi Nie, Xunyuan Liu, Yudong Bai +4GUI AgentsMobile GUI Agents

  2. KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation

    Apr 9, 2026Tongbo Chen, Zhengxi Lu, Zhan Xu +13Mobile Agent BenchmarksAI Agent Benchmarks

  3. POPI: Personalizing LLMs via Optimized Natural Language Preference Inference

    Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7LLM PersonalizationPersonalized Language Models