Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents
Authors: Yan Zhang, Daiqing Wu, Huawen Shen, Liang Li, Gang Cao, Zhi Gong, Wei Dai, Xiaode Zhang, Can Ma, Yu Zhou
Organizations: Institute of Information Engineering, Chinese Academy of Sciences · Tencent · School of Cyber Security, University of Chinese Academy of Sciences · Department of Psychological and Cognitive SciencesTsinghua University · VCIP & TMCC & DISSec, College of Computer Science, Nankai University
Graphical User Interface (GUI) agents enable the fulfillment of complex user instructions through multi-turn interactions with software environments, requiring step-wise reasoning and long-horizon memory to guide actions and retain task-relevant information, respectively. Recent on-policy self-distillation (OPSD) methods have achieved strong performance on GUI grounding, a foundational subtask for GUI agents, owing to dense token-level supervision from privilege-conditioned self-teachers. However, extending existing OPSD methods to multi-turn GUI agents is hindered by self-teachers' limited privilege-following ability and insufficient privileged guidance. In this paper, we introduce GUI-SD-v2, the next version of GUI-SD, which extends OPSD from GUI grounding to multi-turn GUI interaction and addresses key limitations through a two-stage training framework. Specifically, GUI-SD-v2 first strengthens privilege following by jointly optimizing rollouts with and without privileged guidance from the same GUI states. Furthermore, it selectively distills step-specific reasoning and memory guidance through a privilege-conditioned self-teacher, supporting action decisions and the retention of task-relevant information for subsequent interactions. Extensive experiments on two representative GUI agent benchmarks, AndroidWorld and MobileWorld, show that GUI-SD-v2 compares favorably with existing OPSD baselines while consistently outperforming the evaluated state-of-the-art methods in both Pass@1 and Pass@3 success rates. Code and training data will be publicly released.
Figures & tables
Figure 1: Motivation for GUI-SD-v2. (a) Limited privilege following prevents existing OPSD methods from reliably using privileged guidance for action decisions. (b) Coarse privileges offer insufficient reasoning and memory guidance for multi-step interactions. (c) GUI-SD-v2 improves both privilege following and privileged-guidance quality.
Figure 2: Overview of GUI-SD-v2. Stage 1 Privilege-Following Optimization. (a) collects corrective privileges from GUI trajectories and (b) jointly optimizes original-context and privilege-augmented rollouts, strengthening privilege following for subsequent distillation. Stage 2 Informative Privilege Distillation. (c) transfers reasoning and memory guidance to decision and memory outputs, respectively, through self-distillation.
Method
AndroidWorld
MobileWorld
Pass@1
Pass@3
Pass@1
Pass@3
GUI-Owl-7B ( Xu et al., 2026a )
66.4
–
–
–
DART-GUI-7B ( Lu et al., 2026c )
58.0
–
–
–
UI-Venus-7B ( Gu et al., 2025b )
49.1
–
8.5
–
UI-MOPD-8B ( Lian et al., 2026 )
–
–
12.0
–
MobileForge-8B ( Liu et al., 2026 )
50.9
67.2
10.3
–
Table 1: GUI Agent performance on two representative benchmarks, including AndroidWorld and MobileWorld. We report Pass@1 and Pass@3 success rates (%). Bold indicates the best results.
Method
GUI Capability (Stage 2)
Privilege Following (Stage 1)
GUI Capability (Stage 1)
Baseline
55.2
50.0
40.5
SFT
61.2
55.2
44.8
GRPO
60.3
57.8
44.8
PAPO
67.2
62.9
49.1
Table 2: Comparison of training strategies for privilege-following optimization on AndroidWorld. PAPO denotes our proposed privilege-augmented policy optimization. Evaluation covers GUI capability after Stage 2, privilege-following accuracy after Stage 1, and GUI capability after Stage 1.
Memory
Reason
AndroidWorld
Information Retrieval
Task Completion
✗
✗
49.1
44.4 (12/27)
50.6 (45/89)
✗
✓
61.2
51.9 (14/27)
64.0 (57/89)
✓
✗
56.0
55.6 (15/27)
56.2 (50/89)
✓
✓
67.2
59.3 (16/27)
69.7 (62/89)
Table 3: Ablation of the memory and reasoning guidance components in informative privilege distillation. In our evaluation, the 116 AndroidWorld tasks are grouped into information retrieval (27 tasks) and task completion (89 tasks), with the former placing greater demands on retaining key information across interactions.
Granularity
AndroidWorld
Pass@1
Pass@3
Trajectory-level
51.7
60.3
Step-level
67.2
79.3
Table 4: Effect of privileged-guidance granularity on AndroidWorld. We report Pass@1 and Pass@3 success rates (%).
Teacher
AndroidWorld
Qwen3.5-Plus ( Team, 2026 )
57.8
MiniMax-M3 ( Lai et al., 2026b )
59.5
Gemini3.8-Flash ( Team et al., 2023 )
63.7
Kimi-K3 ( Team et al., 2026 )
67.2
Table 5: Effect of privilege guidance from different teachers on AndroidWorld.
Figure 3: Qualitative analysis of informative privilege distillation. (a) The self-teacher assigns lower probabilities to decision-critical tokens in an erroneous student response, providing a corrective distillation signal. (b) Memory guidance encourages the policy to retain the task-relevant event details, whereas the unguided response retains only the navigation operation.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Stage 1: Privilege-Following Optimization
Learning rate
5×10−6
Learning-rate schedule
Cosine decay to zero
Warmup ratio
0.05
Training epochs
1
Per-device micro-batch size
2
Appendix
Table 6: Training hyperparameters for the two stages of GUI-SD-v2.
Figure 4: Tool definition for GUI-SD-v2 .
Figure 5: Response instructions specifying a structured output with thought, action, tool call, and memory fields.
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · VCIP & TMCC & DISSec, College of Computer Science, Nankai University