Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents
Authors: Yan Zhang, Daiqing Wu, Huawen Shen, Liang Li, Gang Cao, Zhi Gong, Wei Dai, Xiaode Zhang, Can Ma, Yu Zhou
Organizations: Institute of Information Engineering, Chinese Academy of Sciences · Tencent · School of Cyber Security, University of Chinese Academy of Sciences · Department of Psychological and Cognitive SciencesTsinghua University · VCIP & TMCC & DISSec, College of Computer Science, Nankai University
Graphical User Interface (GUI) agents enable the fulfillment of complex user instructions through multi-turn interactions with software environments, requiring step-wise reasoning and long-horizon memory to guide actions and retain task-relevant information, respectively. Recent on-policy self-distillation (OPSD) methods have achieved strong performance on GUI grounding, a foundational subtask for GUI agents, owing to dense token-level supervision from privilege-conditioned self-teachers. However, extending existing OPSD methods to multi-turn GUI agents is hindered by self-teachers' limited privilege-following ability and insufficient privileged guidance. In this paper, we introduce GUI-SD-v2, the next version of GUI-SD, which extends OPSD from GUI grounding to multi-turn GUI interaction and addresses key limitations through a two-stage training framework. Specifically, GUI-SD-v2 first strengthens privilege following by jointly optimizing rollouts with and without privileged guidance from the same GUI states. Furthermore, it selectively distills step-specific reasoning and memory guidance through a privilege-conditioned self-teacher, supporting action decisions and the retention of task-relevant information for subsequent interactions. Extensive experiments on two representative GUI agent benchmarks, AndroidWorld and MobileWorld, show that GUI-SD-v2 compares favorably with existing OPSD baselines while consistently outperforming the evaluated state-of-the-art methods in both Pass@1 and Pass@3 success rates. Code and training data will be publicly released.
Figures & tables
Figure 1: Motivation for GUI-SD-v2. (a) Limited privilege following prevents existing OPSD methods from reliably using privileged guidance for action decisions. (b) Coarse privileges offer insufficient reasoning and memory guidance for multi-step interactions. (c) GUI-SD-v2 improves both privilege following and privileged-guidance quality.
Figure 2: Overview of GUI-SD-v2. Stage 1 Privilege-Following Optimization. (a) collects corrective privileges from GUI trajectories and (b) jointly optimizes original-context and privilege-augmented rollouts, strengthening privilege following for subsequent distillation. Stage 2 Informative Privilege Distillation. (c) transfers reasoning and memory guidance to decision and memory outputs, respectively, through self-distillation.
Method
AndroidWorld
MobileWorld
Pass@1
Pass@3
Pass@1
Pass@3
GUI-Owl-7B ( Xu et al., 2026a )
66.4
–
–
–
DART-GUI-7B ( Lu et al., 2026c )
58.0
–
–
–
UI-Venus-7B ( Gu et al., 2025b )
49.1
–
8.5
–
UI-MOPD-8B ( Lian et al., 2026 )
–
–
12.0
–
MobileForge-8B ( Liu et al., 2026 )
50.9
67.2
10.3
–
Table 1: GUI Agent performance on two representative benchmarks, including AndroidWorld and MobileWorld. We report Pass@1 and Pass@3 success rates (%). Bold indicates the best results.
Method
GUI Capability (Stage 2)
Privilege Following (Stage 1)
GUI Capability (Stage 1)
Baseline
55.2
50.0
40.5
SFT
61.2
55.2
44.8
GRPO
60.3
57.8
44.8
PAPO
67.2
62.9
49.1
Table 2: Comparison of training strategies for privilege-following optimization on AndroidWorld. PAPO denotes our proposed privilege-augmented policy optimization. Evaluation covers GUI capability after Stage 2, privilege-following accuracy after Stage 1, and GUI capability after Stage 1.
Memory
Reason
AndroidWorld
Information Retrieval
Task Completion
✗
✗
49.1
44.4 (12/27)
50.6 (45/89)
✗
✓
61.2
51.9 (14/27)
64.0 (57/89)
✓
✗
56.0
55.6 (15/27)
56.2 (50/89)
✓
✓
67.2
59.3 (16/27)
69.7 (62/89)
Table 3: Ablation of the memory and reasoning guidance components in informative privilege distillation. In our evaluation, the 116 AndroidWorld tasks are grouped into information retrieval (27 tasks) and task completion (89 tasks), with the former placing greater demands on retaining key information across interactions.
Granularity
AndroidWorld
Pass@1
Pass@3
Trajectory-level
51.7
60.3
Step-level
67.2
79.3
Table 4: Effect of privileged-guidance granularity on AndroidWorld. We report Pass@1 and Pass@3 success rates (%).
Teacher
AndroidWorld
Qwen3.5-Plus ( Team, 2026 )
57.8
MiniMax-M3 ( Lai et al., 2026b )
59.5
Gemini3.8-Flash ( Team et al., 2023 )
63.7
Kimi-K3 ( Team et al., 2026 )
67.2
Table 5: Effect of privilege guidance from different teachers on AndroidWorld.
Figure 3: Qualitative analysis of informative privilege distillation. (a) The self-teacher assigns lower probabilities to decision-critical tokens in an erroneous student response, providing a corrective distillation signal. (b) Memory guidance encourages the policy to retain the task-relevant event details, whereas the unguided response retains only the navigation operation.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Stage 1: Privilege-Following Optimization
Learning rate
5×10−6
Learning-rate schedule
Cosine decay to zero
Warmup ratio
0.05
Training epochs
1
Per-device micro-batch size
2
Appendix
Table 6: Training hyperparameters for the two stages of GUI-SD-v2.
Figure 4: Tool definition for GUI-SD-v2 .
Figure 5: Response instructions specifying a structured output with thought, action, tool call, and memory fields.
Graphical User Interface (GUI) grounding maps natural language instructions to the visual coordinates of target elements and serves as a core capability for autonomous GUI agents. Recent reinforcement learning methods (e.g., GRPO) have achieved strong performance, but they rely on expensive multiple rollouts and suffer from sparse signals on hard samples. These limitations make on-policy self-distillation (OPSD), which provides dense token-level supervision from a single rollout, a promising alternative. However, its applicability to GUI grounding remains unexplored. In this paper, we present GUI-SD, the first OPSD framework tailored for GUI grounding. First, it constructs a visually enriched privileged context for the teacher using a target bounding box and a Gaussian soft mask, providing informative guidance without leaking exact coordinates. Second, it employs entropy-guided distillation, which adaptively weights tokens based on digit significance and teacher confidence, concentrating optimization on the most impactful and reliable positions. Extensive experiments on six representative GUI grounding benchmarks show that GUI-SD consistently outperforms GRPO-based methods and naive OPSD in both accuracy and training efficiency. Code and training data are available at https://zhangyan-ucas.github.io/GUI-SD/.
Yan Zhang, Daiqing Wu, Huawen Shen +2
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · VCIP & TMCC & DISSec, College of Computer Science, Nankai University
Improving GUI agents typically relies on behavior cloning on expert trajectories. However, as the current policy deviates from the expert policy, it inevitably encounters policy-induced off-trajectory states during closed-loop execution, i.e., states that fall outside the expert trajectories. Since expert trajectories provide no demonstrations for these unseen states, such states receive no effective supervision, leaving the policy unable to select the correct action. To close this supervision gap, we propose Skill-Guided Continuation Distillation (SGCD), an iterative self-improvement framework. SGCD first runs the plain policy without skill guidance for a few steps to reach realistic off-trajectory states. From these states, a skill-guided policy then completes the task and produces successful continuations, which are mixed with expert trajectories to supply supervision over policy-induced off-trajectory states. The skills are extracted from both successful and failed rollouts, consisting of Continuation Plans, Critical Targets, Failure Traps, and Success Criteria. On OSWorld-Verified, SGCD improves the success rate of three base models from the low-30% range to over 50%, demonstrating its effectiveness and generality.
Zhimin Fan, Hongwei Yu, Yeqing Shen +9
1StepFun · University of Science and Technology Beijing · 4Nanyang Technological University +1
GUI agents are commonly trained offline from successful interaction trajectories. Standard training decomposes each trajectory into prefix-action pairs: the agent predicts an action from the current screen and interaction history, while the subsequent observation is discarded. This removes the rationale of why an action is correct: the evidence often appears only on the subsequent screen. For example, to enable Soft Wrap, the agent should click Edit or View, but nothing reveals this until the menu opens. Without such evidence, standard imitation gives the model little chance of ever sampling and thus learning the correct reasoning. To address this issue, we propose Gated Hindsight Distillation (GHD), which uses the next screenshot as privileged information during training. A student predicts from the observable trajectory prefix, while a parameter-sharing teacher additionally observes the next screenshot and re-scores the student's on-policy responses. We apply distillation only when the student fails and the hindsight-conditioned teacher recovers the demonstrated action. GHD improves task success over GRPO on AndroidWorld and AndroidLab across two vision-language models. The code and checkpoints will be made available.
Weiwei Li, Junzhuo Liu, Tong Chu +2
University of Electronic Science and Technology of China