Organizations: Artificial Intelligence Thrust, Information Hub The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Baidu Inc. Beijing, China · Department of Electrical and Computer Engineering University of Alberta, Edmonton, Alberta, Canada
Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) have shown strong potential for automating tasks across diverse digital environments, where reinforcement learning (RL) has become a dominant training paradigm. However, widely used methods such as Group Relative Policy Optimization (GRPO) suffer from reward-gradient misalignment, leading to inefficient and unstable optimization. Recent work addresses this issue by reformulating RL with verifiable rewards (RLVR) as contrastive or classification-based objectives, which improve stability by eliminating problematic gradient behaviors. Despite this progress, existing contrastive RLVR methods rely primarily on outcome-level supervision and fail to capture fine-grained differences in trajectory quality within the same outcome category. In this paper, we propose Length-Aware Contrastive Learning for GUI Agents (LACL-GUI), a contrastive RLVR framework that incorporates trajectory-level quality signals into policy optimization. LACL-GUI introduces structured preferences within both successful and failed trajectories, encouraging concise successful executions and differentiating failure quality based on divergence from successful trajectories, while preserving optimization stability. Experiments on GUI agent benchmarks show that LACL-GUI provides more effective learning signals and consistently improves agent performance over prior methods, highlighting the value of trajectory-level supervision in contrastive RLVR.
Figures & tables
Figure 1: Overview of LACL-GUI framework. A decoupled architecture collects informative trajectory groups and maintains a Success Buffer for reference trajectories. The proposed objective introduces length-aware positive preference and divergence-aware negative refinement on top of REAL contrastive objective.
Model
Max Steps
gimp
calc
impress
writer
os
thunderbird
vlc
vs_code
Overall
Closed-source Models
OpenAI CUA o3 ( 24 )
100
38.5
10.6
10.6
30.4
62.5
26.7
39.2
39.1
27.8
OpenAI CUA ( 24 )
50
34.6
14.9
29.7
26.1
70.8
66.7
11.6
69.6
36.5
Claude 3.7 Sonnet ( 1 )
50
38.5
31.9
36.1
43.5
50.0
53.3
23.5
56.5
40.1
Seed1.5-VL-250717 ( 12 )
100
50.0
34.8
48.9
56.5
39.1
73.3
35.3
56.5
45.1
Claude 4 Sonnet ( 2 )
50
50.0
31.9
46.7
60.9
45.8
73.3
41.2
60.9
47.3
Table 1: Task success rate (%) on the OSWorld benchmark across different application domains. Results are reported following the official OSWorld evaluation protocol. Qwen3-VL-GRPO and Qwen3-VL-REAL denote models trained with GRPO and REAL objectives, respectively, based on the corresponding Qwen3-VL backbones.
Configuration
Success Rate (%)
Gain (%)
Prompt Baseline
42.7
–
+ Decoupled Architecture
45.4
+2.7
+ Success Buffer
46.5
+1.1
+ Trajectory Filter
47.2
+0.7
+ LACL-GUI Objective
50.0
+2.8
Table 2: Ablation study of different components in the LACL-GUI framework. Each row incrementally introduces one additional component over the previous configuration. The prompt-based baseline uses Qwen3-VL-8B-Thinking, while GRPO is adopted for all training configurations except the proposed LACL-GUI objective.
Figure 2: Effect of length-aware preference modeling in LACL-GUI. We compare average trajectory lengths on tasks where all three methods achieve the same outcome.
Figure 3: Sensitivity analysis of the similarity threshold Θ for divergence estimation in LACL-GUI. The default setting ( Θ=0.85 ) is highlighted, and results are reported across different domains.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Number of Tasks
Gimp
26
LibreOffice Calc
47
LibreOffice Impress
47
LibreOffice Writer
23
OS
24
Thunderbird
15
Appendix
Table 3: Distribution of the 222 selected OSWorld tasks across application domains.
Domain
Base
LACL-GUI
REAL
Rel. to REAL
Gimp
36.46
12.08
13.38
−9.7%
Calc
17.50
10.17
16.00
−36.4%
Impress
4.25
4.25
4.25
0.0%
Writer
10.73
5.36
5.27
+1.7%
OS
21.67
4.25
5.50
−22.7%
Thunderbird
7.86
6.71
7.29
−8.0%
Appendix
Table 4: Average trajectory lengths on the 70 tasks successfully completed by all three methods. The last column shows the relative change of LACL-GUI with respect to REAL.
Domain
Base
LACL-GUI
REAL
Rel. to REAL
Gimp
33.80
5.20
25.00
−79.2%
Calc
39.91
30.12
33.97
−11.3%
Impress
30.48
27.86
31.24
−10.8%
Writer
37.83
46.67
42.67
+9.4%
OS
35.67
10.67
22.67
−52.9%
Thunderbird
50.00
26.75
27.75
−3.6%
Appendix
Table 5: Average trajectory lengths on the 93 tasks where all three methods fail. The last column shows the relative change of LACL-GUI with respect to REAL.
Figure 4: LACL-GUI Step 1 (Success)
Figure 5: LACL-GUI Step 2 (Success)
Figure 6: LACL-GUI Step 3 (Success)
Figure 7: LACL-GUI Step 4 (Success)
Figure 8: LACL-GUI Step 5 (Success)
Figure 9: LACL-GUI Step 6 (Success)
Figure 10: A redundant interaction in the long successful trajectory (Step 5).
Figure 11: An erroneous repeated action after the UI state changes (Step 17).
Figure 12: Premature termination after an incomplete state transition (Step 4 in 4-step Failed Trajectory).
Figure 13: Additional ineffective actions after divergence from the successful execution (Step 5 in 6-step Failed Trajectory).