Recent advances in multimodal agents have improved computer-use interaction and tool-usage, yet most existing systems remain reactive, optimizing actions in isolation without reasoning about future states or long-term goals. This limits planning coherence and prevents agents from reliably solving high-level, multi-step tasks. We introduce TraceR1, a two-stage reinforcement learning framework that explicitly trains anticipatory reasoning by forecasting short-horizon trajectories before execution. The first stage performs trajectory-level reinforcement learning with rewards that enforce global consistency across predicted action sequences. The second stage applies grounded reinforcement fine-tuning, using execution feedback from frozen tool agents to refine step-level accuracy and executability. TraceR1 is evaluated across seven benchmarks, covering online computer-use, offline computer-use benchmarks, and multimodal tool-use reasoning tasks, where it achieves substantial improvements in planning stability, execution robustness, and generalization over reactive and single-stage baselines. These results show that anticipatory trajectory reasoning is a key principle for building multimodal agents that can reason, plan, and act effectively in complex real-world environments.
Figures & tables
Figure 1: TraceR1 overview: anticipatory planning and grounded execution. The planning model takes a user instruction together with the current screenshot and interaction history, predicts a trajectory of future actions, where only the first predicted step is executed by the tool agent while later steps are unexecuted lookahead predictions , and generates step-level instructions for tool agents to complete tasks across diverse GUI environments and tool-use platforms.
Figure 2: Two-stage training framework of TraceR1. Stage 1 performs anticipatory trajectory optimization using trajectory-level alignment rewards, while Stage 2 applies grounded RL fine-tuning with step-level rewards derived from tool-agent execution feedback.
Agent Model
Params
AndroidWorld
OSWorld-Verified
Proprietary Models
o3 [ 29 ]
-
-
23.0
OpenAI CUA-o3 [ 29 ]
-
52.5
38.1
Seed1.5-VL [ 14 ]
-
62.1
36.7
Claude 4 Sonnet [ 3 ]
-
-
41.4
Claude 4.5 Sonnet [ 3 ]
-
-
62.9
Table 1: Success rate (%) on AndroidWorld and OSWorld-Verified. OSWorld-Verified is evaluated under a 100-step maximum setting.
Agent Model
Params
AndroidControl-High
GUI Odyssey
Multimodal-Mind2Web
Proprietary Models
GPT-4o [ 29 ]
-
21.2
5.4
4.3
Claude-computer-use [ 3 ]
-
12.5
3.1
52.5
Agent System (w/ Proprietary Models)
OmniParser-v2.0 w/ GPT-4o [ 22 ]
-
58.8
62.0
41.3
UI-TARS-7B w/ GPT-4.1 [ 30 ]
7B w/ -
74.8
89.1
66.0
Table 2: Step success rate (%) on AndroidControl-High, GUI-Odyssey, and Multimodal-Mind2Web. Step SR reflects the proportion of correctly executed steps, where both the predicted action and its arguments (e.g., click coordinates) match the ground truth. Results on Multimodal-Mind2Web are averaged over its cross-task, cross-website, and cross-domain evaluation splits.
Agent Model
Params
GAIA
GTA
AnsAcc
Level 1
Level 2
Level 3
AnsAcc
ToolAcc
CodeExec
Proprietary Models
GPT-4o [ 16 ]
-
33.4
47.1
31.4
11.5
57.1
63.4
95.1
GPT-4.1 [ 16 ]
-
50.3
58.5
50.0
34.6
58.4
65.1
94.3
GPT-5 [ 28 ]
-
59.3
67.9
58.1
46.1
60.9
68.3
98.7
Open-Source Models
Table 3: Results on the GAIA and GTA benchmarks. GAIA assesses AI assistants across three difficulty levels, where final answer accuracy ( AnsAcc ) reflects overall tool-usage reasoning correctness. GTA further evaluates multimodal tool-use ability using three metrics: AnsAcc for answer correctness, ToolAcc for accurate tool selection and summarization, and CodeExec for the percentage of generated code that executes without errors.
Setting
AndroidWorld
OSWorld-Verified
GTA
w/ Stage 2
64.8
41.2
56.7
w/o Stage 2
57.2
36.3
50.2
Figure 3: Two-stage Training Ablation. Performance (%) comparison on AndroidWorld, OSWorld-Verified and GTA benchmarks. Stage 2 provides consistent improvements in both settings.
Figure 4: Example trajectory: coordination between TraceR1 (planner) and UI-TARS-1.5-7B (executor) on OSWorld-Verified.
Figure 5: Ablation results. (a) shows the effect of predictive trajectory horizon length on AndroidWorld. (b–c) show the impact of removing λrep and γ during training on AndroidControl-High and GTA.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Category
Hyperparameter
Value
Actor Optimization
Learning rate
1×10−6
Optimizer
AdamW (bf16)
Weight decay
0.01
Warmup ratio
0
Training steps
143
Max grad norm
1.0
Appendix
Table 4: GRPO training hyperparameters used for both Stage 1 and Stage 2. All reinforcement learning stages share the same GRPO and optimization settings.