Recent advances in multimodal agents have improved computer-use interaction and tool-usage, yet most existing systems remain reactive, optimizing actions in isolation without reasoning about future states or long-term goals. This limits planning coherence and prevents agents from reliably solving high-level, multi-step tasks. We introduce TraceR1, a two-stage reinforcement learning framework that explicitly trains anticipatory reasoning by forecasting short-horizon trajectories before execution. The first stage performs trajectory-level reinforcement learning with rewards that enforce global consistency across predicted action sequences. The second stage applies grounded reinforcement fine-tuning, using execution feedback from frozen tool agents to refine step-level accuracy and executability. TraceR1 is evaluated across seven benchmarks, covering online computer-use, offline computer-use benchmarks, and multimodal tool-use reasoning tasks, where it achieves substantial improvements in planning stability, execution robustness, and generalization over reactive and single-stage baselines. These results show that anticipatory trajectory reasoning is a key principle for building multimodal agents that can reason, plan, and act effectively in complex real-world environments.
Figures & tables
Figure 1: TraceR1 overview: anticipatory planning and grounded execution. The planning model takes a user instruction together with the current screenshot and interaction history, predicts a trajectory of future actions, where only the first predicted step is executed by the tool agent while later steps are unexecuted lookahead predictions , and generates step-level instructions for tool agents to complete tasks across diverse GUI environments and tool-use platforms.
Figure 2: Two-stage training framework of TraceR1. Stage 1 performs anticipatory trajectory optimization using trajectory-level alignment rewards, while Stage 2 applies grounded RL fine-tuning with step-level rewards derived from tool-agent execution feedback.
Agent Model
Params
AndroidWorld
OSWorld-Verified
Proprietary Models
o3 [ 29 ]
-
-
23.0
OpenAI CUA-o3 [ 29 ]
-
52.5
38.1
Seed1.5-VL [ 14 ]
-
62.1
36.7
Claude 4 Sonnet [ 3 ]
-
-
41.4
Claude 4.5 Sonnet [ 3 ]
-
-
62.9
Table 1: Success rate (%) on AndroidWorld and OSWorld-Verified. OSWorld-Verified is evaluated under a 100-step maximum setting.
Agent Model
Params
AndroidControl-High
GUI Odyssey
Multimodal-Mind2Web
Proprietary Models
GPT-4o [ 29 ]
-
21.2
5.4
4.3
Claude-computer-use [ 3 ]
-
12.5
3.1
52.5
Agent System (w/ Proprietary Models)
OmniParser-v2.0 w/ GPT-4o [ 22 ]
-
58.8
62.0
41.3
UI-TARS-7B w/ GPT-4.1 [ 30 ]
7B w/ -
74.8
89.1
66.0
Table 2: Step success rate (%) on AndroidControl-High, GUI-Odyssey, and Multimodal-Mind2Web. Step SR reflects the proportion of correctly executed steps, where both the predicted action and its arguments (e.g., click coordinates) match the ground truth. Results on Multimodal-Mind2Web are averaged over its cross-task, cross-website, and cross-domain evaluation splits.
Agent Model
Params
GAIA
GTA
AnsAcc
Level 1
Level 2
Level 3
AnsAcc
ToolAcc
CodeExec
Proprietary Models
GPT-4o [ 16 ]
-
33.4
47.1
31.4
11.5
57.1
63.4
95.1
GPT-4.1 [ 16 ]
-
50.3
58.5
50.0
34.6
58.4
65.1
94.3
GPT-5 [ 28 ]
-
59.3
67.9
58.1
46.1
60.9
68.3
98.7
Open-Source Models
Table 3: Results on the GAIA and GTA benchmarks. GAIA assesses AI assistants across three difficulty levels, where final answer accuracy ( AnsAcc ) reflects overall tool-usage reasoning correctness. GTA further evaluates multimodal tool-use ability using three metrics: AnsAcc for answer correctness, ToolAcc for accurate tool selection and summarization, and CodeExec for the percentage of generated code that executes without errors.
Setting
AndroidWorld
OSWorld-Verified
GTA
w/ Stage 2
64.8
41.2
56.7
w/o Stage 2
57.2
36.3
50.2
Figure 3: Two-stage Training Ablation. Performance (%) comparison on AndroidWorld, OSWorld-Verified and GTA benchmarks. Stage 2 provides consistent improvements in both settings.
Figure 4: Example trajectory: coordination between TraceR1 (planner) and UI-TARS-1.5-7B (executor) on OSWorld-Verified.
Figure 5: Ablation results. (a) shows the effect of predictive trajectory horizon length on AndroidWorld. (b–c) show the impact of removing λrep and γ during training on AndroidControl-High and GTA.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Category
Hyperparameter
Value
Actor Optimization
Learning rate
1×10−6
Optimizer
AdamW (bf16)
Weight decay
0.01
Warmup ratio
0
Training steps
143
Max grad norm
1.0
Appendix
Table 4: GRPO training hyperparameters used for both Stage 1 and Stage 2. All reinforcement learning stages share the same GRPO and optimization settings.
Multimodal agents have achieved notable progress on complex reasoning tasks through tool use, yet remain limited by two issues: statically predefined tool inventories fail to generalize to unseen scenarios, and indiscriminate tool invocation incurs redundant cost and noise-induced errors. We propose MetaForge, a multimodal agent framework that learns when to invoke tools and how to evolve its toolset on demand. MetaForge factorizes agentic behavior into four coupled stages: Decide (judging whether tool use is warranted), Retrieve (selecting suitable tools), Adapt (grounding tool parameters in task context), and Forge (synthesizing new skills online and recycling them into the tool library for reuse), forming a closed judge-retrieve-adapt-forge-recycle loop. A unified orchestration policy enables the agent to choose among answering directly, reusing existing tools, or forging new ones. We jointly optimize invocation necessity, retrieval accuracy, execution effectiveness, and forged-skill reusability via reinforcement learning, with an explicit invocation-cost penalty discouraging redundant calls. Across 12 benchmarks, MetaForge consistently surpasses 16 baselines in accuracy, efficiency, and generalization, validating a paradigm shift from static tool inventories to on-demand self-evolution.
Shouang Wei, Houcheng Min, Xinpeng Dong +8
East China Normal University · Zhejiang University · Tsinghua University +2
Recent advances in multimodal recommenders excel at feature fusion but remain opaque and inefficient decision-makers, lacking explicit reasoning and self-awareness of uncertainty. We introduce ReasonRec, a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline. Specifically, we propose a reasoning-aware visual instruction tuning strategy that systematically transforms diverse recommendation tasks into unified CoT prompts, enabling the VLM to explicitly articulate intermediate decision steps. Additionally, our evidence-horizon curriculum progressively enhances the reasoning complexity to better handle cold-start and long-tail user scenarios, significantly boosting model generalization. Furthermore, the uncertainty-guided delegation mechanism empowers the agent to assess its own confidence, strategically allocating computational resources to optimize both recommendation accuracy and inference efficiency. Comprehensive experiments on four standard recommendation tasks across five real-world datasets demonstrate that ReasonRec achieves over 30% relative improvement in key ranking metrics compared to state-of-the-art multimodal recommenders. Crucially, ReasonRec substantially reduces inference latency by dynamically delegating up to 35% of queries to efficient sub-models without compromising accuracy. Extensive ablation studies further confirm that each proposed reasoning and planning mechanism individually contributes substantially to ReasonRec's overall effectiveness. Collectively, our results illustrate a clear pathway towards interpretable, adaptive, and efficient multimodal recommendation through explicit reasoning and agentic design.
Yihua Zhang, Mingfu Liang, Jiyan Yang +11
Meta AI · Michigan State University · The University of North Carolina at Chapel Hill +1
Language model (LM)-based agents have demonstrated promising capabilities in automating complex tasks from natural language instructions, yet they continue to struggle with long-horizon planning and reasoning. To address this, we propose an enhanced multi-agent framework that decomposes automation into three roles: a planner for high-level decision-making, an actor for task execution, and a memory manager for contextual reasoning. While this modular decomposition aligns with established design patterns, our core contribution lies in a systematic compute-allocation analysis, revealing that planning is the dominant factor influencing task performance. Execution and memory management require significantly less compute and model capacity to achieve competitive results. Building on these insights, we introduce a planner-centric reinforcement learning approach, which exclusively optimizes the planner using trajectory-level rewards from a VLM-as-judge, while freezing the other components. Extensive experiments on benchmarks spanning web navigation, OS control, and tool use demonstrate that concentrating model capacity and learning on high-level planning yields robust and compute-efficient improvements in long-horizon agent automation. Our code is publicly released.