Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal Large Language Models (MLLMs) offer enhanced visual perception and reasoning, their agentic deployment faces three challenges: costly data annotation, imprecise action-intent alignment, and inefficient exploration from discarded failed trajectories. To address these, we introduce Iron, an intent-aligned, self-improved, and annotation-efficient framework for training GUI agents. Iron employs a novel dual learning strategy that utilizes a stepwise cycle-consistent (SCC) reward to achieve fine-grained alignment between low-level actions and high-level intents, thereby improving instruction grounding and intent understanding. Concurrently, Iron introduces a hindsight reproduction mechanism to repurpose failed trajectories for training, improving both learning efficiency and task diversity. Extensive experiments demonstrate that Iron-trained generalist agents consistently improve performance on cross-environment and cross-device tasks, outperforming models trained with three times more data. Iron also achieves a substantial 25.06% relative improvement on unseen web tasks, with further gains observed on inherently complex tasks, demonstrating the feasibility of building more capable virtual agents.
Figures & tables
Figure 1: (a) Scarcity of successful trajectories in GPT-4V’s annotation of VisualWebArena. (b) In 50 sampled GPT-4o trajectories, failures (right bar) vastly outnumber successes (left bar). (c) Comparison between the ‘Mechanical Imitation’ of traditional agents and an Iron-trained agent in both similar and unseen scenarios.
Figure 2: High-level overview of our Iron framework. (a) We first define two complementary and mutually promoting tasks from the Dual Learning strategy. (b) Then we achieve fine-grained Intent-Action alignment by introducing a stepwise cycle-consistent reward mechanism. (c) Thirdly, in the process of MCTS, we used hindsight reproduction to transform failed trajectories into high-diversity, actionable training signals fully. Finally, Iron continuously optimizes the model through self-training. The illustration features an expansion of two child nodes for exemplary purposes.
Figure 3: An example illustrating the positive (left) and negative (right) filtering of trajectories using five meticulously designed metrics. By filtering either already collected successful or failed trajectories (after modifying their objectives), we ensure dataset quality and achieve further improvements in accuracy.
Category
Agent
OSWorld
AndroidWorld
OS
Office
Daily
Profess
Workflow
Avg.
Avg.
Closed-source
GPT-4o
8.33%
3.58%
6.07%
4.08%
5.58%
5.03%
29.52%
GPT-4V
12.50%
1.86%
7.58%
4.08%
6.04%
5.26%
30.47%
Gemini-ProV
8.33%
3.58%
6.55%
16.33%
2.08%
5.80%
21.40%
Gemini-Pro-1.5
12.50%
6.99%
2.71%
6.12%
3.60%
5.40%
20.80%
Claude-3-Opus
4.17%
1.87%
2.71%
2.04%
2.61%
2.42%
24.70%
Table 1: Model Performance on OSWorld and AndroidWorld. Bold values indicate the best performance among open-weight models. IL denotes imitation learning baselines. Iron variants use 10k, 20k, and 30k self-training samples. Full uses 3 × IL data for SFT, and DPO uses Step-DPO. Detailed baseline settings and statistical values are provided in Appendix F.2 and F.3.
Agent
VisualWebArena
Classifieds
Reddit
Shopping
Overall
GPT-4o
11.31%
13.67%
14.33%
13.25%
GPT-4V
10.26%
13.88%
12.16%
12.54%
Qwen3-VL-8B-IL
12.45%
14.86%
15.62%
14.31%
Qwen3-VL-8B- Iron-10k
13.28%
15.73%
16.44%
15.15%
Qwen3-VL-8B- Iron-20k
14.16%
16.82%
17.53%
16.25%
Table 2: Model Performance Comparison on VisualWebArena. Due to space constraints, the model configurations in the table follow those in Section 4.2 .
Agent
Method
OSworld
AndroidWorld
VWA
OS-Atlas-IL
Baseline
2.97%
17.14%
7.94%
OS-Atlas-20k
Non-Iterative
4.46%
19.39%
8.83%
OS-Atlas- Iron-20k
Iterative
4.76%
19.51%
9.05%
OS-Atlas-30k
Non-Iterative
5.03%
19.49%
9.01%
OS-Atlas- Iron-30k
Iterative
5.95%
21.13%
9.93%
Table 3: Performance Comparison of Iterative and Non-Iterative Self-Training. Models labeled with Iron employ our continuous iterative training procedure, whereas other models are trained in a non-iterative, one-shot manner.
Figure 4: Reliability analysis of SCC reward and intent understanding. (a) Improvements in SCC reward, intent understanding, independent validation metrics, and downstream performance across self-training iterations. (b) Correlation between SCC scores and CLIP-based instruction–intent similarity, providing semantic validation of SCC reward.
Figure 5: A qualitative analysis showing superior intent understanding of our Iron -trained model over baselines.
Model
OSWorld
AndroidWorld
a. Iron
5.95%
21.13%
b. w/o Dual Learning.
5.65% ( ↓ 5.0%)
20.01% ( ↓ 5.3%)
c. w/o SCC Reward.
5.35% ( ↓ 10.1%)
19.66% ( ↓ 7.0%)
d. w/o Hindsight Reprod.
5.06% ( ↓ 15.0%)
18.79% ( ↓ 11.1%)
e. w/o Trajectory Filter.
2.08% ( ↓ 65.0%)
10.94% ( ↓ 48.2%)
Table 4: Ablation study of Iron components on OSWorld and AndroidWorld. Parentheses show performance drop relative to the full model (row a).
Building generalist embodied agents capable of solving complex real-world tasks remains a fundamental challenge in AI. Multimodal Large Language Models (MLLMs) have significantly advanced the reasoning capabilities of such agents through strong vision-language knowledge and chain-of-thought (CoT) reasoning, yet remain brittle when faced with challenging out-of-distribution scenarios. To address this, we propose Verifier-Guided Action Selection (VegAS), a test-time framework designed to improve the robustness of MLLM-based embodied agents through an explicit verification step. At inference time, rather than committing to a single decoded action, VeGAS samples an ensemble of candidate actions and uses a generative verifier to identify the most reliable choice, without modifying the underlying policy. Crucially, we find that using an MLLM off-the-shelf as a verifier yields no improvement, motivating our LLM-driven data synthesis strategy, which automatically constructs a diverse curriculum of failure cases to expose the verifier to a rich distribution of potential errors at training time. Across embodied reasoning benchmarks spanning the Habitat and ALFRED environments, VeGAS consistently improves generalization, achieving up to a 36% relative performance gain over strong CoT baselines on the most challenging multi-object, long-horizon tasks.
Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottlenecked by two fundamental issues: current data synthesis methods for GUI Agents rely on specific environments and struggle to generate diverse data, while existing evaluators either suffer from limited scalability or provide inaccurate and unreliable reward signals. To overcome these challenges, we introduce GSAR (Goal-State-Anchor Reward), a RL reward framework that supports scalable task generation and delivers reliable reward signals for stable and efficient policy optimization. Our approach features self-evolving data synthesis, which produces multiple environments through task execution and generates diverse tasks and goal states. Complementing this, a state-anchor mechanism automatically annotates task-relevant UI elements in successful goal states as reference anchors. During RL training, these reference anchors provide accurate, scalable reward signals that substantially enhance efficiency. Extensive evaluations demonstrate that our framework achieves over 90% accuracy on offline trajectory verification and performs closest to rule-based methods. Furthermore, agents trained using our reward framework exhibit strong performance on both AndroidWorld and our constructed benchmark, establishing a scalable approach for GUI agent training.
Vision-language models (VLMs) have shown strong perception and reasoning abilities for instruction-following embodied agents. However, despite these abilities and their generalization performance, they still face limitations in environmental understanding, often failing on interactions or relying on environment metadata during execution. To address this challenge, we propose a novel framework named Environmental Understanding Embodied Agent (EUEA), which fine-tunes four core skills: 1) object perception for identifying relevant objects, 2) task planning for generating interaction subgoals, 3) action understanding for judging success likelihood, and 4) goal recognition for determining goal completion. By fine-tuning VLMs with EUEA skills, our framework enables more reliable task execution for instruction-following. We further introduce a recovery step that leverages these core skills and a group relative policy optimization (GRPO) stage that refines inconsistent skill predictions. The recovery step samples alternative actions to correct failure cases, and the GRPO stage refines inconsistent skill predictions. Across ALFRED tasks, our VLM significantly outperforms a behavior-cloning baseline, achieving an 8.86% improvement in average success rate. The recovery and GRPO stages provide an additional 3.03% gain, further enhancing overall performance. Finally, our skill-level analyses reveal key limitations in the environmental understanding of closed- and open-source VLMs and identify the capabilities necessary for effective agent-environment interaction.