Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal Large Language Models (MLLMs) offer enhanced visual perception and reasoning, their agentic deployment faces three challenges: costly data annotation, imprecise action-intent alignment, and inefficient exploration from discarded failed trajectories. To address these, we introduce Iron, an intent-aligned, self-improved, and annotation-efficient framework for training GUI agents. Iron employs a novel dual learning strategy that utilizes a stepwise cycle-consistent (SCC) reward to achieve fine-grained alignment between low-level actions and high-level intents, thereby improving instruction grounding and intent understanding. Concurrently, Iron introduces a hindsight reproduction mechanism to repurpose failed trajectories for training, improving both learning efficiency and task diversity. Extensive experiments demonstrate that Iron-trained generalist agents consistently improve performance on cross-environment and cross-device tasks, outperforming models trained with three times more data. Iron also achieves a substantial 25.06% relative improvement on unseen web tasks, with further gains observed on inherently complex tasks, demonstrating the feasibility of building more capable virtual agents.
Figures & tables
Figure 1: (a) Scarcity of successful trajectories in GPT-4V’s annotation of VisualWebArena. (b) In 50 sampled GPT-4o trajectories, failures (right bar) vastly outnumber successes (left bar). (c) Comparison between the ‘Mechanical Imitation’ of traditional agents and an Iron-trained agent in both similar and unseen scenarios.
Figure 2: High-level overview of our Iron framework. (a) We first define two complementary and mutually promoting tasks from the Dual Learning strategy. (b) Then we achieve fine-grained Intent-Action alignment by introducing a stepwise cycle-consistent reward mechanism. (c) Thirdly, in the process of MCTS, we used hindsight reproduction to transform failed trajectories into high-diversity, actionable training signals fully. Finally, Iron continuously optimizes the model through self-training. The illustration features an expansion of two child nodes for exemplary purposes.
Figure 3: An example illustrating the positive (left) and negative (right) filtering of trajectories using five meticulously designed metrics. By filtering either already collected successful or failed trajectories (after modifying their objectives), we ensure dataset quality and achieve further improvements in accuracy.
Category
Agent
OSWorld
AndroidWorld
OS
Office
Daily
Profess
Workflow
Avg.
Avg.
Closed-source
GPT-4o
8.33%
3.58%
6.07%
4.08%
5.58%
5.03%
29.52%
GPT-4V
12.50%
1.86%
7.58%
4.08%
6.04%
5.26%
30.47%
Gemini-ProV
8.33%
3.58%
6.55%
16.33%
2.08%
5.80%
21.40%
Gemini-Pro-1.5
12.50%
6.99%
2.71%
6.12%
3.60%
5.40%
20.80%
Claude-3-Opus
4.17%
1.87%
2.71%
2.04%
2.61%
2.42%
24.70%
Table 1: Model Performance on OSWorld and AndroidWorld. Bold values indicate the best performance among open-weight models. IL denotes imitation learning baselines. Iron variants use 10k, 20k, and 30k self-training samples. Full uses 3 × IL data for SFT, and DPO uses Step-DPO. Detailed baseline settings and statistical values are provided in Appendix F.2 and F.3.
Agent
VisualWebArena
Classifieds
Reddit
Shopping
Overall
GPT-4o
11.31%
13.67%
14.33%
13.25%
GPT-4V
10.26%
13.88%
12.16%
12.54%
Qwen3-VL-8B-IL
12.45%
14.86%
15.62%
14.31%
Qwen3-VL-8B- Iron-10k
13.28%
15.73%
16.44%
15.15%
Qwen3-VL-8B- Iron-20k
14.16%
16.82%
17.53%
16.25%
Table 2: Model Performance Comparison on VisualWebArena. Due to space constraints, the model configurations in the table follow those in Section 4.2 .
Agent
Method
OSworld
AndroidWorld
VWA
OS-Atlas-IL
Baseline
2.97%
17.14%
7.94%
OS-Atlas-20k
Non-Iterative
4.46%
19.39%
8.83%
OS-Atlas- Iron-20k
Iterative
4.76%
19.51%
9.05%
OS-Atlas-30k
Non-Iterative
5.03%
19.49%
9.01%
OS-Atlas- Iron-30k
Iterative
5.95%
21.13%
9.93%
Table 3: Performance Comparison of Iterative and Non-Iterative Self-Training. Models labeled with Iron employ our continuous iterative training procedure, whereas other models are trained in a non-iterative, one-shot manner.
Figure 4: Reliability analysis of SCC reward and intent understanding. (a) Improvements in SCC reward, intent understanding, independent validation metrics, and downstream performance across self-training iterations. (b) Correlation between SCC scores and CLIP-based instruction–intent similarity, providing semantic validation of SCC reward.
Figure 5: A qualitative analysis showing superior intent understanding of our Iron -trained model over baselines.
Model
OSWorld
AndroidWorld
a. Iron
5.95%
21.13%
b. w/o Dual Learning.
5.65% ( ↓ 5.0%)
20.01% ( ↓ 5.3%)
c. w/o SCC Reward.
5.35% ( ↓ 10.1%)
19.66% ( ↓ 7.0%)
d. w/o Hindsight Reprod.
5.06% ( ↓ 15.0%)
18.79% ( ↓ 11.1%)
e. w/o Trajectory Filter.
2.08% ( ↓ 65.0%)
10.94% ( ↓ 48.2%)
Table 4: Ablation study of Iron components on OSWorld and AndroidWorld. Parentheses show performance drop relative to the full model (row a).