Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.
Figures & tables
Figure 1: Enabling long-horizon exploration with a video planner. Left: Random exploration struggles to reach later stages of long-horizon tasks that require specific sequences of actions. Right: The video planner guides exploration. As the V2A model learns from achieved transitions, the agent follows the plan more accurately and reaches progressively later stages of the task. These new transitions further improve the model, eventually enabling the agent to execute the full plan.
Figure 2: Re-planning holds the agent at the milestone until noise supplies the critical action.
Algorithm 1 Learning without action labels: VP-guided self-training of the V2A model
Figure 3: The causal video planner. (a) Tokenized frames and text are fed into the 14B-parameter Cosmos3-Nano ( NVIDIA, 2026 ) model, forming a causal stream of contexts whose keys and values are cached and reused. The denoiser generates K future frames while cross-attending to the context KV cache, so each observation and instruction is encoded only once. (b) Attention mask. Video history is encoded block-causally. The future plan attends to this history and bidirectionally to itself. Sliding-window attention lets the planner generalize beyond the window while keeping the KV cache within a constant budget.
Figure 4: Learning from human demonstration. Mirror can complete a variety of open-ended BASALT tasks from self-play without any expert action labels or reward supervision. (a) The VP inherits long-horizon planning capabilities learned from video pre-training, while the V2A model learns to ground the video plans to action via online interaction. (b) The constructed waterfalls are diverse and similar to human creations. (c) Mirror achieves near-human performance on FindCave.
No action labels
With action labels
Vision-language-action (VLA) models
Heuristic
Human
Task
Mirror (Ours)
VIPER
Mirror (Ours)
KAIROS
BC (VPT-SFT)
UniTeam
STEVE-1
GROOT
JARVIS- VLA
OpenHA
GoUp
FindCave
82%
0%
91%
27%
27%
64%
45%
45%
36%
45%
36% †
95%
MakeWaterfall
36%
0%
45%
0%
0%
0%
0%
0%
0%
0%
36% †
100%
CreateAnimalPen
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
18% †
95%
BuildVillageHouse
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
91% †
100%
Table 1: Solving fuzzy tasks from BASALT. Success rates for methods without action labels (our self-improving method and VIPER ( Escontrela et al., 2023 ) ), action-supervised and heuristic BASALT baselines ( Milani et al., 2022 ) , vision-language-action (VLA) models, and humans.
Figure 5: Minecraft Diamond Challenge. Success rates across twelve acquisition milestones. Offline methods learn from VPT human data without environment interaction. Online methods learn by interacting with the environment for about 10M steps ( ∼ 140 in-game hours) without expert action labels. In the offline setting, Mirror (supervised) sets a new state of the art. In the online setting, Mirror (self-improving) is the only method to reach the iron pickaxe without action labels or rewards. Mirror is evaluated on 128 episodes of 60 minutes with respawning, following VPT and Dreamer V4. Offline baselines are from Dreamer V4 ( Hafner et al., 2025b ) . Online baselines at 10M environment steps are from Dreamer V3 ( Hafner et al., 2025a ) .
Figure 6: Emergent in-context learning from failure. (a) We challenge our agent to escape a lava lake without fine-tuning. (b) Retaining context between deaths causes a directional bias in the agent’s trajectories, indicating in-context adaptation. (c) Shared history allows the agent to learn from its past failures in-context, leading to higher escape rates. Retaining recurrent state does not provide a comparable benefit for VPT BC.
Figure 7: Text-conditioned video planning. We evaluate text-conditioned control on 10 tasks across two unseen environments (see Appendix C.1.4 ). Each task is specified only through a text prompt, with no task-specific fine-tuning. VIPER ( Escontrela et al., 2023 ) is included as a reward-free video-based baseline. Combined with our self-improving V2A approach, the agent achieves high success rates although it has never seen expert demonstrations on these tasks. Left: Success rates of each method. Right: Text guidance leads the agent to complete different tasks.
Figure 8: Imitation learning performance across three benchmark suites. Left: learning curves on 13 DeepMind Control (DMC) tasks, four OGBench tasks, and seven Atari games. Colors identify methods; dashed method curves use an oracle VP and solid method curves use a generative VP. The horizontal dashed line marks expert performance. Curves average tasks equally, with shaded regions showing ±1 standard error across three seeds per task. The Atari VIPER curve is digitized from Escontrela et al. (2023) over its first 1M decisions and rescaled to % of expert. Right: final performance at 1M environment transitions (mean ± SE, equal task weights), separated by planner type.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Variable prediction length training. Prediction windows of variable length over the same 32 s clip. The clean history is encoded once by the context encoder and shared by all prediction windows. With sliding-window attention, the clean history tokens and the noisy plan windows attend to the preceding 12 s and 8 s clean frames, respectively.
Steps
GPUs
Global batch
Learning rate
Wall-clock
0–30k
8 H200
8
1.5×10−5
25 h
30k–80k
16 H200 (2 nodes)
16
2.0×10−5
42 h
80k–215k
32 H200 (4 nodes)
32
2.7×10−5
115 h
215k–220k
8 H200
32
5×10−6 (cooldown)
10 h
Appendix
Table 2: Pretraining schedule of the video planner on VPT dataset.
Figure 10: Rolling cache at inference. (1) Each new observation is encoded once by the context encoder, which attends to the 12 s cache. (2) The resulting keys and values are appended and the oldest context is evicted, keeping the cache size fixed. (3) The denoiser generates the next 1.6 s of video while attending to the last 8 s of cached context and, optionally, to a control-level caption. Cache cells schematically group frames.
Count
Per video (median)
Span (s)
Tokens / caption
Videos
58,121
–
241.0 (duration)
–
Descriptions
13,352,359
244
3.5 [1.0, 8.0]
19.8
Instructions
13,155,310
237
3.5 [1.0, 8.0]
15.2
Task segments
1,506,066
26 (5 partitions)
31.5 [12.0, 62.6]
16.1
Total tokens
489.1M
Appendix
Table 3: Caption corpus statistics for the VPT dataset. Spans are median [10th, 90th percentile]. Tokens are counted with the Cosmos3-Nano tokenizer.
Figure 11: Subgoal task planner. (1) Once per plan, the planner reads the agent’s inventory, health, and whether a GUI is open, and sends a subgoal caption and a guidance scale w to the video planner; the V2A model never sees them. (2) Guidance-scale rules; the dashed one is used only during training. (3) Subgoals in the order the planner picks them from an empty inventory. Each 3 × 3 craft happens at a crafting table placed for it and picked up afterwards.
Figure 12: Video post-processing for the diamond task. (a) The original VPT video recordings do not include the cursor inside GUI menus. (b) As in VPT’s data loader ( Baker et al., 2022 ) , we draw the cursor from the mouse position log. However, VPT’s original 16 × 16 cursor is too small for the Wan2.2 VAE, which blurs or drops it (Figure 13 ). Thus, for the diamond task we draw a high-visibility cursor: VPT’s cursor enlarged 2 × , tinted coral, and outlined in black and white. (c) The VAE also blurs the small item counts or turns them into other digits, so we crop the 184 × 24-pixel hotbar from the video, enlarge it 3 × with nearest-neighbor upsampling, and paste it back at the bottom center, moving the health and food bars up. (d) Finally, we resize the 640×360 frames to 320×192 with bilinear interpolation and antialiasing. Of the filters we tried, it is the blurriest but keeps all task-relevant detail and causes the fewest VAE artifacts (reconstruction PSNR 32.7 dB, versus 24.5–30.4 dB for bilinear without antialiasing, bicubic, area, and Lanczos).
Figure 13: The Wan2.2 VAE loses the VPT cursor. The same frame with (a) VPT’s cursor and (b) the high-visibility cursor, before and after a round trip through the Wan2.2 VAE. VPT’s small cursor nearly vanishes in the reconstruction, which is why we draw a larger, outlined cursor for GUI-heavy tasks.
Method
Inputs
Outputs
Minecraft Data (hours)
Offline
Web
Online
Dreamer V3
Video, inventory state
Keyboard, camera, abstract crafting
–
–
1.4K
Dreamer V4
Video
Keyboard, mouse
2.5K
–
–
VPT (BC)
Video
Keyboard, mouse
2K
70K
–
VPT (RL)
Video
Keyboard, mouse
2K
70K
230K
Video Planner (Ours)
Video, text prompt (optional)
Video
3K
–
–
Appendix
Table 4: Data scale of different Minecraft agents
Milestone
PPO
Rainbow
IMPALA
Dreamer V3
Mirror (self-improving)
Log
75.2
74.4
71.0
95.0
99.2
Planks
74.6
73.3
69.7
95.0
99.2
Crafting table
69.8
65.4
61.8
91.3
99.2
Stick
74.2
72.3
69.3
94.6
99.2
Wooden pickaxe
51.2
48.4
45.5
76.9
94.5
Cobblestone
26.2
23.9
31.0
67.4
57.0
Appendix
Table 5: Diamond Challenge under the Dreamer V3 protocol. Success rate (%) per milestone after about 10M environment steps, with 30-minute episodes that end at the first death. Baseline values are read from Figure 9 of Hafner et al. (2025a) ; Mirror is evaluated on 128 episodes.
Figure 14: Text-conditioned planning on Camp. We demonstrate text-conditioned planning on the camp environment. While our self-improving approach is able to solve many tasks, it struggles on tasks with fine-grained environment interactions such as placing torches in the cave. Meanwhile, VIPER, our action-free baseline, struggles to learn the action space of Minecraft. BC solves some tasks, but remains limited in its generalization ability.
Environment
Ours
Random-V2A
BCO-V2A
Grounding
Cube Single
78.3 ± 2.8
25.3 ± 4.2
8.5 ± 2.5
5.0 ± 0.8
Cube Double
27.0 ± 4.6
20.7 ± 2.8
2.0 ± 1.6
1.0 ± 0.7
Scene
32.5 ± 3.5
50.0 ± 5.4
1.5 ± 1.5
3.0 ± 0.5
AntMaze Large
25.8 ± 1.5
0.0 ± 0.0
0.0 ± 0.0
0.0 ± 0.0
Appendix
Table 6: Transfer to unseen OGBench goals. Success rate (%, mean ± SE across five seeds), averaged over target goals. All methods use target-task oracle planners; BCO-V2A uses BCO’s saved V2A model.
Figure 15: Per-task performance on DeepMind Control. Episode return on 13 tasks. Curves show the mean ± one standard error over three seeds.
Figure 16: Per-task performance on OGBench. Success rate on four single-task environments. Curves show the mean ± one standard error over three seeds.
Figure 17: Per-game performance on Atari. Scores on seven games, expressed as a percentage of each game’s expert score. Curves show the mean ± one standard error over three seeds.
Figure 18: MakeWaterfall comparisons. Mirror is capable of the long horizon planning necessary for MakeWaterfall. The agent must make a waterfall, remember its location, and move to another location to take a picture of it. Even though BC, UniTeam and KAIROS are trained with action labels, they achieve 0% success on this long-horizon task. They often go out of distribution. GoUp, which relies on scripted actions, cannot adapt to unseen scenarios such as when underwater. While it is able to place the water bucket with a pre-programmed instruction, it does not know what to do when underwater. The VLA approaches, STEVE-1, GROOT, JARVIS-VLA, and OpenHA, struggle at following the task instructions. VIPER, our action-free baseline, struggles to learn the Minecraft action space.
Video generative models have emerged as a promising robotics backbone, capable of generating videos that depict the completion of complex tasks across embodiments and environments. Recent work proposes robot foundation models that jointly predict future observations and actions by finetuning video models with action-labeled data. In this paper, we test the limits of an alternative approach: leave the video planner as-is while training an embodiment-specific inverse dynamics model (IDM). This decoupling offers several natural benefits: the video planner remains embodiment-agnostic, different video models can be interchanged easily without re-training the IDM, and the IDM can be independently trained with readily available self-play data. We present a closed-loop, video-to-action policy that combines an action-free video world model with a carefully-designed IDM based on the robot embodiment Jacobian. We demonstrate that our IDM design is both data-efficient and scalable to high-dimensional action spaces. Our policy, which we coin the Video-to-Embodied Robot Action Model (VERA), achieves strong performance across simulated and real-world benchmarks, including zero-shot Panda arm manipulation and 16-DoF Allegro-hand dexterous cube re-orientation. The same video planner can be used across multiple embodiments by pairing it with different embodiment-specific IDMs. Our results show that decoupled video planning plus faithful video-to-action translation is a viable alternative route towards zero-shot, cross-embodiment, and generalizable robot control. More results are available on our project website: https://vera.csail.mit.edu.
A central question in robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation, embodied practice, and the experience of failure. Human videos provide the first of these in abundance, and prior work has shown they can initialize useful policies. Far less clear is whether they can support the second and third: whether priors extracted from human videos can ground a robot's own attempts well enough to evaluate them, correct them, and improve from them. In this work, we show that human videos can be used to learn embodiment-agnostic action, dynamics, and value representations that transfer across robot embodiments, providing the predictive foundation required for robots to autonomously improve from their own rollouts and failures. We introduce Dynamics-Guided Action Correction (DGAC), a training-free approach that leverages these adapted models to repair failed states: each failure becomes a query for which the learned models propose and rank corrective actions, turning failures into supervision for the next policy update. Across seven real-world manipulation tasks spanning both a mobile manipulator and a static manipulator arm, our approach improves success rates from 40% to 81% across multiple policy backbones, demonstrating cross-embodiment robot self-improvement from human-video priors. These results show that human priors and robot failures can be combined to enable scalable autonomous policy improvement. Project page: https://ethz-mrl.github.io/robot-self-improvement-website/.
Hanzhi Chen, Anran Zhang, Simon Schaefer +5
1ETH Zurich · 2Technical University of Munich · 4MCML +1
Recent advances in robot foundation models trained on large-scale human teleoperation data have enabled robots to perform increasingly complex real-world tasks. However, scaling these systems remains difficult because collecting task-specific demonstrations is expensive and labor-intensive. Synthetic data, especially generated videos, offer a promising direction, but existing World Models (WMs) are not directly suitable for policy learning since they do not provide paired action trajectories. World-Action (WA) models partially address this by predicting actions with visual outputs, yet often lack strong video-action alignment, while two-stage pipelines that generate video first and then infer actions introduce inefficiency and error accumulation. To address these limitations, we propose VAG, a unified flow-matching-based dual-stream framework that jointly generates video and action under visual and language conditioning. By synchronizing denoising in both branches and using an adaptive 3D pooling mechanism to transfer compact global video context to the action branch, VAG improves cross-modal consistency during generation. Across both simulated and real-world settings, VAG produces aligned video-action pairs with competitive prediction quality, supports executable trajectory replay, and provides useful synthetic pretraining data that improves downstream policy generalization, indicating its potential as a practical world-action model for embodied data synthesis.
Xiaolei Lang, Yang Wang, Yukun Zhou +10
GigaAI · Zhejiang University · Peking University +2