Action-conditioned robot world models generate future video frames of the manipulated scene given a robot action sequence, offering a promising alternative for simulating tasks that are difficult to model with traditional physics engines. However, these models are optimized for short-term prediction and break down when deployed autoregressively: each predicted clip feeds back as context for the next, causing errors to compound and visual quality to rapidly degrade. We address this through the following contributions. First, we introduce a reinforcement learning (RL) post-training scheme that trains the world model on its own autoregressive rollouts rather than on ground-truth histories. We achieve this by adapting a recent contrastive RL objective for diffusion models to our setting and show that its convergence guarantees carry over exactly. Second, we design a training protocol that generates and compares multiple candidate variable-length futures from the same rollout state, reinforcing higher-fidelity predictions over lower-fidelity ones. Third, we develop efficient, multi-view visual fidelity rewards that combine complementary perceptual metrics across camera views and are aggregated at the clip level for dense, low-variance training signal. Fourth, we show that our approach establishes a new state-of-the-art for rollout fidelity on the DROID dataset, outperforming the strongest baseline on all metrics (e.g., LPIPS reduced by 14% on external cameras, SSIM improved by 9.1% on the wrist camera), winning 98% of paired comparisons, and achieving an 80% preference rate in a blind human study.
Figures & tables
Figure 1: Autoregressive video rollout quality. We use an action-conditioned robot world model to generate multi-view image predictions from a single observed state. The ground-truth ( blue ) is compared against the baseline ( red ) and our post-trained model PersistWorld ( green ). While the baseline accumulates error and destroys the object (cyan bowl) within seconds, our method maintains structural integrity and spatial consistency, establishing a new state-of-the-art in rollout fidelity.
Figure 2: Overview of our method: (Top) Autoregressive inference: A robot policy generates actions fed to the world model, which produces multi-view frames that are appended to the history buffer and condition the next generation step. (Bottom) RL post-training: (S1) A shared variable-length prefix is rolled out autoregressively from a ground-truth initial condition. (S2) K independent candidate continuations are branched from the frozen prefix state. (S3) Candidates are scored against ground-truth using multi-view perceptual rewards. (S4) Reward weights r scale implicit positive/negative x0 predictions used in contrastive model updates via loss L .
Evaluated Cameras
Model
Pixel/Structure
Perceptual
SSIM ↑
PSNR ↑
LPIPS ↓
External Camera
WPE ∗
0.77
20.33
0.131
IRASim ∗
0.77
21.36
0.117
Ctrl-World ∗
0.83
23.56
0.091
Ctrl-World †
0.84
23.02
0.081
Ours
0.86
24.42
0.070
Table 1: Visual quality metrics for 14-step autoregressive rollouts ( ≈ 11 s) on the DROID validation split. Values represent averages over the full rollout duration. Results marked with ∗ are from [ 12 ] ; † indicates our reproduction.
Figure 3: Qualitative comparison of autoregressive rollout stability. We compare long-horizon (11 s) generations from the baseline [ 12 ] against our PersistWorld for the wrist camera. Left: Object-centric fidelity. The baseline model suffers from rapid decoherence; as errors compound in the history buffer, manipulated objects like the cup lose their structural identity and dissolve into amorphous textures. In contrast, our method maintains the spatial consistency and structural integrity of the object throughout the rollout. Right: Robot-centric consistency. The baseline exhibits significant robot decoherence , where the generated robot arm loses their geometric structure. Our approach maintains structural persistence. Please see additional video results on the associated project page.
Figure 4: Δmetric of paired videos from the validation dataset. On 1−1 paired comparison, our PersistWorld world model is better than the baseline on ∼98% of the sample ( p<10−6 ).
Figure 5: Temporal evolution of wrist camera metrics. While both models exhibit natural degradation over longer horizons (x-axis), our post-trained model, PersistWorld (green), consistently maintains higher fidelity and slower error accumulation compared to the baseline (orange). Specifically, our method preserves a higher PSNR and SSIM while suppressing LPIPS drift, effectively extending the stable prediction horizon for complex, fine-grained interactions. See Fig. 11 in Appendix. 0.E.2 for external camera results.
Evaluated Cameras
Model
Object-Only
Robot-Only
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
External Camera
Ctrl-World †
0.88
22.25
0.025
0.82
17.62
0.039
Ours
0.89
23.60
0.021
0.86
19.25
0.033
Wrist Camera
Ctrl-World †
0.73
18.52
0.088
0.83
25.50
0.027
Ours
0.76
19.87
0.078
0.86
27.24
0.023
Table 2: Masked visual metrics for 14-step autoregressive rollouts ( ≈ 11 s) . We isolate object-only and robot-only pixels to evaluate task-relevant spatial and control consistency. Our model demonstrates superior fidelity in these dynamic regions compared to baselines, confirming that performance gains are driven by accurate interaction modeling rather than background reconstruction. All metrics are averaged over the full rollout horizon.
Figure 6: Human preference results.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Learning Rate
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
3×10−4
0.865
25.02
0.1230
0.706
19.95
0.3603
1×10−4 (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 3: Ablation of the post-training learning rate.
Reward
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
LPIPS only
0.868
25.17
0.1156
0.708
19.90
0.3242
SSIM only
0.872
25.40
0.1178
0.724
20.24
0.3516
PSNR only
0.871
25.57
0.1189
0.716
20.45
0.3544
Combined (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 4: Ablation of the reward signal used during DiffusionNFT post-training.
Strategy
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
Fixed
Size 3
0.872
25.53
0.1170
0.723
20.51
0.3381
Size 6
0.871
25.41
0.1174
0.719
20.33
0.3423
Curriculum [ 41 ]
0.871
25.50
0.1169
0.721
20.43
0.3412
Random
Appendix
Table 5: Ablation of the context-window curriculum. Curriculum is the growing schedule from [ 41 ] ; Random samples uniformly from the specified range each step; Fixed keeps a constant window size throughout training.
Config
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
Group Size 4
0.875
25.62
0.1142
0.724
20.56
0.3379
Group Size 8
0.873
25.61
0.1176
0.721
20.45
0.3471
Group Size 16
No best of N
0.874
25.69
0.1127
0.725
20.64
0.3255
Best of 5 (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 6: Ablation of the GRPO group size and the number of candidates retained via best-of- N filtering (evaluated at group size 16).
Horizon H
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
H=3
0.872
25.41
0.1169
0.719
20.31
0.3424
H=1 (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 7: Ablation of the future prediction horizon H (number of frames generated per autoregressive step) used during post-training.
wwrist
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
wwrist=2
0.872
25.45
0.1169
0.722
20.47
0.3357
wwrist=1 (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 8: Ablation of the wrist-camera loss weight wwrist relative to the external cameras (weight 1).
KL Reg.
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
Without
0.873
25.50
0.1163
0.720
20.38
0.3392
With (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 9: Effect of KL regularisation applied to the post-training objective.
Reference Policy EMA
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
No EMA schedule
0.872
25.51
0.1162
0.722
20.46
0.3390
EMA rising to 0.5 (Ours)
0.872
25.52
0.1162
0.721
20.40
0.3387
Appendix
Table 10: Effect of applying a warming EMA schedule to the frozen reference (old) policy, which gradually interpolates it toward the learning policy during training.
Policy EMA
External
Wrist
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
EMA =0.9
0.876
25.81
0.1105
0.726
20.68
0.3215
No EMA (Group Size 16, BoN 2)
0.876
25.79
0.1107
0.727
20.69
0.3212
Appendix
Table 11: Effect of applying an exponential moving average (EMA) to the learning (fine-tuned) policy weights during post-training.
Figure 7: Initial Conditions for the WM-to-Real Correlation. We run rollouts in the world models given these starting conditions for the three tasks. For the policy rollout, the middle view (external camera) and the wrist view are provided as inputs.
Skill
Task
Progression
Put
Put Banana in Box
Reach → Grasp → Lift → Move Close → IsInside
Put Green Block in Bowl
Rotate
Rotate Marker
Reach → Grasp → Rotate 45∘
Appendix
Table 12: Partial Progress of the Different Tasks .
Figure 8: WM to Real comparison: While both the world models tend to make the task easier for the policies (shown by the higher values task performance on the WM), our post-trained model maintains better correlation and MMRV amongst the policies compared to the baseline [ 12 ] .
Figure 9: 2AFC Website Onboarding: We provide a small description of the task and potential hints to look at when judging the quality of the generated rollouts compared to the reference.
Figure 10: 2AFC Website Comparison UI: On the comparison webpage, the users are provided with the reference video on the top and the two generated videos below it. The users can use the scrubber to move through the frames or watch the entire video. After which, the user needs to choose between Option A or Option B.
Figure 11: Temporal evolution of external camera metrics. While both models exhibit natural degradation over longer horizons (x-axis), our post-trained model, PersistWorld (green), consistently maintains higher fidelity and slower error accumulation compared to the baseline (orange). Specifically, our method preserves a higher PSNR and SSIM while suppressing LPIPS drift, effectively extending the stable prediction horizon for complex, fine-grained interactions.
Reward
SSIM ↑
PSNR ↑
LPIPS ↓
training sec/iter ↓
HPSv3 (theirs)
0.802
22.58
0.193
129.2
Ours (top- k=2 )
0.826
24.09
0.181
41.3
Appendix
Table 13: Our reward vs HPSv3 [ 30 ] : To compare to WorldCompass [ 41 ] , we compare their reward structure with HPSv3 [ 30 ] to our light-weight rewards.
Figure 12: Qualitative comparison of autoregressive rollout stability on Bridge [ 9 ] and RT-1 [ 7 ] . We compare long-horizon (11s) generations from the baseline against our PersistWorld . In both the cases, we see that the baseline model struggles to maintain object consistency. In left, we see that the saucepan has completely disappeared in the baseline model, while PersistWorld is able to maintain it. In right, we see that the baseline model has lost the bottle, and instead hallucinates a different object within the robot’s gripper. In contrast, PersistWorld is able to maintain the bottle and correctly shows the gripper grasping the bottle.
Dataset
Method
SSIM ↑
PSNR ↑
LPIPS ↓
Bridge
IRASim [ 49 ] (long-traj) †
–
23.26
–
SVD backbone
0.864
25.20
0.1061
+ PersistWorld ( Ours )
0.876
26.36
0.0957
RT-1
IRASim [ 49 ] (long-traj) †
–
24.62
–
SVD backbone
0.833
23.81
0.1683
+ PersistWorld ( Ours )
0.865
26.63
0.1305
Appendix
Table 14: Generalization Beyond DROID: We compare the performance of the standard action conditioned pretraining on SVD backbone to PersistWorld on a long rollout with 10 consecutive interactions. We also report IRASim [ 49 ] on their long trajectory setup ( † indicates the number is taken from original paper).
Figure 13: Ours vs SF style baseline: Our reward exceeds the SF-style baseline. Our procedure achieves the same performance ∼10× faster and achieves a much higher reward at the same number of steps.
Figure 14: Policy Improvement Setup: We place 4 blocks on a table and command the robot to pick up the top-right/top-left/bottom-right/bottom-left block in the 2×2 grid. We also place some distractors on the table such as the bowl and the tape.
Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with a task-progress-aware vision-language model scoring. To enable reliable long-horizon autoregressive world-model rollouts, we propose Step Forcing, which combines anchored and one-step self-forwarded contexts to reduce train-test mismatch while preserving action-observation dynamics. Together, these components enable RoboWorld to align strongly with real-world robot evaluation across tasks and environments, achieving Pearson's r = 0.989 and Spearman's ρ = 0.970.
Predictive world models enable robots to plan by imagining the outcomes of their actions, but their value for control hinges on generating many rollouts quickly. This creates a bottleneck for diffusion-based world models: multistep sampling makes each rollout expensive, limiting large-scale action search at inference time. We introduce DriftWorld, an action-conditioned world model based on drifting generative models. Rather than denoising iteratively at inference, DriftWorld learns an action-conditioned drift during training, allowing it to generate future frames from the current observation and a candidate action sequence in a single forward pass at 30+ fps, which is 17x faster on average than diffusion based baselines. We evaluate DriftWorld on standard vision-based robotic manipulation benchmarks, including Bridge-V2, RT-1, Language Table, Push-T, and Robomimic. By producing rollouts that are both accurate and fast, DriftWorld achieves state-of-the-art decision-making performance with far less inference time than diffusion-based world model baselines. Beyond online control, DriftWorld can also serve as an offline simulator for ranking real-world robot policies, with rollout-based scores correlating with ground truth at up to 0.99. These results show that drifting models are a strong fit for robot world modeling, where fast, high-quality imagination directly supports planning and policy evaluation.
Susie Lu, Haonan Chen, Weirui Ye +1
Massachusetts Institute of Technology · Harvard University