Robot world models predict action-conditioned future visual states to support robotic planning and decision making. However, visual reconstruction does not fully capture task progress, semantic evaluation alone provides limited constraints on fine-grained visual errors, and homogeneous predictions limit comparative feedback. We present RoboImagine, a framework for post-training autoregressive robotic world models through candidate-future comparisons with semantic and visual feedback. Diverse search promotes diversity among candidate trajectories under shared visual observations, action sequences, and task instructions. A frozen multimodal evaluator provides task-relevant semantic preferences through pairwise video comparisons in both presentation orders, complemented by pixel- and perceptual-level visual consistency feedback that constrains local structural and appearance errors. These signals drive policy optimization through group-relative advantages. Fixed initial context and branch-specific rolling histories support long-sequence generation with bounded context and consistent conditioning between candidate generation and policy optimization. Experiments with separately trained models on RT-1, BridgeV2, and RoboCasa cover real-world and simulated manipulation. On RT-1, under matched initialization, update counts, and evaluation protocols, RoboImagine improves PSNR by 1.26 dB and reduces LPIPS by 11.93% relative to continued supervised fine-tuning. Ablations show that diverse search improves prediction quality and that combined semantic and visual feedback yields higher external multimodal evaluation scores than visual feedback alone while maintaining comparable visual fidelity. Rolling histories further improve long-horizon prediction. These results support improving visual fidelity, task relevance, and long-horizon prediction through candidate comparison and complementary feedback.
Figures & tables
Fig. 1: Complementary feedback for robot world models. Preserved appearance can coexist with stalled task progress, while task advancement can coexist with local visual artifacts. Visual fidelity and task semantics therefore provide complementary feedback. Scenes are conceptual illustrations, not experimental predictions or measured rankings.
Fig. 2: Multimodal adaptation and learning from complementary feedback in RoboImagine . Stage 1 learns dynamics-token prediction conditioned on observations, actions, and text. Stage 2 constructs diverse candidate futures with branch-specific rolling histories and combines semantic preferences from a frozen Jev evaluator with pixel- and perceptual-level visual feedback. Only the visual-feedback branch accesses recorded future frames during post-training. The generator and its text cross-attention receive policy gradients; the visual codec and feedback models remain frozen.
Fig. 3: Quantized representation and autoregressive prediction in RoboImagine . (a) Context-conditioned visual encoding produces 1,280 context tokens and 80 dynamics tokens per frame through finite scalar quantization (FSQ); discretized actions form the shared visual–action prefix. FSQ uses an implicit grid with 4,375 codes per visual branch. (b) A 12-layer LLaMA decoder predicts dynamics tokens. The expanded layer shows causal attention, rotary positions, output projection, and the gated feed-forward block with residual connections. Text cross-attention is inserted before layers 4, 8, and 12; text features are not discrete prefix tokens.
Fig. 4: Fixed-anchor branch memory for long-horizon generation in RoboImagine . (a) The retained prefix combines fixed initial context and an observed dynamics anchor with six recent predicted states and paired actions. (b) A previously generated BridgeV2 rollout illustrates the shift from z3:8 to z4:9 . Each search branch updates its own history using shared AR parameters. (c) The model predicts a new state, shifts the state–action window, and rebuilds its prefix and KV cache for the next frame. Both anchors and earlier video outputs remain stored. Cells represent token blocks; zj abbreviates zjdyn and aj denotes a^j .
Fig. 5: Candidate construction with different search strategies. (a) Likelihood-guided MCTS selects and expands frame prefixes, rolls out the remaining horizon, and backs up returns along the selected ancestor path. (b) Ordinary beam search ranks token prefixes in a single pool by cumulative model log probability. (c) GDBS maintains separate pools and penalizes tokens selected by all earlier groups at the same position, retaining one complete candidate per group. Colors identify groups; trees are schematic. The depicted search budgets differ.
Method
Task Alignment
Physical Realism
Total
Instr. ↑
Manip. ↑
Act.-Out. ↑
Temp. ↑
Contact ↑
Phys. ↑
↑
Real videos
3.0
2.0
1.0
1.0
1.0
2.0
10.0
Closed video models
Kling 2.6
2.42 ±0.09
1.38 ±0.10
0.46 ±0.08
0.58 ±0.07
0.82 ±0.06
1.18 ±0.11
6.84 ±0.22
Runway Gen-4.5
2.34 ±0.10
1.29 ±0.09
0.43 ±0.07
0.55 ±0.08
0.79 ±0.06
1.12 ±0.10
6.52 ±0.24
MiniMax Hailuo 02
2.18 ±0.11
1.17 ±0.10
0.38 ±0.07
0.47 ±0.09
0.71 ±0.08
1.01 ±0.12
5.92 ±0.28
TABLE I: Robot world-model performance on RT-1 from RobotWorldBench , evaluated with RoboAlign-Judge (mean ± std.). Superscripts 1, 2, and 3 indicate the top three distinct model scores in each column; ties share a rank.
Fig. 6: Post-training dynamics from a shared SFT initialization. (a–d) Four methods use the same 128 RT-1 trajectories, 32-frame horizon, and evaluation decoder; search variants use combined feedback. Markers denote evaluated checkpoints, and red gains compare GDBS with the shared 50,000-step SFT initialization. (e) The same drawer-closing case at prediction frame 32 compares all four methods after 100, 250, and 500 updates. Ground truth and initial SFT are shared references; gold boxes mark identical regions enlarged across methods and checkpoints.
Fig. 7: Training cost and prediction quality. Both methods perform 500 updates from the same SFT checkpoint. GPU-hours measure the training stage; PSNR changes are relative to the shared initialization. Update counts are matched, while computational costs differ.
Fig. 8: Selected qualitative comparisons of RoboImagine and its supervised initialization. Rows show ground truth, supervised fine-tuning, and RoboImagine after 500 post-training steps under the same inputs. The RT-1 case uses visual-only feedback; BridgeV2 and RoboCasa use combined feedback. Frames 8 and 32 show motion over time; gold boxes mark identical regions enlarged from frame 32.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
Qwen ↑
A. Reward composition
SFT
21.37
0.7860
0.1734
7.04
Jev only
21.95 3
0.7945 3
0.1637 3
7.69 3
Visual only
22.67 1
0.8062 1
0.1510 1
7.97 2
Hybrid
22.57 2
0.8035 2
0.1541 2
8.30 1
B. Candidate search
TABLE II: Ablations of reward composition and candidate search on RT-1. Superscripts rank the top three scores within each block. Visual metrics use 128 trajectories; Qwen scores use the same 97 trajectories in both blocks. Hybrid in A and GDBS in B denote the same configuration.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
GPU-s/ traj. ↓
SFT initialization
21.37
0.7860
0.1734
—
Random sampling
21.47
0.7869
0.1729
23.78
MCTS
21.49 3
0.7871 3
0.1722 3
84.29
Ordinary beam
22.07 2
0.7962 2
0.1627 2
24.81
GDBS
22.57 1
0.8035 1
0.1541 1
25.34
TABLE III: Search comparison after 500 post-training updates on all 128 RT-1 trajectories (32-frame prediction). Cost: training GPU-seconds per trajectory presentation. Superscripts rank visual metrics.
Fig. 9: Prediction differences and feedback. All 128 RT-1 trajectories in Table III . Cells show mean between-model dynamics-token distances. Semantic feedback uses bidirectional comparisons by the frozen training Jev; visual feedback is −(MAE+LPIPS) . Both feedback columns are higher-is-better; larger token distances do not imply better quality.
Fig. 10: Recorded GDBS search: moving a chip bag near a can. Thick paths trace returned candidates; thin paths show other retained beams. Keyframes accompany post-search feedback; bold scores mark column maxima. This pre-update training case has the largest semantic-score spread increase over ordinary beam among eight conditions.
Fig. 11: Long-horizon memory comparison on RT-1.
Frames
Window
PSNR ↑
SSIM ↑
LPIPS ↓
MAE ↓
Time (s) ↓
Memory (GiB) ↓
16
W=1
20.99
0.7869
0.1730
0.0463
12.46 1
2.206 1
W=3
23.33 1
0.8128 1
0.1424 1
0.0359 2
12.67 2
2.206 2
W=6
23.27 2
0.8115 2
0.1430 2
0.0357 1
13.42 3
2.324 3
W=12
22.94 3
0.8076 3
0.1471 3
0.0374 3
14.39
2.586
32
W=1
19.77
0.7671
0.1941
0.0546
24.94 1
2.230 1
W=3
22.08 2
0.7963 2
0.1599 2
0.0429 2
25.36 2
2.230 2
TABLE IV: History-window quality and inference cost with fixed weights and anchors. Time denotes whole-rollout latency; memory denotes peak allocated GPU memory. Three timing repeats are averaged within each trajectory. Superscripts rank the top three values within each length at full precision. This experiment uses the same 30 long-sequence-eligible RT-1 test trajectories, whereas Table 2 uses 128 RT-1 trajectories. Thus, the 32-frame means are not directly comparable across tables (e.g., W=6 : 22.20 versus 22.57 dB).
Fig. 12: History-window sensitivity and inference cost. Cells show mean quality; outlines mark the best mean at each length. Bars show whole-rollout latency and peak allocated GPU memory, with 95% trajectory-bootstrap intervals. Three timing repeats are averaged within each trajectory. All five lengths use the same 30 long-sequence-eligible RT-1 test trajectories, rather than the 128 trajectories in Table 2. Their 32-frame means are therefore not directly comparable (e.g., W=6 : 22.20 versus 22.57 dB).
TABLE V: Default implementation settings for the controlled RT-1 post-training experiments. Dataset-specific and comparator exceptions are described in the text.
Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
Planning with world models offers a powerful paradigm for robotic control. Conventional approaches train a model to predict future frames conditioned on current frames and actions, which can then be used for planning. However, the objective of predicting future pixels is often at odds with the actual planning objective; strong pixel reconstruction does not always correlate with good planning decisions. This paper posits that instead of reconstructing future frames as pixels, world models only need to predict task-relevant semantic information about the future. For such prediction the paper poses world modeling as a visual question answering problem about semantic information in future frames. This perspective allows world modeling to be approached with the same tools underlying vision language models. Thus, vision-language models can be trained as "semantic" world models through a supervised finetuning process on image-action-text data, enabling planning for decision-making while inheriting many of the generalization and robustness properties from the pretrained vision-language models. In this paper, we introduce this new method and validate it on simulated robotic tasks leading to improvements over typical paradigms of reconstruction-based action-conditional world modeling. Website available at https://weirdlabuw.github.io/swm.
Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with a task-progress-aware vision-language model scoring. To enable reliable long-horizon autoregressive world-model rollouts, we propose Step Forcing, which combines anchored and one-step self-forwarded contexts to reduce train-test mismatch while preserving action-observation dynamics. Together, these components enable RoboWorld to align strongly with real-world robot evaluation across tasks and environments, achieving Pearson's r = 0.989 and Spearman's ρ = 0.970.