Robot world models predict action-conditioned future visual states to support robotic planning and decision making. However, visual reconstruction does not fully capture task progress, semantic evaluation alone provides limited constraints on fine-grained visual errors, and homogeneous predictions limit comparative feedback. We present RoboImagine, a framework for post-training autoregressive robotic world models through candidate-future comparisons with semantic and visual feedback. Diverse search promotes diversity among candidate trajectories under shared visual observations, action sequences, and task instructions. A frozen multimodal evaluator provides task-relevant semantic preferences through pairwise video comparisons in both presentation orders, complemented by pixel- and perceptual-level visual consistency feedback that constrains local structural and appearance errors. These signals drive policy optimization through group-relative advantages. Fixed initial context and branch-specific rolling histories support long-sequence generation with bounded context and consistent conditioning between candidate generation and policy optimization. Experiments with separately trained models on RT-1, BridgeV2, and RoboCasa cover real-world and simulated manipulation. On RT-1, under matched initialization, update counts, and evaluation protocols, RoboImagine improves PSNR by 1.26 dB and reduces LPIPS by 11.93% relative to continued supervised fine-tuning. Ablations show that diverse search improves prediction quality and that combined semantic and visual feedback yields higher external multimodal evaluation scores than visual feedback alone while maintaining comparable visual fidelity. Rolling histories further improve long-horizon prediction. These results support improving visual fidelity, task relevance, and long-horizon prediction through candidate comparison and complementary feedback.
Figures & tables
Fig. 1: Complementary feedback for robot world models. Preserved appearance can coexist with stalled task progress, while task advancement can coexist with local visual artifacts. Visual fidelity and task semantics therefore provide complementary feedback. Scenes are conceptual illustrations, not experimental predictions or measured rankings.
Fig. 2: Multimodal adaptation and learning from complementary feedback in RoboImagine . Stage 1 learns dynamics-token prediction conditioned on observations, actions, and text. Stage 2 constructs diverse candidate futures with branch-specific rolling histories and combines semantic preferences from a frozen Jev evaluator with pixel- and perceptual-level visual feedback. Only the visual-feedback branch accesses recorded future frames during post-training. The generator and its text cross-attention receive policy gradients; the visual codec and feedback models remain frozen.
Fig. 3: Quantized representation and autoregressive prediction in RoboImagine . (a) Context-conditioned visual encoding produces 1,280 context tokens and 80 dynamics tokens per frame through finite scalar quantization (FSQ); discretized actions form the shared visual–action prefix. FSQ uses an implicit grid with 4,375 codes per visual branch. (b) A 12-layer LLaMA decoder predicts dynamics tokens. The expanded layer shows causal attention, rotary positions, output projection, and the gated feed-forward block with residual connections. Text cross-attention is inserted before layers 4, 8, and 12; text features are not discrete prefix tokens.
Fig. 4: Fixed-anchor branch memory for long-horizon generation in RoboImagine . (a) The retained prefix combines fixed initial context and an observed dynamics anchor with six recent predicted states and paired actions. (b) A previously generated BridgeV2 rollout illustrates the shift from z3:8 to z4:9 . Each search branch updates its own history using shared AR parameters. (c) The model predicts a new state, shifts the state–action window, and rebuilds its prefix and KV cache for the next frame. Both anchors and earlier video outputs remain stored. Cells represent token blocks; zj abbreviates zjdyn and aj denotes a^j .
Fig. 5: Candidate construction with different search strategies. (a) Likelihood-guided MCTS selects and expands frame prefixes, rolls out the remaining horizon, and backs up returns along the selected ancestor path. (b) Ordinary beam search ranks token prefixes in a single pool by cumulative model log probability. (c) GDBS maintains separate pools and penalizes tokens selected by all earlier groups at the same position, retaining one complete candidate per group. Colors identify groups; trees are schematic. The depicted search budgets differ.
Method
Task Alignment
Physical Realism
Total
Instr. ↑
Manip. ↑
Act.-Out. ↑
Temp. ↑
Contact ↑
Phys. ↑
↑
Real videos
3.0
2.0
1.0
1.0
1.0
2.0
10.0
Closed video models
Kling 2.6
2.42 ±0.09
1.38 ±0.10
0.46 ±0.08
0.58 ±0.07
0.82 ±0.06
1.18 ±0.11
6.84 ±0.22
Runway Gen-4.5
2.34 ±0.10
1.29 ±0.09
0.43 ±0.07
0.55 ±0.08
0.79 ±0.06
1.12 ±0.10
6.52 ±0.24
MiniMax Hailuo 02
2.18 ±0.11
1.17 ±0.10
0.38 ±0.07
0.47 ±0.09
0.71 ±0.08
1.01 ±0.12
5.92 ±0.28
TABLE I: Robot world-model performance on RT-1 from RobotWorldBench , evaluated with RoboAlign-Judge (mean ± std.). Superscripts 1, 2, and 3 indicate the top three distinct model scores in each column; ties share a rank.
Fig. 6: Post-training dynamics from a shared SFT initialization. (a–d) Four methods use the same 128 RT-1 trajectories, 32-frame horizon, and evaluation decoder; search variants use combined feedback. Markers denote evaluated checkpoints, and red gains compare GDBS with the shared 50,000-step SFT initialization. (e) The same drawer-closing case at prediction frame 32 compares all four methods after 100, 250, and 500 updates. Ground truth and initial SFT are shared references; gold boxes mark identical regions enlarged across methods and checkpoints.
Fig. 7: Training cost and prediction quality. Both methods perform 500 updates from the same SFT checkpoint. GPU-hours measure the training stage; PSNR changes are relative to the shared initialization. Update counts are matched, while computational costs differ.
Fig. 8: Selected qualitative comparisons of RoboImagine and its supervised initialization. Rows show ground truth, supervised fine-tuning, and RoboImagine after 500 post-training steps under the same inputs. The RT-1 case uses visual-only feedback; BridgeV2 and RoboCasa use combined feedback. Frames 8 and 32 show motion over time; gold boxes mark identical regions enlarged from frame 32.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
Qwen ↑
A. Reward composition
SFT
21.37
0.7860
0.1734
7.04
Jev only
21.95 3
0.7945 3
0.1637 3
7.69 3
Visual only
22.67 1
0.8062 1
0.1510 1
7.97 2
Hybrid
22.57 2
0.8035 2
0.1541 2
8.30 1
B. Candidate search
TABLE II: Ablations of reward composition and candidate search on RT-1. Superscripts rank the top three scores within each block. Visual metrics use 128 trajectories; Qwen scores use the same 97 trajectories in both blocks. Hybrid in A and GDBS in B denote the same configuration.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
GPU-s/ traj. ↓
SFT initialization
21.37
0.7860
0.1734
—
Random sampling
21.47
0.7869
0.1729
23.78
MCTS
21.49 3
0.7871 3
0.1722 3
84.29
Ordinary beam
22.07 2
0.7962 2
0.1627 2
24.81
GDBS
22.57 1
0.8035 1
0.1541 1
25.34
TABLE III: Search comparison after 500 post-training updates on all 128 RT-1 trajectories (32-frame prediction). Cost: training GPU-seconds per trajectory presentation. Superscripts rank visual metrics.
Fig. 9: Prediction differences and feedback. All 128 RT-1 trajectories in Table III . Cells show mean between-model dynamics-token distances. Semantic feedback uses bidirectional comparisons by the frozen training Jev; visual feedback is −(MAE+LPIPS) . Both feedback columns are higher-is-better; larger token distances do not imply better quality.
Fig. 10: Recorded GDBS search: moving a chip bag near a can. Thick paths trace returned candidates; thin paths show other retained beams. Keyframes accompany post-search feedback; bold scores mark column maxima. This pre-update training case has the largest semantic-score spread increase over ordinary beam among eight conditions.
Fig. 11: Long-horizon memory comparison on RT-1.
Frames
Window
PSNR ↑
SSIM ↑
LPIPS ↓
MAE ↓
Time (s) ↓
Memory (GiB) ↓
16
W=1
20.99
0.7869
0.1730
0.0463
12.46 1
2.206 1
W=3
23.33 1
0.8128 1
0.1424 1
0.0359 2
12.67 2
2.206 2
W=6
23.27 2
0.8115 2
0.1430 2
0.0357 1
13.42 3
2.324 3
W=12
22.94 3
0.8076 3
0.1471 3
0.0374 3
14.39
2.586
32
W=1
19.77
0.7671
0.1941
0.0546
24.94 1
2.230 1
W=3
22.08 2
0.7963 2
0.1599 2
0.0429 2
25.36 2
2.230 2
TABLE IV: History-window quality and inference cost with fixed weights and anchors. Time denotes whole-rollout latency; memory denotes peak allocated GPU memory. Three timing repeats are averaged within each trajectory. Superscripts rank the top three values within each length at full precision. This experiment uses the same 30 long-sequence-eligible RT-1 test trajectories, whereas Table 2 uses 128 RT-1 trajectories. Thus, the 32-frame means are not directly comparable across tables (e.g., W=6 : 22.20 versus 22.57 dB).
Fig. 12: History-window sensitivity and inference cost. Cells show mean quality; outlines mark the best mean at each length. Bars show whole-rollout latency and peak allocated GPU memory, with 95% trajectory-bootstrap intervals. Three timing repeats are averaged within each trajectory. All five lengths use the same 30 long-sequence-eligible RT-1 test trajectories, rather than the 128 trajectories in Table 2. Their 32-frame means are therefore not directly comparable (e.g., W=6 : 22.20 versus 22.57 dB).
TABLE V: Default implementation settings for the controlled RT-1 post-training experiments. Dataset-specific and comparator exceptions are described in the text.