Action-conditioned world models let robots imagine the outcomes of their actions for planning and offline policy evaluation, but these uses need many rollouts, which multi-step diffusion makes expensive. Drifting models move the iteration from inference to training and generate samples in one network pass. DriftWorld brought this paradigm to action-conditioned world models; to strengthen action following, it repels predictions from the current frame, a construction borrowed from classifier-free guidance. We show that this accentuation has a target bias: the push persists even when a prediction matches the recorded future, in kernel, balanced, and unbalanced Wasserstein drifting fields alike. We introduce Mass-Share Accentuation (MSA) to remove this bias. MSA treats the current frame as a counterfactual destination competing with the recorded future in a semi-relaxed optimal transport problem; subtracting the closed-form share of the correct future makes the guidance vanish at the target, and the remainder is shared by all predictions. We prove that the recorded future is the unique fixed point along the motion direction for collapsed predictions, and that the guidance fades with prediction error. On Push-T, Robomimic, and BridgeV2, where trained models under-react rather than overshoot, MSA improves action ranking, response slope, and moving-region error over latent-space baselines at no inference cost, though these benchmarks do not isolate bias removal as the cause.
Figures & tables
Figure 1: In a controlled toy, accentuation overshoots the recorded future. Each point of a block (blue) is moved by an action to its recorded future (gray); orange shows a trained one-step world model’s predictions and purple lines their motion. Accentuation drives predictions past the target, increasingly with α . Without debiasing, a small overshoot remains; MSA lands on the recorded future. See Sec. 5.5 .
Action
Motion
Standard
1 px / 2 mm
3 px / 5 mm
Dataset
Method
τ↑
Slope ↑
Corr. ↑
Noise ↓
Mov. ↓
Drift 9–60↓
Static ↓
Late ↓
Late ↓
MSE ↓
SSIM ↑
PSNR ↑
LPIPS ↓
Kernel
0.682
0.699
0.826
3.43
0.1806
0.0218
0.0192
0.42
0.45
0.0247
0.951
24.33
0.0607
Kernel + acc.
0.683
0.698
0.843
3.33
0.1846
0.0221
0.0195
0.39
0.42
0.0248
0.951
24.31
0.0602
Push-T
W-Flow
0.693
0.713
0.830
1.49
0.1720
0.0208
0.0183
0.41
0.46
0.0244
0.952
24.48
0.0608
W-Flow + acc.
0.678
0.698
0.852
1.32
0.1761
0.0220
0.0194
0.42
0.42
0.0270
0.948
23.93
0.0663
Table 1: Results on Push-T and Robomimic. Action: Kendall τ , response slope (ideally 1) and correlation of the predicted object displacement across proposals, and sample noise (px / mm). Motion: moving-region MSE, static drift (frames 9–60), static MSE, and late-contact fraction at two thresholds. Standard: frame MSE, SSIM, PSNR, LPIPS. Best per dataset in bold .
Policy 1
Policy 2
Method
IoU ↑
s/plan ↓
IoU ↑
s/plan ↓
Base policy
0.597±0.163
–
0.644±0.138
–
Kernel
0.704±0.188
1.92
0.720±0.207
1.96
Kernel + acc.
0.706±0.213
1.84
0.721±0.166
1.94
W-Flow
0.717±0.207
1.88
0.722±0.169
1.87
W-Flow + acc.
0.717±0.221
1.79
0.721±0.173
1.78
Table 2: GPC-Rank on Push-T. Each world model inside GPC-Rank [ 43 ] , two proposal policies. IoU: final T-block/goal overlap (mean ± s.d. over 50 episodes); s/plan: seconds per plan. Best in bold .
Figure 2: Accentuation scale sweep. Curves trace α∈{1,…,4} ; MSA (no α ) is a star. (a) Slope ↑ vs. outcome bias ↓ . (b) τ↑ vs. moving-region MSE ↓ .
tˉ at α
Endpoint err. ↓
Method
1.75
2.5
3.25
5
( α=2.5 )
W-Flow
1.001
1.001
1.001
1.001
.003
Kernel + acc.
1.019
1.031
1.049
1.100
.021
UW-Flow + acc.
1.001
1.001
1.001
1.001
.004
W-Flow + acc.
1.754
2.504
3.253
5.004
1.050
MSA w/o debiasing
1.069
1.106
1.135
1.187
.072
Table 3: Target bias in the T-block toy (Fig. 1 ). Converged progress along the recorded motion (1 = recorded future) for accentuation scale α (for MSA, weight β=α−1 ), and endpoint error at α=2.5 ; mean over 3 seeds (s.d. ≤0.001 ). Balanced accentuation converges to tˉ=α (Prop. 2 ); MSA converges to the recorded future for every weight (Prop. 4 ). Best in bold .
Figure 3: Qualitative rollouts. Last context frame, ground truth six steps later, and each model’s prediction under the recorded actions. BridgeV2: top-left 191×191 crop. The green contour is the ground-truth object (arm on BridgeV2) at +6 .
Controllability
Motion probe
Visual and temporal quality
Method
Δ PSNR ↑
PSNR ↑
Prog. ↑
Prog. mov↑
Off-mot. ↓
Spread
SSIM ↑
PSNR ↑
LPIPS ↓
FID ↓
FVD ↓
Copy-last-frame
0
19.99
0
0
–
–
.832
20.07
.111
4.17
251.7
Unrelated real video
–
–
–
–
–
–
.211
10.07
.656
23.60
115.1
Kernel
2.108
23.03
0.427
0.464
0.339
0.025
.818
23.28
.159
55.49
326.1
Kernel + acc.
2.111
23.00
0.436
0.450
0.347
0.025
.831
23.23
.148
55.44
304.4
W-Flow
2.185
22.99
0.426
0.440
0.326
0.031
.827
23.18
.148
56.40
323.3
Table 4: Results on BridgeV2. Controllability: PSNR with true actions and its drop Δ PSNR when shuffled (489 windows). Motion probe: progress along the real motion, off-motion error, and spread (200 windows × 16 samples). Quality: 8-frame rollouts on 491 validation episodes. Best among learned models in bold .
Method
Agent err (px) ↓
Block xy (px) ↓
Block angle ( ∘ ) ↓
EPE mov. ↓
Mov. MSE ↓
Static drift ↓
Kernel
2.56
44.3
44.1
1.263
.209
.037
Kernel + acc.
2.20
44.7
44.6
1.238
.207
.036
W-Flow
1.56
44.1
44.3
1.231
.198
.036
W-Flow + acc.
1.54
45.4
44.1
1.268
.206
.036
MSA (ours)
1.44
42.1
43.6
1.181
.182
.037
Table 5: Full-episode rollouts. Same 1000 starts and recorded actions, about 250 frames; the block pose is read from the generated frames. Best in bold .
15k
40k
Method
Extra hyperparameters
Sinkhorn solves/step
Zero push at target
τ↑
Slope ↑
Noise ↓
Mov. MSE ↓
EPE mov. ↓
τ↑
Slope ↑
Noise ↓
Tail share ↓
W-Flow
–
2
n/a
.301
.349
3.30
.303
1.695
.538
.579
2.33
.000
W-Flow + acc.
α (train + inference)
3
×
.266
.279
2.74
.312
1.696
.471
.446
1.81
.000
UW-Flow
λ
2
n/a
.266
.341
7.80
.314
1.724
.471
.546
5.15
.000
UW-Flow + acc.
λ (matched ×3 ), α
3
×
.320
.337
3.67
.286
1.659
.177
.364
39.8
.542
MSA (ours)
β fixed (no knob)
2 + gate
✓
.315
.397
3.63
.286
1.610
.553
.612
2.23
.000
Table 7: Comparison with UW-Flow. Cost and properties, with metrics at 15k and 40k steps (noise in px). UW-Flow + acc. matches λ across three transport problems; MSA adds a closed-form gate. Best in bold .
Variant
P / Z / S
τ↑
Slope ↑
Noise ↓
Bias ↓
Mov. MSE ↓
W-Flow
– / – / –
0.693
0.713
1.49
–
0.1720
+ constant push
× / × / ✓
0.678
0.698
1.32
0.484
0.1761
+ joint softmax
✓ / × / ×
0.694
0.718
1.44
0.492
0.1743
MSA (ours)
✓ / ✓ / ✓
0.712
0.746
1.55
0.462
0.1699
Table 6: Gate components. Accentuation added to W-Flow as (α−1)d , βrid , and βGd ( Eq. 16 ). P/Z/S: progress-aware, zero at target, shape-neutral ( Sec. 4.2 ). Best in bold .
Figure 4: Gate annealing. (a) Gate. (b) Progress tˉ . (c) Slope and signal ratio. Details: Supp. E.1 .
Predictive world models enable robots to plan by imagining the outcomes of their actions, but their value for control hinges on generating many rollouts quickly. This creates a bottleneck for diffusion-based world models: multistep sampling makes each rollout expensive, limiting large-scale action search at inference time. We introduce DriftWorld, an action-conditioned world model based on drifting generative models. Rather than denoising iteratively at inference, DriftWorld learns an action-conditioned drift during training, allowing it to generate future frames from the current observation and a candidate action sequence in a single forward pass at 30+ fps, which is 17x faster on average than diffusion based baselines. We evaluate DriftWorld on standard vision-based robotic manipulation benchmarks, including Bridge-V2, RT-1, Language Table, Push-T, and Robomimic. By producing rollouts that are both accurate and fast, DriftWorld achieves state-of-the-art decision-making performance with far less inference time than diffusion-based world model baselines. Beyond online control, DriftWorld can also serve as an offline simulator for ranking real-world robot policies, with rollout-based scores correlating with ground truth at up to 0.99. These results show that drifting models are a strong fit for robot world modeling, where fast, high-quality imagination directly supports planning and policy evaluation.
Susie Lu, Haonan Chen, Weirui Ye +1
Massachusetts Institute of Technology · Harvard University
World models aim to improve robotic decision making by predicting the consequences of actions. However, in practice, their predictions often become unreliable once the robot encounters states outside the training distribution, limiting their effectiveness at deployment. We observe that execution itself provides a natural but underutilized signal: after each action, the robot directly observes the true next state, revealing the mismatch between predicted and actual outcomes. Building on this insight, we propose feedback world model, an inference-time mechanism that closes the loop between prediction and observation at inference time. Instead of treating the world model as a static open-loop predictor, our method maintains a lightweight feedback state that is updated online to iteratively correct future predictions, compensating for model errors using real-time observations without additional training data or parameter updates. We show that this process can be interpreted as a latent-space observer and admits convergence guarantees under bounded, asymptotically convergent dynamics residuals and a stable feedback gain. We further introduce action-aware guidance to better translate corrected predictions into control by emphasizing action-controllable components while suppressing irrelevant variations. Experiments on LIBERO-Plus, Robomimic, and real-world manipulation tasks demonstrate that our method substantially improves both prediction accuracy and policy performance under distribution shift, reducing world model prediction error by up to 76.4%. The same feedback mechanism improves the predictions of the pretrained foundation model Fast-WAM-Joint without updating its parameters, demonstrating its plug-and-play applicability. These results show that incorporating real-time feedback at inference time provides a simple yet powerful alternative to static world modeling.
World models of visual information predict future image frames conditioned on given action sequences. This predicted scene evolution is driven by deterministic changes such as controlled motion on a robot platform, as well as stochastic changes such as rendering of manipulated objects or previously occluded regions. Our key insight is that the spatial location of these stochastic changes is itself learnable. We leverage this insight by jointly regressing both the deterministic next-state prediction and the spatial uncertainty map, then using the uncertainty map to guide a conditional diffusion process. We propose Decoupled Drift-Diffusion World Model (D3WM), which uses a transformer trained with a Gaussian negative-log-likelihood objective to produce both the deterministic next-state prediction and a spatial map of heteroscedastic aleatoric uncertainty. The learned uncertainty map then provides a structured spatial gate for a conditional diffusion model: deterministic regions are anchored to the predicted state, while uncertain regions, such as disocclusions or dynamic textures, are sampled by the diffusion process. We frame this decomposition under a variance-gated stochastic differential equation that separates deterministic drift from stochastic diffusion. D3WM shows improved results over competitive baselines in two distinct types of action spaces: robotic tabletop manipulation and camera motion.