Effective world modeling requires compressing salient history information that facilitates future prediction. However, most existing approaches inherit tokenization and conditioning mechanisms from video generation, where objectives are centered around perceptual realism rather than predictive utility. This discrepancy limits the world model's effectiveness, especially in environments dominated by visual nuisances or task-irrelevant background details. To this end, we introduce Compression Forcing (CF), a general framework that aligns history compression with the predictive objective of world modeling. Compression Forcing features Dynamics-Aware Tokenization (DAT), a novel tokenization method that compresses history observations into a sequence ordered by decreasing dynamics saliency. Built on a diffusion backbone, this structure enables Hierarchical History Guidance (HG-H), allowing selective amplification of dynamics-salient information during inference. Extensive experiments across various environments demonstrate improved prediction fidelity, physical realism, and downstream decision-making performance, establishing Compression Forcing as a highly flexible and effective approach for world modeling.
Figures & tables
Figure 2: Tokenization Attention map . We visualize attention maps in the cross-attention layer between DAT tokenized tokens and image patches in the dynamic decoder. As the slot index increases, cross-attention shifts from regions with dynamic motion (the athlete in motion) to static objects in the background.
Figure 3: Overview of the Compression Forcing Framework . (a) DAT encodes history context into a hierarchical token sequence, with leading tokens capturing dynamics and trailing tokens capturing static details. (b) The Diffusion Backbone predicts the next frame conditioned on arbitrary-length prefixes of DAT -encoded latents and the agent’s action. (c) HG-H composes score estimates from varying-length prefixes, enabling inference-time amplification of dynamically salient history information.
Figure 4: Sampling with Hierarchical History Guidance ( HG-H ) . By composing score functions conditioned on the full history representation and its high-saliency leading subset, the model selectively amplifies essential motion dynamics while suppressing visual nuisances, enabling robust prediction.
Tokenizer
Guidance
SSIM ↑
PSNR ↑
FID ↓
LPIPS ↓
SD-VAE
✗
0.543
18.53
90.57
0.351
ElasticTok
✗
0.544
18.55
91.23
0.351
ElasticTok
✓
0.546
18.60
90.74
0.349
DAT
✗
0.551
18.77
89.26
0.344
DAT
HG-H
0.553
18.83
88.13
0.343
Table 1: Dynamic Video Prediction on Kinetics-600. We evaluate against baselines sharing the diffusion-based world model backbone architecture and training budget; rows differ only in history tokenization method (SD-VAE, ElasticTok, or DAT ) and whether history guidance is applied. Details can be found in Appendix C.5 .
Figure 5: Qualitative Generation Results on Multiple Domains. Compression Forcing enables accurate and high-fidelity auto-regressive predictions in CS:GO, RT-1 and Kinetics. Zoom in for details. Extended visualizations in Appendix D .
Method
SSIM ↑
PSNR ↑
FVD ↓
FID ↓
LPIPS ↓
DIAMOND-Fast
0.449
18.2
577.1
115.6
0.547
DIAMOND-HQ
0.447
18.3
368.5
87.2
0.510
SD-VAE
0.496
17.29
231.9
70.25
0.514
ElasticTok( HG-H )
0.494
17.22
197.1
74.2
0.519
DAT ( HG-H )
0.479
17.24
123.0
53.6
0.493
Table 2: Game Simulation on CS:GO. We evaluate against DIAMOND ( Alonso et al., 2024 ) , a strong established baseline, and same-architecture variants differing only in history tokenization (SD-VAE, ElasticTok). For all methods, we auto-regressively roll out 12 frames from 4 context frames. Details in Appendix C.6 .
Method
SSIM ↑
PSNR ↑
FVD ↓
FID ↓
LPIPS ↓
SD-VAE
0.722
20.47
120.4
42.82
0.267
ElasticTok( HG-H )
0.721
20.42
105.4
44.86
0.267
DIAMOND
0.6130
14.05
382.4
104.50
0.3267
Dreamer 4
0.2718
16.27
984.1
365.67
0.6050
DAT ( HG-H )
0.721
20.43
101.5
42.77
0.266
Table 3: Robot Manipulation on RT-1. We evaluate against baselines sharing the diffusion-based world model backbone architecture and training budget (except DIAMOND and Dreamer 4); rows differ only in history tokenization method (SD-VAE, ElasticTok, or DAT ) For all methods, we auto-regressively roll out 16 frames from 8 context frames. Details in Appendix C.7 .
Figure 6: Spatial Distribution of Prediction Errors. While ElasticTok and SD-VAE accumulate error around the robot end-effector and contact points, regions most critical for accurate physical interaction, Compression Forcing maintains high fidelity in these dynamic regions across the full prediction horizon.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Encoder Architecture
Model
ViViT
Depth
12
Hidden size
768
Patch size
16
Attention heads
12
Appendix
Table 4: Shared DAT hyperparameters. We share our encoder, decoder, diffusion, and training configurations are across environments, as listed in this table, unless stated otherwise in environment-specific sections.
RT-1
RT-1
RT-1
RT-1-X
(Begin)
(15%)
(Conv.)
Ground Truth
0
70.4
81.5
51.9
CF (Ours)
2
54
64
40
ElasticTok
6
40
48
36
SD-VAE
2
36
32
30
Appendix
Table 5: Policy evaluation success rates (%). Each policy is evaluated in the real environment (Ground Truth) and in three world models.
Figure 7: Policy evaluation correlation. Each line represents a world model; each marker denotes a policy. Points closer to the y=x diagonal indicate more faithful policy ranking.
Figure 8: Object Tracking Evaluation Overview. Given the ground-truth frame at generation onset, SAM extracts class-agnostic object masks from which query points are uniformly sampled. CoTracker3 then tracks these points across both ground-truth and predicted videos.
Method
Base ↓
Penalty ↓
Total ↓
Tracked%
Win Rate
ElasticTok
0.871
3.854
4.725
81.3%
6.2%
SD-VAE
1.473
2.624
4.097
91.3%
45.3%
DAT (Ours)
1.462
2.451
3.914
90.8%
48.4%
Appendix
Table 6: Object Tracking Evaluation on RT-1. We measure tracking fidelity across 64 episodes using CoTracker3 with SAM-initialized query points.
Figure 9: Object Tracking Trajectories on RT-1. CoTracker3 trajectories overlaid on ground-truth and predicted frames.
Figure 10: Extended generation results on Kinetics-600. Comparison between ground truth and generated videos across diverse natural motion sequences. Conditioned on 16 frames as context, autoregressive generation.
Figure 11: Extended generation results on CS:GO. Comparison between ground truth and generated videos across diverse gameplay scenarios. The first four frames are provided as context; subsequent frames are autoregressively generated.
Figure 12: Action following: scope transitions in CS:GO. Top: the player exits the scope view; the model correctly captures the transition from zoomed-in to standard field of view. Bottom: the player enters scope mode; the generated frames accurately reflect the visual change. Zoom in for details.
Figure 13: Environment stochasticity: player respawn in CS:GO. Upon death, CS:GO teleports the agent to a random spawn location. In both examples, the model generates a plausible but incorrect post-respawn scene, reflecting the inherent stochasticity of the underlying environment. Zoom in for details
Figure 14: Extended generation results on RT-1. Comparison between ground truth and generated videos across diverse manipulation tasks. The first eight frames are provided as context; subsequent frames are autoregressively generated. Zoom in for details.
Figure 15: Extended Tokenization Attention Maps. Cross-attention maps between DAT slot tokens and image patches in the dynamic decoder across additional samples. Consistent with Figure 2 , earlier slots attend to regions of active motion, while later slots progressively shift toward static background elements, confirming the generality of DAT ’s dynamics-to-static ordering.
Figure 16: Extended Visualizations of Pixel-wise Prediction Error . Compression Forcing induces less error compared to both baselines (ElasticTok and SD-VAE), specially around regions of dynamic contact.
Anticipating diverse future states is a central challenge in video world modeling. Discriminative world models produce a deterministic prediction that implicitly averages over possible futures, while existing generative world models remain computationally expensive. Recent work demonstrates that predicting the future in the feature space of a vision foundation model (VFM), rather than a latent space optimized for pixel reconstruction, requires significantly fewer world model parameters. However, most such approaches remain discriminative. In this work, we introduce DeltaTok, a tokenizer that encodes the VFM feature difference between consecutive frames into a single continuous "delta" token, and DeltaWorld, a generative world model operating on these tokens to efficiently generate diverse plausible futures. Delta tokens reduce video from a three-dimensional spatio-temporal representation to a one-dimensional temporal sequence, for example yielding a 1,024x token reduction with 512x512 frames. This compact representation enables tractable multi-hypothesis training, where many futures are generated in parallel and only the best is supervised. At inference, this leads to diverse predictions in a single forward pass. Experiments on dense forecasting tasks demonstrate that DeltaWorld forecasts futures that more closely align with real-world outcomes, while having over 35x fewer parameters and using 2,000x fewer FLOPs than existing generative world models. Code and weights: https://deltatok.github.io.
Tommie Kerssies, Gabriele Berton, Ju He +5
Amazon · Eindhoven University of Technology · Johns Hopkins University
A common approach to world-model simulation for vision-language-action (VLA) systems is to predict future RGB observations and then re-encode them into policy inputs, introducing an indirect interface between simulation and downstream policy execution. We instead investigate whether world dynamics can be modeled in a compact, policy-oriented state derived from VLM visual tokens. A key challenge is that raw VLM visual tokens are high-dimensional, making efficient and accurate autoregressive dynamics modeling challenging. To address this, we introduce Token-World, an action-conditioned world model that compresses VLM features into a compact token state, learns future dynamics in this reduced space, and maps predicted states back to the original policy-facing representation for downstream use. Across manipulation benchmarks, Token-World improves open-loop feature fidelity and policy-action consistency over recent world-model simulators, with slower degradation over long rollout horizons. In closed-loop evaluation, its simulated policy performance correlates more strongly with reference policy performance than Ctrl-World (r=0.794 vs.\ 0.583), while requiring lower simulation latency. Ablations further show that compact-representation design and dimensionality substantially affect future-state prediction. Code will be available at https://chuyaofu.github.io/Token-World/.
Chuyao Fu, Xiaowei Chi, Yuhan Rui +14
Southern University of Science and Technology · MUKA Robotics · Hong Kong University of Science and Technology +3
World models trained via pixel reconstruction can struggle in visually complex environments, where irrelevant information dominates the objective and distract the model from information relevant to planning and control. We present Contrastive World Models, an approach for learning latent dynamics models without pixel reconstruction. Building on Dreamer, we replace observation reconstruction in the standard world model objective with a Deep InfoMax-like lower bound that maximizes the mutual information between state-action sequences and local patch features of future observations, encouraging state representations to retain information that is predictive of the future without requiring the model to reconstruct visually irrelevant details. We evaluate our approach in small-scale experiments across three settings of increasing visual complexity. Our method matches Dreamer and a momentum prediction baseline in the default setting, and substantially outperforms both once distractors or natural video backgrounds are introduced, while also training more efficiently by removing the pixel decoder entirely. Our approach is general and makes minimal assumptions beyond access to state-action sequences and future observations. These results suggest that contrastive, infomax-based objectives are a principled and promising direction for building world models that are robust to visual nuisance factors, a property particularly relevant for transferring model-based RL agents to the real world.