Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · ETH Zürich · City College of New York, City University of New York, USA · Technical University of Munich · Fuzhou University · Xiamen Institute of Data Intelligence, Xiamen, China
Diffusion-based world models have shown strong potential for unified world simulation, but the iterative denoising remains too costly for interactive use and long-horizon rollouts. While feature caching can accelerate inference without training, we find that policies designed for single-modal diffusion transfer poorly to world models due to two world-model-specific obstacles: token heterogeneity from multi-modal coupling and spatial variation, and non-uniform temporal dynamics where a small set of hard tokens drives error growth, making uniform skipping either unstable or overly conservative. We propose WorldCache, a caching framework tailored to diffusion world models. We introduce Curvature-guided Heterogeneous Token Prediction, which uses a physics-grounded curvature score to estimate token predictability and applies a Hermite-guided damped predictor for chaotic tokens with abrupt direction changes. We also design Chaotic-prioritized Adaptive Skipping, which accumulates a curvature-normalized, dimensionless drift signal and recomputes only when bottleneck tokens begin to drift. Experiments on diffusion world models show that WorldCache delivers up to \textbf{3.7×} end-to-end speedups while maintaining 98% rollout quality, demonstrating the vast advantages and practicality of WorldCache in resource-constrained scenarios. Our code is released in https://github.com/FofGofx/WorldCache.
Figures & tables
Figure 2: Overview of the proposed WorldCache framework. The pipeline alternates between FULL backbone evaluation and CACHE approximation. (Top) In each full computation step, tokens are partitioned into Stable, Linear, and Chaotic groups based on their curvature κ . (Bottom) During caching steps, heterogeneous predictors (Reuse, Linear Extrapolation, or Damped Update) are applied accordingly. (Left) The Chaotic-prioritized Adaptive Skipping (CAS) mechanism accumulates a curvature-normalized drift score Eacc specifically from chaotic tokens, triggering a full computation only when critical drift is detected.
Figure 3: An illustration of token heterogeneity. (a) Modality and Spatial Variance: Distinct patterns between modalities and across spatial regions. (b) Trajectory Dynamics: Three trajectory trends: static, predictable, and sharp, non-linear direction shifts that defy simple extrapolation. More analysis in Appendix Sec. E
Figure 4: Mechanism and effectiveness of the Damped Update. (a) Trajectory Illustration: Damped update stabilizes prediction through historical vt⋆−1 . (b) Quantitative Error Analysis: Damped update reduces chaotic tokens cache error as the prediction window k increases.
Figure 5: An illustration of non-uniform temporal dynamics. We plot the feature difference magnitude across denoising steps for different token percentiles ( p25 to p100 ). The global drift is dominated by a small subset of ”hard” tokens (top percentile, red line), while the majority remain stable. More analysis in Appendix Sec. F .
Method
Benchmark Evaluation
Perceptual Metrics
Acceleration & Memory
WorldScore Static ↑
WorldScore Dynamic ↑
PSNR ↑
SSIM ↑
LPIPS ↓
Latency(s) ↓
Speed ↑
Memory Overhead(GB) ↓
HunyuanVoyager-13B ( Huang et al., 2025 ) ( 512×768p,frames=49 )
Voyager
66.28
46.40
∞
1.000
0.000
1053.7
1.00 ×
50.44
DuCa
53.87
37.71
16.66
0.508
0.486
811.5*
1.30 ×
109.70
ToCa
47.49
33.24
15.51
0.409
0.558
1038.4*
1.01 ×
107.35
TaylorSeer
62.46
43.72
18.32
0.615
0.293
1195.2*
0.88 ×
163.79
Table 1: Quantitative comparison on Image-to-World generation. Bold : the best result. *Note: Layer-wise methods marked with * exceed single-GPU memory limits, requiring CPU offloading which incurs transmission latency.
Table 9: Broader evaluation on longer LingBot rollouts beyond the 49-frame main setting.
Model
WorldCache Overhead (s) ↓
End-to-End Latency (s) ↓
Overhead / Latency ↓
LingBot-14B
0.189
348.7
0.054%
Voyager-13B
0.137
288.6
0.047%
Aether-5B
0.064
107.2
0.060%
Appendix
Table 10: Runtime overhead of WorldCache control logic.
Context Length
Baseline Mem.(GB) ↓
WorldCache Mem.(GB) ↓
Extra Mem.(GB) ↓
Speed ↑
49
49.37
49.39
0.02
2.41 ×
81
55.87
55.89
0.02
2.47 ×
161
71.32
71.35
0.03
2.53 ×
Appendix
Table 11: Memory behavior of WorldCache as context length increases.
Figure 7: More visualization of token heterogeneity. We visualize both RGB tokens and Depth tokens of different prompts across 50 denoising steps in HunyuanVoyager ( Huang et al., 2025 ) . The distinct patterns between RGB and Depth, as well as the spatial variance within each map, underscore the necessity of heterogeneous processing.
Figure 8: Visualization of non-uniform temporal error dynamics. We plot the prediction error accumulation across different token percentiles ( p25 to p100 ). The results show that global variance is dominated by the top percentile of “chaotic” tokens (red line), while the majority of tokens ( p50 and below) remain stable. This supports our strategy of monitoring the Chaotic group rather than the global mean.
Figure 9: Visualization of temporal scale variance. Histograms show the distribution of feature values at different denoising steps. The numerical range fluctuates by orders of magnitude (e.g., thousands at Step 1 vs. single digits at Step 37), proving that fixed thresholds on raw values are unreliable and validating the necessity of our dimensionless drift indicator.
Figure 10: More visual comparison between WorldCache and existing methods.
Figure 11: More visual comparison between WorldCache and existing methods.
Figure 12: More visual comparison between WorldCache and existing methods.
Figure 13: More visual comparison between WorldCache and existing methods.
Diffusion Transformers (DiTs) power high-fidelity video world models but remain computationally expensive due to sequential denoising and costly spatio-temporal attention. Training-free feature caching accelerates inference by reusing intermediate activations across denoising steps; however, existing methods largely rely on a Zero-Order Hold assumption i.e., reusing cached features as static snapshots when global drift is small. This often leads to ghosting artifacts, blur, and motion inconsistencies in dynamic scenes. We propose WorldCache, a Perception-Constrained Dynamical Caching framework that improves both when and how to reuse features. WorldCache introduces motion-adaptive thresholds, saliency-weighted drift estimation, optimal approximation via blending and warping, and phase-aware threshold scheduling across diffusion steps. Our cohesive approach enables adaptive, motion-consistent feature reuse without retraining. On Cosmos-Predict2.5-2B evaluated on PAI-Bench, WorldCache achieves \textbf{2.3×} inference speedup while preserving 99.4% of baseline quality, substantially outperforming prior training-free caching approaches. Our code can be accessed on World-Cache.
Umair Nawaz, Ahmed Heakl, Ufaq Khan +3
Mohamed bin Zayed University of Artificial Intelligence, UAE · Linköping University, Sweden
Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Transformer evaluations during denoising. Existing caching methods mainly exploit temporal redundancy at the feature or token level, leaving the underlying mathematical structure of diffusion features largely unexplored. In this work, we reveal that world-model features exhibit highly stable singular subspaces across nearby denoising steps, while their singular values follow predictable evolution patterns. Building on this observation, we propose SpectralCache, a training-free spectral caching framework that reuses stable singular subspaces and estimates only low-dimensional singular values through linear extrapolation. We further exploit the spectral consistency between neighboring full-computation features to skip selected expensive backbone evaluations via singular value scaling. Extensive experiments on representative world models demonstrate that SpectralCache consistently improves inference efficiency while preserving generation quality. On HunyuanWorld-Voyager-13B, SpectralCache achieves 5.22x acceleration while maintaining a WorldScore of 65.90 for static scenes, substantially outperforming existing training-free caching methods in inference efficiency.
Zhendong Mi, Pu Zhao, Ziyu Hu +4
Stevens Institute of Technology · Northeastern University · Technische Universität Darmstadt
Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse intermediate features, selectively update tokens, or reuse and extrapolate denoising outputs ac- cording to local drift or short native-space histories. These criteria can miss both approximation-induced latent transition defects that accumulate across skipped steps and phase- or condition-dependent changes in the direction of latent evo- lution. We propose WorldDynCache, a risk-controlled latent dynamics approximation framework with two core compo- nents. First, a lightweight latent-transition risk estimator tracks the accumulated future impact of approximation defects and calibrates its predictions against counterfactual defects ob- served at exact anchors. Second, a condition- and phase- aware lifted latent surrogate approximates latent evolution without extra transformer evaluations. On HunyuanVoyager- 13B and Aether-5B, WorldDynCache achieves 4.92 times and 2.15 times speedups, respectively, while attaining the best gen- eration quality among the compared caching methods across WorldScore, PSNR, SSIM, and LPIPS.