cs.CVOct 12, 2026

RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models

Authors: Yaoyang Liu, Dongyu Yan, Yihua Du, Shengju Qian, Zeke Xie, Xin Wang, Ying-Cong Chen, Long Chen

Organizations: HKUST · HKUST(GZ) · LIGHTSPEED

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Video-based interactive world models aim to enable real-time, user-controllable environments. Yet most existing systems are trained and rolled out at a fixed temporal rate, limiting both runtime-rate adaptation and physical-horizon coverage -- a limitation we term temporal-rate coupling. Specifically, when runtime generation throughput differs from the learned temporal rate, the generated world evolves unintentionally more slowly or quickly. Moreover, using a single temporal rate imposes an inherent trade-off between temporal granularity and physical horizon: higher rates capture finer motion but cover a shorter duration under a fixed context window. To address both limitations, we introduce Rate-Adaptive Rollout (RAR), a framework that enables interactive world models to roll out at controllable temporal rates. At its core, Stride-RoPE aligns latent frames sampled at different rates on a shared physical-time coordinate, enabling consistent multi-rate rollout, unseen-rate generalization, and online rate switching without discarding history. Meanwhile, RAR uses multi-stride video sampling to enable joint learning of fine-grained dynamics and long-horizon evolution. Across action- and camera-controlled environments, RAR achieves temporally consistent multi-rate rollout and online rate switching while preserving high-rate generation quality and extending physical horizons under matched training budgets. RAR also transfers to pretrained world models while preserving perceptual quality.

Figures & tables

Figure 1: (a) : Rate–throughput mismatch . ftrainf_{\mathrm{train}} , frolloutf_{\mathrm{rollout}} , and rr denote the training temporal rate, target rollout rate, and runtime generation throughput. After 1s of wall-clock time, colored cars mark generated positions, while gray cars mark the expected positions; their offsets indicate temporal mismatch. (b) : Rate–horizon trade-off . Under the same fixed budget of NN frames, a higher rate provides finer temporal resolution but covers a shorter physical horizon, whereas a lower rate uses larger temporal steps to cover a longer horizon.
Figure 2: Stride-RoPE . Video sampled with temporal stride ss uses RoPE positions pn=nsp_{n}=ns , ensuring that equal physical-time intervals induce equal relative phases across sampling rates.
Figure 3: RAR pipeline. (a) Multi-stride training and rate-controlled rollout. (b) Rollout-rate switching with stride-aware temporal spacing.
Figure 4: Qualitative multi-rate generation. Within each environment, rows share the same initial condition and underlying physical-time control trajectory, while columns are aligned by physical time. Frame indices are marked at the lower left of each frame. At matched physical times, RAR produces similar scene states across rollout rates.
EnvironmentFPSTypeHorizon (s)LPIPS ↓\downarrowDriftDOVER ↑\uparrowSR ⁣→ ⁣1\mathrm{SR}\!\rightarrow\!1
Doom14Train15.10.42791.170.32050.923
 28Interp.7.60.41791.220.32691.003
 35Train6.10.39911.270.31530.999
 49Interp.4.30.40031.260.31560.986
 70Train3.00.38521.340.30491.082
CS216Train15.00.64891.520.39280.983
Table 1: Multi-rate generation under a fixed rollout budget. Drift is the ratio between last-quarter and first-quarter LPIPS. DOVER is the no-reference overall perceptual quality score in [0,1][0,1] ; higher is better. SR\mathrm{SR} measures rate fidelity and has an ideal value of 1.
Figure 5: Dynamic rollout-rate switching results on FPV model. Dashed curves denote target strides; solid curves show the mean calibrated and regressed stride estimates over retained cases, with shading indicating 95% confidence intervals.
Table 7
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Env.Res.Params.fbasef_{\mathrm{base}}Ftrain\mathcal{F}_{\mathrm{train}}FFBid.DFSF
Doom360p1.8B70{14,35,70}\{14,35,70\}21310k/10−410k/10^{-4}12k/10−412k/10^{-4}5k/10−7, 3 ⁣× ⁣10−85k/10^{-7},\,3\!\times\!10^{-8}
CS2360p1.8B64{16,32,64}\{16,32,64\}24110k/10−410k/10^{-4}10k/10−410k/10^{-4}2k/10−7,3× ⁣10−82k/10^{-7},3\times\!10^{-8}
FPV480p1.3B60{20,30,60}\{20,30,60\}24115k/10−415k/10^{-4}11k/10−411k/10^{-4}5k/10−6,3 ⁣× ⁣10−75k/10^{-6},3\!\times\!10^{-7}
SANA-WM720p1.6B30{10,20,30}\{10,20,30\}9613k/10−53k/10^{-5}—3k/10−5,10−63k/10^{-5},10^{-6}
Appendix
Table 4: Model and training settings. Bid., DF, and SF denote bidirectional training, Diffusion Forcing, and Self-Forcing, respectively. FF is the sequence length in video frames. Bid. and DF entries report steps / learning rate; SF entries report steps / student learning rate / fake-model learning rate.
FPSSingleEmbedRAR
140.1060.9210.923
28 †0.8510.9171.003
351.1100.9360.999
49 †2.4831.3950.986
703.8180.9251.082
MAE (%)109.0813.923.54
Appendix
Table 5: Comparison of rate-control methods. Entries report SR\mathrm{SR} , whose ideal value is 1. †\dagger marks rates unseen by both multi-rate models (Embed and RAR); Single is trained only at the base FPS of each environment. MAE is the equal-weight mean of ∣SR−1∣|\mathrm{SR}-1| over the five displayed rates, expressed as a percentage. Bold indicates the SR closest to 1 in each row or the lowest MAE.
EnvironmentFPSTypeSR ⁣→ ⁣1\mathrm{SR}\!\rightarrow\!1
Doom7Extrap.0.739
 28Interp.1.003
 35Train0.999
 49Interp.0.986
 84Extrap.1.191
CS28Extrap.0.636
Appendix
Table 6: Rate fidelity at out-of-range rollout rates. Out-of-range rates are listed together with in-range interpolation and training rates for comparison. Out-of-range rates lie outside the scope of our claims.
Figure 6: Additional dynamic rollout-rate switching results on Doom and CS2. The evaluation protocol and plot conventions follow Fig. 5 .
Figure 7: Qualitative dynamic rollout-rate switching on Doom, CS2, and FPV. The upper and lower groups show generated frames under smooth and abrupt schedules, respectively.
Figure 8: LPIPS under a fixed training-update budget on Doom. RAR is jointly trained at 14, 35, and 70 FPS, whereas the fixed-rate baseline is trained only at 70 FPS. Light markers denote individual checkpoints, and curves show the corresponding trends.
Figure 9: Long-rollout LPIPS under a fixed training-update budget on Doom. Both models are trained with 213-frame windows and evaluated at 70 FPS using 425- and 849-frame rollouts.
Figure 10: LPIPS under approximately matched wall-clock training time on Doom. RAR is trained with 213-frame multi-rate sequences and compared with fixed-rate models trained directly using 425- or 849-frame sequences at 70 FPS. Each model is evaluated at its indicated rollout length.
More in this topic

Interactive World Models

Interactive world models predict and generate how environments change in response to actions, supporting simulation, planning, and control.

See all 117 papers

Explore similar work

CardsList

  1. Matrix-game 2.0: An open-source, real-time, and streaming interactive world model

    Aug 18, 2025Xianglong He, Chunli Peng, Zexiang Liu +16Action-Conditioned Video GenerationReal-Time Interactive Video Generation

    Skywork AI

  2. Light Interaction: Training-Free Inference Acceleration for Interactive Video World Models

    May 29, 2026Jiacheng Lu, Haoyi Zhu, Sipei Yi +3Real-Time Interactive Video GenerationInteractive World Models

    1Zhejiang University · 2NVIDIA

  3. ReWorld: An Interactive World Model with Long-Horizon Memory

    Aug 24, 2026Zhifei Chen, Luozhou Wang, Guibao Shen +8Interactive World ModelsWorld Model Learning

    HKUST(GZ) · ATH, Alibaba