SPT-Flow: One Integration, Many Futures in Continuous-Time World Models

Mingyang Lyu; Yinqian Sun; Gongce Cao; Yuhao Zhang; Lu Jia; Qian Zhang; Bo Wang; Pin Lyu

cs.LGOct 30, 2026

SPT-Flow: One Integration, Many Futures in Continuous-Time World Models

Authors: Mingyang Lyu, Yinqian Sun, Gongce Cao, Yuhao Zhang, Lu Jia, Qian Zhang, Bo Wang, Pin Lyu

Organizations: Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · Zhongguancun Academy · Xi’an Jiaotong University ·

Abstract

Continuous-time world models generate a latent trajectory through a single ODE integration, but adding stochastic conditioning does not ensure that the model uses it. In our teacher-forced baseline, posterior means vary by only about 1% of the prior scale, and matched and swapped latents yield indistinguishable reconstruction PSNR at the reported precision. We study a supervision-shortcut hypothesis: ground-truth intermediate states can reduce the need for trajectory-associated latent information. We introduce SPT-Flow, which samples one latent per trajectory and matches a free-running ODE rollout to demonstration states at multiple readout times. Re-anchoring experiments support the proposed mechanism. On LIBERO, with both models trained for 30k optimizer steps, SPT-Flow improves best-of-10 PSNR over our retrained ODEWorld baseline by 2.0 and 2.4 dB at the mid-trajectory and final frames and reduces mean LPIPS by 23%, with comparable per-sample inference cost. The rollout control recovers most of the accuracy gain; learned latent conditioning adds diversity and best-of-K gains while reducing mean sample accuracy relative to this control. The diagnostics support trajectory-associated latent utilization; semantic mode coverage under matched conditions in the robotic setting remains unverified.

Figures & tables

Figure 1: Predictions conditioned on the same initial frame and instruction. ODEWorld produces one trajectory; SPT-Flow samples three, each through one integration. Rows show $t/T=0.6$ and $1.0$ . The example is selected for diversity; model settings follow Figure 5 .

Figure 2: SPT-Flow and its training objective. (a) Teacher-forced velocity supervision (Equation 2 ) evaluates the field at a ground-truth latent $z^{\star}_{\tau}$ , which can reveal the demonstrated continuation. (b) Rollout matching (Equation 4 ) samples one $\epsilon$ , integrates from $z_{0}$ , and supervises predictions at $G$ readout times without intermediate ground-truth inputs. (c) Architecture: snowflakes indicate frozen modules and dashed paths are used only during training. At inference, $\epsilon$ is sampled from $p_{\psi}(\epsilon\mid z_{0},z_{g})$ .

$60$ k steps, three seeds per row
	mid-trajectory		final frame
Model	best-of-10	mean	best-of-10	mean	diversity
ODEWorld (PT-Flow)	$17.81$ $\pm.22$	$17.81$ $\pm.22$	$15.98$ $\pm.17$	$15.98$ $\pm.17$	$0.00$
+ latent, teacher-forced	$18.22$ $\pm.15$	$17.86$ $\pm.23$	$16.20$ $\pm.24$	$15.87$ $\pm.26$	$1.46$
+ rollout control	$19.33$ $\pm.06$	$\mathbf{19.02}$ $\pm.05$	$18.18$ $\pm.20$	$\mathbf{17.77}$ $\pm.19$	$0.51$
SPT-Flow (both)	$\mathbf{19.82}$ $\pm.10$	$18.85$ $\pm.10$	$\mathbf{18.42}$ $\pm.08$	$17.29$ $\pm.10$	$\mathbf{2.03}$

Table 1: Results on the LIBERO test set (

250

trajectories), with

K=10

samples. PSNR is reported in dB; diversity denotes mean pairwise terminal latent distance. Both training budgets use three seeds; standard deviations are shown for

30

k steps, and means only for

60

k steps. The rollout control is trained with

\epsilon=0

but retains random conditioning at evaluation, as discussed in the text. Bold indicates the highest PSNR within each training budget.

Figure 3: Best-of-

K

PSNR as a function of sample count at

30

k training steps. Lines show means across three seeds, and shaded bands indicate the minimum-to-maximum seed range. Rollout matching improves performance at

K=1

, while SPT-Flow exhibits larger gains as

K

increases. ODEWorld is independent of

K

Training	specificity	verdict	prior best-of-10	own	swapped
teacher-forced, $30$ k	$0.007$ – $0.012$	not specific $(0/3)$	$17.71$	$17.34$	$17.34$
teacher-forced, $60$ k	$0.010$	not specific	$18.37$	$18.04$	$18.04$
teacher-forced, low KL floor	$0.002$	not specific	$17.56$	$17.10$	$17.10$
rollout matching, $30$ k	$\mathbf{0.306}$ – $\mathbf{0.376}$	specific $(3/3)$	$19.45$	$\mathbf{18.96}$	$17.91$

Table 2: Both diagnostics of Section 3.4 are applied to trained models on held-out trajectories. Specificity measures posterior-mean variation normalized by the prior scale; the threshold

0.25

was fixed before evaluation. Own and swapped use latents inferred from the target trajectory and another trajectory, respectively. PSNR is measured at the true temporal midpoint and is comparable within this evaluation protocol. At

30

k steps, PSNR uses

250

trajectories across three seeds; specificity ranges use

40

trajectories. The

60

k and low-floor settings use seed

0

and

40

trajectories.

	ODEWorld	rollout control	SPT-Flow
LPIPS, mean of $K$	$0.181$ $\pm.008$	$\mathbf{0.129}$ $\pm.001$	$0.139$ $\pm.003$
LPIPS, best of $K$	$0.181$ $\pm.008$	$0.122$ $\pm.001$	$\mathbf{0.116}$ $\pm.004$
FVD	$265$ $\pm 41$	$210$ $\pm 5$	$\mathbf{187}$ $\pm 9$

Table 3: Perceptual and distributional evaluation across three seeds (mean

\pm

standard deviation; lower is better). LPIPS ( Zhang et al., 2018 ) is evaluated at the mid-trajectory frame on

50

held-out trajectories with

K=10

. FVD ( Unterthiner et al., 2018 ) uses

200

recorded reference clips and

50

generated clips for ODEWorld or

100

for each other model. The rollout control has no learned trajectory latent but retains random conditioning during evaluation.

Ground truth enters at	specificity	swap gap	mid best-of-10	final best-of-10	div.
every training signal (teacher forced)	$0.009$ $[.007,.012]$	$0.00$	$18.22$	$16.20$	$1.46$
every readout of the rollout	$0.236$ $[.228,.245]$	$1.07$	$19.94$	$18.58$	$2.67$
every other readout	$0.298$ $[.243,.359]$	$0.97$	$19.72$	$18.55$	$2.95$
never ( SPT-Flow )	$0.332$ $[.306,.376]$	$1.05$ $[.78,1.19]$	$19.82$	$18.42$	$2.03$

Table 4: Ground-truth states are withdrawn from the training rollout in three stages. Rollout variants share the same posterior; teacher forcing follows Equation 2 . Specificity is reported as the mean [minimum, maximum] across three seeds. PSNR and diversity are seed means. The swap gap is own-minus-swapped mid-trajectory PSNR in dB, measured on seed

0

except for SPT-Flow , which reports the three-seed mean and range.

Configuration	mid best-of-10	mid mean	final best-of-10	div.	specificity
$\beta=10^{-3}$	$19.79$	$18.83$	$18.36$	$1.56$	$0.44$
reference ( $\beta=3\times 10^{-3}$ )	$19.82$	$18.85$	$18.42$	$2.03$	$0.31$ – $0.38$
$\beta=10^{-2}$	$19.53$	$18.82$	$17.98$	$1.06$	$0.20$
latent dimension $16$	$19.81$	$18.71$	$\mathbf{18.64}$	$2.66$	$\mathbf{0.58}$
$16$ Euler steps	$\mathbf{19.85}$	$\mathbf{19.13}$	$18.34$	$0.95$	$0.26$

Table 5: Each variant changes one setting relative to the reference:

\beta=3\times 10^{-3}

, latent dimension

32

, and

8

Euler steps during training. Variants use one seed; the reference reports three-seed means and the specificity range. PSNR is in dB.

$T$	mid best-of-10	mid mean	final best-of-10	final mean	div.
$0.5$	$19.53$ $\pm.07$	$\mathbf{18.95}$ $\pm.09$	$18.12$ $\pm.09$	$\mathbf{17.40}$ $\pm.09$	$1.08$
$0.7$	$19.68$ $\pm.09$	$18.91$ $\pm.09$	$18.25$ $\pm.08$	$17.36$ $\pm.10$	$1.48$
$1.0$	$19.82$ $\pm.10$	$18.85$ $\pm.10$	$18.42$ $\pm.08$	$17.29$ $\pm.10$	$2.03$
$1.3$	$\mathbf{19.91}$ $\pm.10$	$18.77$ $\pm.10$	$\mathbf{18.51}$ $\pm.08$	$17.21$ $\pm.11$	$\mathbf{2.52}$

Table 6: Temperature

T

scales the prior standard deviation at inference, without retraining;

T=1

is the default and SPT-Flow is trained for

30

k steps. PSNR is reported as mean

\pm

standard deviation across three seeds, and diversity as the seed mean.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

$10$ tasks	$10$ k	$20$ k	$30$ k
ODEWorld	$18.37$	$18.85$	$18.98$
+ latent, tf	$18.44$	$19.28$	$19.45$
SPT-Flow	$\mathbf{19.73}$	$\mathbf{20.00}$	$\mathbf{19.82}$

Appendix

Table 7: A

10

-task subset (

450

training trajectories): the same runs at matched intermediate steps, one seed, mid-trajectory best-of-10 on that subset’s

40

-trajectory development split (not the test split used elsewhere).

	ODEWorld		SPT-Flow
Step	mid	final	mid best-of-10	mid mean	final best-of-10	diversity
$30$ k	$18.10$	$15.87$	$19.45$	$18.49$	$18.19$	$1.56$
$60$ k	$18.10$	$15.71$	$19.88$	$19.17$	$18.67$	$0.98$
$90$ k	$19.00$	$16.81$	$19.85$	$19.17$	$18.85$	$0.83$
$120$ k	$19.07$	$16.88$	$\mathbf{20.04}$	$\mathbf{19.37}$	$\mathbf{18.95}$	$0.68$

Appendix

Table 8: Seed 0 of each arm trained for

120

k steps on a different accelerator (RTX 5090; other tables use A100s, and the reference model reproduces its Table 1 numbers on this node to four decimals),

250

-trajectory test split,

K=10

. A second seed is reported in the text. Models taken partway through share the

120

k cosine schedule and are not comparable with the endpoints of Table 1 .

Figure 4: Four teleoperated demonstrations of one instruction ( open a drawer and put a snack in the bowl ) on a real arm, one per row; fixed camera except the wrist-camera grasp column. All start with both drawers closed and diverge in which drawer is opened and which snack is taken. $t/T$ is normalized time. Recorded in separate sessions; an illustration of the premise, not an evaluation.

Teacher-forced configuration	mid best-of-10	mid mean	specificity	max. dim.	swap gap
reference ( $\beta=3\times 10^{-3}$ , floor $0.05$ )	$18.52$	$18.19$	$0.016$	$0.026$	$0.001$
$\beta=10^{-3}$	$18.67$	$18.34$	$0.010$	$0.015$	$0.000$
free-bits floor $0.2$	$18.72$	$18.34$	$0.027$	$0.045$	$0.001$

Appendix

Table 9: Teacher-forced training with one KL knob changed; two GPUs with twice the per-GPU batch (effective batch

16

) on the node of Appendix D ,

30

k steps, seed 0,

250

-trajectory test split,

K=10

. The reference row is retrained on the same node, so rows are comparable with each other but not with Table 1 .

Figure 5: One test trajectory ( put both the cream cheese box and the butter in the basket ), six of $50$ frames, decoded through the shared frozen decoder. Rows: recorded continuation, ODEWorld, three samples from the teacher-forced latent model, three samples from SPT-Flow ; all seed- $0$ models at $30$ k steps from Table 1 .

World Model Learning

World model learning builds predictive models of how environments change under actions to support planning, simulation, and decision-making.

See all 423 papers

Aug 25, 2026

LeFlow: Generative Latent Flow Planning for World Models

Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online trajectory optimization for action planning: for every state-goal pair, an iterative optimizer is run from scratch to search for optimal action sequences, treating the world model as a black-box simulator. This approach pays the full iterative optimization cost anew at every replanning step and reuses no planning experience across queries. In this work, we ask whether planning itself can be amortized once a latent world model has been learned. We present LeFlow, which learns a reusable latent trajectory prior operating directly in the latent dynamics space from the world model. LeFlow recasts planning as conditional latent trajectory generation: a rectified-flow model imagines a future latent path between the current and goal embeddings, an inverse dynamics decoder turns latent transitions into action chunks, and the frozen world model verifies each candidate by autoregressive rollout. Across four major goal-conditioned pixel-control benchmarks, LeFlow replaces iterative action-space optimization with amortized latent planning and fixed-budget rollout selection, achieving consistent success-rate gains with an order-of-magnitude reduction in planning time. Our results argue that latent world models should support not only prediction but reusable planning priors. Our code is available at https://github.com/hsiangwei0903/LeFlow.

Hsiang-Wei Huang, Jianxu Shangguan, Junbin Lu +1

University of Washington, United States

Latent-Space Planning World Model-Based Planning Latent World Models

Ask PDF

Sep 23, 2026

Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model

Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.

Xiwen Chen, Rigaudiere Z. Li, Zhiruo Zhou +2

Shenzhen International Graduate School, Tsinghua University · Shanghai Jiaotong University · Wuhan University of Technology

World Model Learning Latent Dynamics Modeling Latent World Models Flow-Based Generative Modeling

Ask PDF

Jul 30, 2026

ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow

In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete-time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical-Time Flow (PT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well-structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT-Flow, we construct ODEWorld, a continuous-time latent world model that is both efficient and versatile. By extracting time-variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long-standing representation collapse issue in latent world model literature. This also enables high-quality image reconstruction even after long-horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete-time models. Lastly, ODEWorld can provide rich planning-oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning-conducive dynamics abstraction with visual realism, excelling in both video generation and robotic control. Project page: https://odeworld.github.io/.

Dongxiu Liu, Haoyi Niu, Peng Cheng +5

Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · 2Berkeley Artificial Intelligence Research (BAIR), University of California, Berkeley, CA, USA

Robotic Control Latent Dynamics Modeling World Models Video Prediction Latent World Models

Ask PDF

Abstract

World Model Learning

Explore similar work

LeFlow: Generative Latent Flow Planning for World Models

Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model

ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow