One-Step Next-Latent Prediction Is Not a World Model

Shitong Wang; Zhongang Cai; Yuzhou Hong

stat.MLSep 28, 2026

One-Step Next-Latent Prediction Is Not a World Model

Authors: Shitong Wang, Zhongang Cai, Yuzhou Hong

Abstract

Next-latent prediction fits a map from the current embedding to the next one. LeNEPA carries this objective to time series, replacing the stop-gradient of next-embedding prediction with the isotropy penalty of LeJEPA. A world model is a transition kernel that can be rolled out. The one-step regression identifies a conditional mean, and a mean is a kernel only in special cases. For a linear-Gaussian Markov latent, the mean transition and the innovation covariance are fixed by the one-step problem, and the open-loop squared error at horizon $K$ equals the trace of the sum of the pushed-forward innovation covariances. That error grows with $K$ after the one-step fit is exact. If the conditional mean is nonlinear, composing it is not the multi-step conditional mean. If the observation is a non-injective function of a Markov state, a memoryless one-step map does not determine future observations, while a short window can. An isotropy penalty is a function of the embedding marginal, so its partial derivative in the transition weights is zero. On a scalar autoregression with coefficient $0.9$ , the one-step mean squared error is $0.998$ and the $16$ -step open-loop error is $5.10$ . On a hidden rotation, an eight-step window reaches $16$ -step error $0.056$ , while the current scalar alone reaches $0.778$ . Raising the isotropy weight from $0.1$ to $10$ leaves eight-step latent error inside $[0.78,0.85]$ on three seeds.

Figures & tables

$K$	one-step $a=0.9001$	unroll $a=0.8930$	oracle $0.9$	analytic
1	0.998	0.998	0.998	0.998
2	1.813	1.813	1.813	1.807
4	3.007	3.009	3.007	2.992
8	4.298	4.300	4.298	4.280
16	5.100	5.101	5.100	5.073

Table 1: Open-loop MSE on the AR(1). The analytic column is equation 5 at

a=0.9

with

\sigma^{2}=0.9982

. Teacher-forced one-step error equals the

K=1

entry at every horizon.

$K$	width $1$ , one-step	width $8$ , one-step	width $8$ , unroll
1	0.165	0.009	0.014
2	0.572	0.015	0.015
4	1.670	0.024	0.021
8	2.268	0.030	0.027
16	0.778	0.056	0.053

Table 2: Open-loop MSE on the hidden rotation, seed

0

. Seeds

1

and

2

keep the same ordering: memoryless error at

K=8

2.602

and

2.537

, and windowed one-step error at

K=8

0.031

and

0.030

	eight-step MSE			off-diagonal energy
$\lambda$	seed 0	seed 1	seed 2	seed 0	seed 1	seed 2
$0$	1.073	2.047	0.419	0.132	0.088	0.019
$0.1$	0.784	0.847	0.781	$8\cdot 10^{-6}$	$8\cdot 10^{-4}$	0.004
$1$	0.799	0.784	0.809	0.001	$3\cdot 10^{-4}$	$8\cdot 10^{-5}$
$10$	0.818	0.845	0.827	0.002	$2\cdot 10^{-8}$	$6\cdot 10^{-5}$

Table 3: Eight-step latent rollout MSE and off-diagonal energy

\|(C-\operatorname{diag}(C))\|_{F}^{2}/4

, three seeds. For

\lambda\geq 0.1

every eight-step entry lies in

[0.781,0.847]

Explore similar work

A Control Theory of Predictability in Latent World Models
Jul 11, 2026Hanzhe You, Yonggang Zhang, Maohao Ran +6Latent World Models Prediction Error
University of Science and Technology of China · The Hong Kong University of Science and Technology, HKGAI · Hong Kong Baptist University
When Does LeJEPA Learn a World Model?
May 25, 2026David Klindt, Yann LeCun, Randall BalestrieroLatent World Models Low-Latency Latent Planning
Cold Spring Harbor Laboratory · New York University · Brown University
Orthogonal JEPA: Factorized Predictive States for Latent World Models
Aug 20, 2026Taoyong Cui, Pheng Ann Heng, Wanli OuyangJoint-Embedding Predictive Architectures Next-State Prediction
The Chinese University of Hong Kong (CUHK)

Jul 11, 2026, cs.LGJ/K move · Enter open · S save

A Control Theory of Predictability in Latent World Models

Hanzhe You, Yonggang Zhang, Maohao Ran, Zhiqin Yang+5

University of Science and Technology of China · The Hong Kong University of Science and Technology, HKGAI · Hong Kong Baptist University

Latent World Models Prediction Error Model Predictive Control Predictability Robust Control World+1

Open paper AskPDF arXiv

Latent world models are trained to predict future states in a learned representation and are then deployed inside a planner that selects actions by simulating them forward. Current practice adopts the prediction error, the single- or multi-step rollout loss on held-out data, as the training and model-selection objective, on the assumption that a lower prediction error yields better control. We show that this assumption is unreliable for a structural reason: a planner does not query the model on the training distribution but on the states that its candidate actions reach, which generally leave the data manifold, so an error averaged over the data cannot by itself govern control. We therefore reframe the objective as the discrepancy between the predicted and the true plan-cost at the plan the planner commits to, and prove that the planner's suboptimality is bounded by twice this discrepancy, whereas the data-averaged prediction error neither bounds nor tracks it. Under a linear-control premise the discrepancy separates into two terms. The first is a small on-manifold residual, on which the predicted and true dynamics agree and which a spectral tax prices through the non-normality of the latent transition operator. The second is an off-manifold divergence, on which an action carries the state off the manifold and the two dynamics diverge; this divergence is the binding term and is bounded by no data-averaged error. Synthetic operators confirm the pricing formulas, and latent model-predictive control experiments confirm the decoupling: across seeds, the single-step validation error is essentially uncorrelated with control success, whereas a fidelity score on the planner-reachable measure tracks it.

Abstract

Explore similar work

A Control Theory of Predictability in Latent World Models

When Does LeJEPA Learn a World Model?

Orthogonal JEPA: Factorized Predictive States for Latent World Models