Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models

Xiaoxiao Lu; Yunlong Dong; Jiahao Shi; Ye Yuan

cs.LGAug 27, 2026

Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models

Authors: Xiaoxiao Lu, Yunlong Dong, Jiahao Shi, Ye Yuan

Organizations: School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · Principia AI

Abstract

World Action Models (WAMs) augment robot policies by predicting how task-relevant scene states may evolve under interaction. Recent WAMs increasingly perform such prediction in latent representation spaces, avoiding full appearance-level generation while preserving control-relevant information. Yet latent transitions are commonly realized with Transformer-based predictors whose inductive structure is centered on token interaction rather than temporal evolution. We study transition realization as an architectural choice distinct from predictive representation and prediction-policy coupling. We introduce the Latent Evolution Operator Network (LEON), which models latent evolution in a learned observable space through context-modulated operator-based propagation and additive forcing. Grounded in the controlled Koopman generator view of evolution, LEON organizes context-dependent transition variation around a shared evolution-operator structure while retaining a complementary path for additive change. Controlled dynamical systems verify the resulting evolution-specific inductive bias and the complementary roles of operator propagation and forcing. Across two WAM formulations that integrate latent prediction into the policy differently, LEON improves closed-loop performance and robustness while remaining effective under full transition replacement. These results establish transition realization as a consequential architectural choice in latent WAMs.

Figures & tables

Figure 2: Prediction–policy coupling in latent WAMs. Both formulations use an action-conditioned latent transition

T_{\theta}(X,Z)

to predict a future latent

X^{+}

, but differ in how the prediction contributes to action generation. (a) In representation-mediated coupling , future prediction provides a training-time signal that shapes policy-relevant representations, while the predicted future is not consumed at inference. (b) In policy-facing coupling , the predicted future remains in the inference path and directly conditions the action expert. Solid arrows denote inference-time computation, and dashed arrows denote training-only predictive signals.

Design component	VLA-JEPA + LEON	LaWAM + LEON
Prediction–policy coupling	Representation-mediated	Policy-facing
Transition depth	8 LEON layers	12 LEON layers
Transition-state / observable dimensions	1024 / 768	1024 / 768
Operator factor width $r$	96	96
Residual scale $\eta$	0.35	0.10
Conditioning representation $Z$	Timestep-wise latent-action tokens	Global latent-action vector

Table 1: LEON configurations in VLA-JEPA and LaWAM. The predictive target and policy coupling are preserved; only transition realization is replaced.

Family	Method	Libero-Spatial	Libero-Object	Libero-Goal	Libero-10	Avg.
Mainstream VLA	OpenVLA-OFT [ 29 ]	97.6	98.4	97.9	94.5	97.1
	$\pi_{0}$ [ 2 ]	98.0	96.8	94.4	88.4	94.4
	$\pi_{0.5}$ [ 21 ]	98.8	98.2	98.0	92.4	96.9
	GR00T-N1.6	97.7	98.5	97.5	94.4	97.0
	GR00T N1 [ 3 ]	94.4	97.6	93.0	90.6	93.9
	$\pi_{0}$ -Fast [ 30 ]	96.4	96.8	88.6	60.2	85.5

Table 2: LIBERO closed-loop success rate (%). Published baseline values are reproduced from the benchmark tables reported in VLA-JEPA and LaWAM [ 8 , 7 ] .

Method	Training scope	Evaluation	Lift Pot	Beat Block Hammer	Dump Bin Bigbin	Hanging Mug	Avg.
LaWAM	Published 50-task mixture	Clean	100	90	97	51	84.50
LaWAM	Published 50-task mixture	Randomized	99	93	95	43	82.50
LaWAM	Matched 4-task protocol	Clean	100	90	97	51	84.50
LaWAM	Matched 4-task protocol	Randomized	99	93	95	51	84.50
LaWAM + LEON	Matched 4-task protocol	Clean	100	91	94	55	85.00
LaWAM + LEON	Matched 4-task protocol	Randomized	100	93	93	47	83.25

Table 3: Success rates on the four-task RoboTwin 2.0 evaluation subset (%). The published reference reports the corresponding four-task results of LaWAM [ 8 ] trained on the full 50-task mixture. The matched comparison uses the same four-task training and evaluation protocol for LaWAM and LaWAM + LEON.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Avg.
UniVLA [ 32 ]	1.8	46.2	69.6	69.0	81.0	21.2	31.9	42.9
OpenVLA-OFT [ 29 ]	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6
$\pi_{0}$ [ 2 ]	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
$\pi_{0}$ -Fast [ 30 ]	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
WorldVLA [ 4 ]	0.1	27.9	41.6	43.7	17.1	10.9	38.0	25.0
VLA-JEPA [ 7 ]	63.3	67.1	85.4	95.6	93.6	66.3	85.1	79.5

Table 4: LIBERO-Plus success rate under seven perturbation families (%). Published baselines are reproduced from VLA-JEPA [ 7 ] ; VLA-JEPA + LEON is evaluated using the same seven-category protocol.

Figure 3: Controlled tests of extrapolation, regime dependence, and transition-path usage. (a) Damped spring under an amplitude shift: amplitude-OOD state RMSE versus rollout time on a logarithmic scale; curves show means over five seeds. (b) Nonlinear pendulum across energy regimes: 8-s cumulative angle RMSE over the complete rollout; error bars denote mean

\pm

SD over three seeds, and the dashed line marks the separatrix (

E\approx 19.62

). (c) Driven oscillator post-hoc path intervention: relative OOD rollout RMSE after removing either the operator or forcing path, normalized by the full-model mean; bars show means over ten seeds. Lower is better in all panels.

Model	Scale-equiv. error	20-s energy MAE	Energy-balance residual
Transformer	$6.20\times 10^{-2}$	5.3546	0.0387
Fixed Koopman	$\mathbf{4.60\times 10^{-8}}$	3.0315	0.0223
Adaptive Koopman	$8.66\times 10^{-5}$	0.0554	0.0018

Table 5: Damped spring: physical fidelity under an amplitude shift. Metrics are evaluated on amplitude-OOD rollouts; lower is better.

	Single-parameter shifts				Joint shifts $(L,c)$
Model	$c=0.06$	$c=0.24$	$L=0.7$	$L=1.3$	$(0.7,0.06)$	$(0.7,0.24)$	$(1.3,0.06)$	$(1.3,0.24)$
Transformer	0.3637	0.3360	0.6351	1.0508	0.9096	0.3871	1.1981	0.7583
Fixed Koopman	0.3792	0.2834	0.6066	1.0356	0.8638	0.3763	1.1776	0.7446
Adaptive Koopman	0.3344	0.3238	0.5707	1.0559	0.8440	0.3616	1.2068	0.7582

Table 6: Nonlinear pendulum: terminal prediction error under parameter shifts. We report 8-s terminal angle RMSE (rad) under individual and joint shifts in pendulum length

L

and damping

c

. Lower is better.

Model	Matched RMSE $\downarrow$	Cross-condition RMSE	Swap Penalty $\uparrow$	Condition Retrieval $\uparrow$
Fixed structured	0.84688	0.84688	0.00000	50.0%
Transformer	0.68908	0.71303	0.02395	54.5%
Residual MLP	0.22717	0.24707	0.01990	50.3%
Full structured	0.08434	0.24944	0.16510	81.4%

Table 7: Driven oscillator: condition sensitivity and identification. The metrics use symmetric averaging over paired condition swaps.

World Models for Robotics

World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.

See all 386 papers

LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
Oct 8, 2026Shashank Hegde, Alexander Popov, Elie Aljalbout +1World Action Models Latent World Models
NVIDIA
LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments
Jul 27, 2026Pei Liu, Nan Zheng, Lang Zhang +8Robot Policy Learning World Models for Robotics
1The Hong Kong University of Science and Technology (Guangzhou) · 2The Hong Kong University of Science and Technology · 3Leapmotor +1
LeWAM: Latent evolving World Action Model
Sep 23, 2026Xueji Fang, Boqiang Duan, Hua Wu +2Joint-Embedding Predictive Architecture Action-Conditioned World Models
Zhejiang University · Westlake University · Baidu Inc.

Oct 8, 2026J/K move · Enter open · S save

LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC

Shashank Hegde, Alexander Popov, Elie Aljalbout, Nikolai Smolyanskiy

NVIDIA

World Action Models Latent World Models World Model-Based Planning Joint-Embedding Predictive Architecture Model Predictive Control

Open paper AskPDF arXiv

World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1) Alignment: linear probes read robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.

Abstract

World Models for Robotics

Explore similar work

LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC

LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments

LeWAM: Latent evolving World Action Model