K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation
Authors: Santosh Mohan Rajkumar, Sriram Narayanan, Samuel E. Otto, Debdipta Goswami
Organizations: The Ohio State University; Department of Mechanical & Aerospace Engineering (in Columbus, OH, USA) · West Virginia University; Department of Mechanical, Materials, and Aerospace Engineering (in Morgantown, WV, USA) · Cornell University; Sibley School of Mechanical and Aerospace Engineering (in Ithaca, NY, USA)
World models learn internal simulators of environment dynamics for prediction, planning, and control, yet learning them from limited, noisy, and partial observations remains challenging. While much progress has focused on pixel-based observations, many physical systems instead provide vector-valued measurements that reveal only part of the underlying state. We introduce K-PWM, a probabilistic world model for prediction and control under partial observation. K-PWM couples Koopman-bilinear latent dynamics with a nonlinear observation decoder and performs probabilistic latent-state inference through the learned model via extended Kalman filtering and smoothing, without a separately trained inference network. Its bilinear structure enables exact propagation of the maintained Gaussian belief through the latent transition and efficient rollouts for uncertainty-aware model predictive path integral (MPPI) control. For training K-PWM, we present a generalized expectation conditional maximization (GECM) procedure with closed-form updates for the generators, noise covariances, and latent prior, complemented by gradient-based decoder learning. A Koopman-consistent initialization selects the latent dimension from data and initializes the action-dependent dynamics. With limited training data, K-PWM achieves lower long-horizon prediction error than the evaluated baselines on partially observed Gymnasium MuJoCo systems, while substantially reducing training time relative to the strongest prediction baseline. On high-dimensional MuJoCo Menagerie robots, it achieves competitive tracking accuracy with low per-step computation times, demonstrating potential for real-time deployment. These results support structured probabilistic dynamics as a data- and compute-efficient approach to world modeling and control under partial observation.
Figures & tables
Figure 1: World and latent evolution (with K-PWM ).
Table 2Table 3Table 4Table 5
Figure 7: K-PWM +MPPI Control: representative EE trajectory tracking demonstration.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Acronym
Definition
Acronym
Definition
Ac-RKN
Action-Conditional Recurrent Kalman Network
AI
Artificial Intelligence
CM
Conditional Maximization
CPU
Central Processing Unit
DCWM
Discrete Codebook World Model
DK
DeepKoopman
DKUC
Deep Koopman Operator with Control
DVBF
Deep Variational Bayes Filter
E2C
Embed to Control
EDMD
Extended Dynamic Mode Decomposition
EE
End Effector
EKF
Extended Kalman Filter
Appendix
Table 4: Acronyms used throughout the paper.
Figure 8: WMs are inspired by the implicit mental models humans and animals use to anticipate action consequences.
Figure 9: World models enable embodied systems to anticipate action consequences and plan ahead.
Figure 10: An illustration of pixel-based partial observation in a real-world setting.
Figure 11: Two sets of systems in MuJoCo experiments considered.
Environment
nyfull
ny
Masked
Observed indices
Masked indices
(Δt,T)
HalfCheetah
17
9
8
[0:8]
[9:16]
(0.1,12.0)
Hopper
11
7
4
[0:6]
[7:10]
(0.002,0.3)
Swimmer
10
7
3
[0:4]∪[8:9]
[5:7]
(0.1,6.0)
Appendix
Table 5: Partial-observation setting for Gymnasium MuJoCo systems.
Figure 12: The actions with time of the training trajectories for the Gymnasium MuJoCo systems.
Figure 13: The actions with time of the training trajectories for the MuJoCo Menagerie systems.
World model serves as a promising tool to infer environment dynamics under high-dimensional observations and candidate actions. Recently, LeCun's JEPA provides a compelling framework for learning such models in representation space. Its action-conditioned extension plays a central role in visual control and latent-space planning, but leaves a fundamental question: can it recover the controlled dynamics from nonlinear observations? This paper presents a joint identifiability condition for controlled world models with Gaussian latent states, which consists of two coupled components: (1) representation identifiability and (2) transition identifiability. The former depends on the spectral separation property while the latter is related to non-degenerate variation of conditional action. We prove that when this condition holds, minimizing the LeJEPA-style predictive objective can recover both latent states and controlled dynamics in the sense of orthogonal transformation. We further prove that the upper bound of transition prediction error is inversely proportional to the spectral separation margin. We also characterize an attainable amplification of counterfactual prediction error that scales inversely with the weakest conditional action-excitation margin. The theoretical predictions are empirically supported across four nonlinear observation settings.
Xiangteng Zhang, Yang Guan, Bo Zhang +3
School of Vehicle and Mobility, Tsinghua University · 2Didi Voyager Labs, Didi Autonomous Driving · School of Computing and Data Science, The University of Hong Kong +2
Latent world models underpin much of modern model-based control, yet current action-conditioned formulations supervise the next-latent transition with a single, undifferentiated target, forcing a monolithic learning signal to absorb every source of state change. In real world, however, transitions arise from two heterogeneous sources: an action-driven component induced by the agent, and an action-invariant world effect -- the change that would still occur under a null action, dictated by the environment's intrinsic dynamics (e.g., gravity-driven sliding, inertia, contact rebound, and persistent drift). Fusing them into a single target entangles the two inside the latent transition, prevents the model from attributing observed changes to their underlying causes, and undermines the transferability of the learned dynamics. We introduce DWM (Decomposed World Model), a supervision-level framework that operationalizes this decomposition. DWM augments the predictor of a latent world model with an auxiliary world head, regularized by a normalized world-contrastive objective to be action-invariant, while the original pred head is coupled to it via an orthogonality constraint; together, the two signals induce an explicit additive decomposition of the predicted transition into an action-invariant and a complementary action-driven component, without altering the underlying architecture or inference pipeline. To evaluate DWM under persistent world effects, we construct W-variants of three standard control benchmarks -- PushT-W, Reacher-W, and TwoRoom-W -- each instantiating a distinct action-invariant dynamic. DWM matches strong baselines on the flat counterparts and delivers a mean absolute improvement of 13.1% in CEM planning success across the W-variants.
World models jointly learn latent representations and dynamics that predict how high-dimensional observations evolve under actions. In this work, we propose a JEPA-style world model in which, rather than learning arbitrary latent dynamics, we restrict them to follow a bilinear parameterization. This structure enables efficient planning and control while shifting the modeling burden onto the encoder, encouraging richer representations that expose the controllable geometry of the system. In particular, this structured parameterization allows us to structurally enforce action recoverability, thereby preventing representation collapse by construction. Although prescribing a bilinear parametrization may appear restrictive, we show that a broad class of nonlinear dynamical systems admits a transformation under which the dynamics become bilinear. Empirically, we show across standard 2D and 3D control tasks that representations with bilinear-parameterized dynamics can be learned directly from high-dimensional observations, reducing planning time by nearly three orders of magnitude while retaining or even improving control accuracy. We also propose more demanding regimes of longer-horizon planning and real-time control, and demonstrate that our method succeeds in both, moving JEPA-style world models beyond short-horizon offline planning.