K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation
Organizations: The Ohio State University; Department of Mechanical & Aerospace Engineering (in Columbus, OH, USA) · West Virginia University; Department of Mechanical, Materials, and Aerospace Engineering (in Morgantown, WV, USA) · Cornell University; Sibley School of Mechanical and Aerospace Engineering (in Ithaca, NY, USA)
Abstract
World models learn internal simulators of environment dynamics for prediction, planning, and control, yet learning them from limited, noisy, and partial observations remains challenging. While much progress has focused on pixel-based observations, many physical systems instead provide vector-valued measurements that reveal only part of the underlying state. We introduce K-PWM, a probabilistic world model for prediction and control under partial observation. K-PWM couples Koopman-bilinear latent dynamics with a nonlinear observation decoder and performs probabilistic latent-state inference through the learned model via extended Kalman filtering and smoothing, without a separately trained inference network. Its bilinear structure enables exact propagation of the maintained Gaussian belief through the latent transition and efficient rollouts for uncertainty-aware model predictive path integral (MPPI) control. For training K-PWM, we present a generalized expectation conditional maximization (GECM) procedure with closed-form updates for the generators, noise covariances, and latent prior, complemented by gradient-based decoder learning. A Koopman-consistent initialization selects the latent dimension from data and initializes the action-dependent dynamics. With limited training data, K-PWM achieves lower long-horizon prediction error than the evaluated baselines on partially observed Gymnasium MuJoCo systems, while substantially reducing training time relative to the strongest prediction baseline. On high-dimensional MuJoCo Menagerie robots, it achieves competitive tracking accuracy with low per-step computation times, demonstrating potential for real-time deployment. These results support structured probabilistic dynamics as a data- and compute-efficient approach to world modeling and control under partial observation.
Figures & tables
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Acronym | Definition | Acronym | Definition |
|---|---|---|---|
| Ac-RKN | Action-Conditional Recurrent Kalman Network | AI | Artificial Intelligence |
| CM | Conditional Maximization | CPU | Central Processing Unit |
| DCWM | Discrete Codebook World Model | DK | DeepKoopman |
| DKUC | Deep Koopman Operator with Control | DVBF | Deep Variational Bayes Filter |
| E2C | Embed to Control | EDMD | Extended Dynamic Mode Decomposition |
| EE | End Effector | EKF | Extended Kalman Filter |
| Environment | Masked | Observed indices | Masked indices | |||
|---|---|---|---|---|---|---|
| HalfCheetah | 17 | 9 | 8 | |||
| Hopper | 11 | 7 | 4 | |||
| Swimmer | 10 | 7 | 3 |
World Model Learning
World model learning builds predictive models of how environments change under actions to support planning, simulation, and decision-making.
See all 459 papers