Latent World Models

Latent world models encode observations and histories into compact states, then predict how those states change over time or under actions, avoiding the need to reconstruct every pixel. Common approaches include recurrent state-space models such as Dreamer, joint-embedding predictive architectures (JEPAs), and diffusion or flow predictors, often shaped by inverse dynamics, temporal objectives, physical structure, or causal supervision. These models support imagined rollouts and planning in visual reinforcement learning, robot manipulation and navigation, autonomous driving, and other forecasting settings. A central challenge is that accurate short-term predictions or interpretable latents do not necessarily preserve action effects and goal-relevant structure over long horizons, leaving planning evaluation, uncertainty calibration, and robust transfer open.