World Model-Based Planning
Overview
World model-based planning studies how an agent can predict the consequences of actions and use those predictions to reach goals, avoid risks, or coordinate with others. Models may predict images, compact latent states, object relations, or physical dynamics, and planners use methods such as model-predictive control, CEM, tree search, and learned action policies. Work spans robot manipulation and navigation, autonomous driving, games, and language-guided agents, with approaches including hierarchical subgoals, uncertainty estimates, and symbolic or physics-based structure. A central challenge is that accurate-looking or low-error predictions may still rank actions poorly, especially over long horizons or under distribution shift, so researchers are developing planning-focused representations, diagnostics, and more efficient closed-loop adaptation.
Latest papers
263 papers70 papers in the last four weeks, up 63% on the four weeks before. 0.4% of all new papers.
Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving
Policy-Guided World Model Planning for Language-Conditioned Visual Navigation
World Reasoning Arena
Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation
Temporal Straightening for Latent Planning
MWM: Mobile World Models for Action-Conditioned Consistent Prediction
Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model
ICPRL: Acquiring Physical Intuition from Interactive Control
GeoWorld: Geometric World Models
OOWM: Structuring Embodied Reasoning and Planning via Object-Oriented Programmatic World Modeling
Compositional Planning with Jumpy World Models
JEPA-Bisim: Learning Robust Visual Representations for Planning with Joint-Embedding Predictive World Models
H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
ITP), a unified framework for agent learning via lookahead imagination, where an agent's policy model interacts with the learned world model, yielding multi-step ``imagined'' trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially observable and imaginable Markov decision process to guide policy learning. We instantiate ITP with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that ITP significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents' reasoning capability, providing valuable insights into addressing broader, complex tasks. Our code and data will be publicly available at https://github.com/loyiv/ITP.