World Models

Follow this topic Ask about this topic

World models learn or encode how states, objects, and events evolve, often predicting what will happen under different actions or observations. These predictions can help agents plan ahead, learn with fewer real-world trials, test decisions safely, and reason about situations they cannot currently observe. Approaches range from latent-state predictors and action-conditioned video generators to object- or graph-based models, executable programs, and physics-informed dynamics. A central challenge is making predictions useful and reliable over long horizons: visual plausibility and one-step accuracy do not guarantee correct causal dynamics, persistent memory, or better decisions.

Latest papers

369 papers72 papers in the last four weeks, up 71% on the four weeks before. 0.4% of all new papers.

Coupled Local and Global World Models for Efficient First Order RL
Feb 5, 2026Joseph Amigo, Rooholla Khorrambakht, Nicolas Mansard +1Model-Based RL World Models for Robotics
Machines in Motion Laboratory, New York University, USA · LAAS-CNRS, Universit´e de Toulouse, CNRS, Toulouse, France · Artificial and Natural Intelligence Toulouse Institute, Toulouse, France
MobileDreamer: Generative Sketch World Model for GUI Agent
Jan 7, 2026Yilin Cao, Yufeng Zhong, Zhixiong Zeng +6Mobile GUI Automation World Model Learning
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences · Meituan · School of Artificial Intelligence, University of Chinese Academy of Sciences
Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
Dec 29, 2025Guo Ye, Zexi Zhang, Xu Zhao +4Contact-Rich Robotic Manipulation World Models
Northwestern University, Evanston, IL, USA
Soft Geometric Inductive Bias for Object Centric Dynamics
Dec 17, 2025Hampus Linander, Conor Heins, Alexander Tschantz +2World Model Learning World Models
VERSES AI · University of Sussex, Department of Informatics
MobileWorldBench: Towards Semantic World Modeling For Mobile Agents
Dec 16, 2025Shufan Li, Konstantinos Kallidromitis, Akash Gokul +3World Models Mobile GUI Agents
UCLA · Panasonic AI Research · Salesforce AI Research
Thinking Ahead: Foresight Intelligence in MLLMs and World Model
Nov 24, 2025Zhantao Gong, Liaoyuan Fan, Qing Guo +3VLM Evaluation VLM Adaptation
College of Software, Nankai University, China · The University of Hong Kong, China · NKIARI, Shenzhen Futian, China +2
Deep Active Inference with Diffusion Policy and Multiple Timescale World Model for Real-World Exploration and Navigation
Oct 27, 2025Riko Yokozawa, Kentaro Fujii, Yuta Nomura +1Active Inference Robot Navigation
School of Integrated Design Engineering, Keio University, Yokohama, Kanagawa 223-8522, Japan
RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulation
Oct 10, 2025Chuanrui Zhang, Zhengxian Wu, Guanxing Lu +2World Models for Robotics World Models
Nanyang Technological University, Singapore · Tsinghua University, Beijing, China
MAPF-World: Action World Model for Multi-Agent Path Finding
Aug 16, 2025Zhanjiang Yang, Yueming Li, Yang Shen +2Multi-Agent Path Finding Multi-Agent Planning
Shenzhen Technology University, China. · University of Technology Sydney, Australia.

Feb 5, 2026J/K move · Enter open · S save

Coupled Local and Global World Models for Efficient First Order RL

Joseph Amigo, Rooholla Khorrambakht, Nicolas Mansard, Ludovic Righetti

Machines in Motion Laboratory, New York University, USA · LAAS-CNRS, Universit´e de Toulouse, CNRS, Toulouse, France · Artificial and Natural Intelligence Toulouse Institute, Toulouse, France

Model-Based RL World Models for Robotics World Models Data-Efficient Robot Learning Robotic Manipulation

Open paper AskPDF arXiv

World models offer a promising avenue for more faithfully capturing complex dynamics, including contacts and non-rigidity, as well as complex sensory information, such as visual perception, in situations where standard simulators struggle. However, these models are computationally complex to evaluate, posing a challenge for popular RL approaches that have been successfully used with simulators to solve complex locomotion tasks but yet struggle with manipulation. This paper introduces a method that bypasses simulators entirely, training RL policies inside world models learned from robots' interactions with real environments. At its core, our approach enables policy training with large-scale diffusion models via a novel decoupled first-order gradient (FoG) method: a full-scale world model generates accurate forward trajectories, while a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation. This coupling of a local and global world model ensures high-fidelity unrolling alongside computationally tractable differentiation. We demonstrate the efficacy of our method on the Push-T manipulation task, where it significantly outperforms PPO in sample efficiency. We further evaluate our approach through an ego-centric object manipulation task with a quadruped. Together, these results demonstrate that learning inside data-driven world models is a promising pathway for solving hard-to-model RL tasks in image space without reliance on hand-crafted physics simulators.