World Models for Robotics

Follow this topic Ask about this topic

World models learn to predict what will happen when a robot acts, giving it a way to plan ahead, learn from imagined experience, and reduce costly or risky real-world trials. Approaches range from latent-dynamics models such as JEPA and Dreamer, often paired with model-predictive control or reinforcement learning, to video-based world-action models that predict future observations alongside executable actions. Researchers also add structure—3D geometry, object-centric states, physics, memory, and tactile or force signals—to handle contact, navigation, and long-horizon tasks. A central challenge is that visually convincing predictions may not follow actions faithfully or support reliable control; balancing physical consistency, robustness, evaluation quality, and inference speed remains open.

Latest papers

386 papers81 papers in the last four weeks, up 5% on the four weeks before. 0.5% of all new papers.

MAD: Mapping-Aware World Models for Agile Quadrotor Flight
Jun 3, 2026Xinhong Zhang, Runqing Wang, Yunfan Ren +6Robot Policy Learning World Models for Robotics
State Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China · Zhongguancun Academy, Beijing 100094, China · School of Computer Science and Technology, Tongji University +2
AirDreamer: Generalist Drone Navigation with World Models
Jun 2, 2026Zian Liu, Andong Yang, Chunkai Yang +3RL for Robotics World Models for Robotics
Institute for AI Industry Research, Tsinghua University, Beijing, China. · Department of Electronic Engineering, Tsinghua University, Beijing, China. · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China.
GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
Jun 2, 2026Fulong Ma, Daojie Peng, Wenjun Yue +4World Models for Robotics Next-Action Prediction
HKUST(GZ) · OC · HKU +4
Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group
Jun 2, 2026Hongbo WangWorld Models for Robotics World Models
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA
Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
Jun 1, 2026Shahram Najam Syed, Arthur Jakobsson, Haoran Hao +1World Models for Robotics Robotic Manipulation
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA
World-Task Factorization for Robot Learning
Jun 1, 2026Eduardo Sebastián, Adrian Pfisterer, Vito Mengers +2Robotic Control Robot Policy Learning
Department of Computer Science and Technology, University of Cambridge, United Kingdom · Robotics and Biology Laboratory, Technische Universität Berlin, Germany · Robotics and Biology Laboratory, Technische Universität Berlin +1
Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects
Jun 1, 2026Jens U. Kreber, Lukas Mack, Joerg Stueckler3D Gaussian Splatting Dynamical Systems
Intelligent Perception in Technical Systems Group, University of Augsburg, Augsburg, Germany
Beyond Task Success: Behavioral and Representational Diagnostics for WAM and VLA
May 31, 2026Hung Mai, Bin Zhu, Tuan DoWorld Models for Robotics Robotic Policy Evaluation
National Economics University, Vietnam · N2TP Technology · Singapore Management University +1
$τ_0$ -WM: A Unified Video-Action World Model for Robotic Manipulation
May 31, 2026Pengfei Zhou, Shengcong Chen, Di Chen +17Action-Conditioned Video Generation Long-Horizon Robotic Manipulation
1Shanghai Innovation Institute · 2AGIBOT Finch
AR Forcing: Towards Long-Horizon Robot Navigation World Model
May 29, 2026Yifei Yang, Zehua Fan, Huan Li +9World Models for Robotics Autoregressive Diffusion
Institute for AI Industry Research, Tsinghua University · Shanghai Jiao Tong University · School of Safety Science, Tsinghua University +2
Physically Viable World Models: A Case for Query-Conditioned Embodied AI
May 28, 2026Adam J. Thorpe, Stepan Tretiakov, Cheng-Hsi Hsiao +6World Models for Robotics Causal World Models
The University of Texas at Austin
MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models
May 28, 2026Tianzhuo Yang, Zihan Shen, Zirui Mi +7World Models for Robotics Action-Conditioned World Models
Institute for Artiﬁcial Intelligence, Peking University · 2Physis Lab
World Models for Robotic Manipulation: A Survey
May 27, 2026Fangyuan Wang, Ziyuan Wang, Guorui Pei +15World Model Learning World Models for Robotics
Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China · Department of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen, China · School of Advanced Engineering, Great Bay University, Dongguan, Guangdong, China +9
GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation
May 26, 2026Boxiang Qiu, Liliang Chen, Yue Liao +12Action-Conditioned Video Generation World Models for Robotics
AgiBot · BUAA · LV-NUS Lab +1
Drift-Resistant Navigation World Model with Anchored Epipolar Guidance
May 23, 2026Po-Chien Luan, Zimin Xia, Wuyang Li +2World Models for Robotics Model-Based Planning
Point Tracking Improves World Action Models
May 22, 2026Jiarui Guan, Wenshuai Zhao, Yue Pei +3Point Tracking Robot Policy Learning
Aalto University · ELLIS Institute Finland · Beihang University +3
World Model Control by Trajectory Reachability Metrics
May 21, 2026Liangyu Li, Shengzhi Wang, Libin Qiu +2Reachability Analysis World Models for Robotics
Tongji University
EvoScene-VLA: Evolving Scene Beliefs Inside the Action Decoder for Chunked Robot Control
May 21, 2026Chushan Zhang, Ruihan Lu, Jinguang Tong +3World Models for Robotics Vision-Language-Action Models
Australian National University · The University of Queensland · Beijing Normal University
Demo-JEPA: Joint-Embedding Predictive Architecture for One-shot Cross-Embodiment Imitation
May 20, 2026Jingyang He, Guangrun Li, Jieyu Zhang +3Robot Skill Learning World Models for Robotics
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · University of Washington
GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation
May 20, 2026Zijian Zhang, Yuqing Jiang, Qian Cheng +6World Models for Robotics Robotic Manipulation
Tuojing Intelligence · University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences +5
SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution
May 19, 2026Yiren Song, Yihan Wang, Xiyao Deng +2World Models for Robotics Robotic Manipulation
Show Lab, National University of Singapore · Central South University
WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
May 18, 2026Zhuohang Li, Yiding Ma, Yu Shang +26World Models for Robotics World Models
Tsinghua University · Shanghai Jiao Tong University · Zhejiang University +7
RoboFlow4D: A Lightweight Flow World Model Toward Real-Time Flow-Guided Robotic Manipulation
May 17, 2026Sixu Lin, Junliang Chen, Huaiyuan Xu +8World Models for Robotics Language-Conditioned Robot Manipulation
School of Data Science, The Chinese University of Hong Kong (Shenzhen) · The Hong Kong Polytechnic University · Shenzhen Loop Area Institute +1
DeTrack: A Benchmark and Altitude-Aware Dual World Model for Drone-embodied Tracking
May 17, 2026Guyue Hu, Haoming Liu, Siyuan Song +3Aerial Robotics World Models for Robotics
School of Artificial Intelligence, State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, and Anhui Provincial Key Laboratory of Security Artificial Intelligence, Anhui University, Hefei 230601, China · Hefei Si Valley Technology Development Co., Ltd, and also with the Institute of Embodied Intelligence, Anhui University, Hefei 230601, China · School of Computer Science and Technology, State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, and Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Anhui University, Hefei 230601, China
Feedback World Model Enables Precise Guidance of Diffusion Policy
May 15, 2026Tuo An, Jindou Jia, Gen Li +8World Model Learning World Models for Robotics
MARS Lab, Nanyang Technological University
Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations
May 14, 2026Jonathan Spieler, Angel Villar-Corrales, Sven BehnkeWorld Models for Robotics World Model-Based Planning
Autonomous Intelligent Systems, Computer Science Institute VI - Intelligent Systems and Robotics, Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence, University of Bonn, Germany
3D-Belief: Embodied Belief Inference via Generative 3D World Modeling
May 12, 2026Yifan Yin, Zehao Wen, Suyu Ye +10World Models for Robotics 3D Scene Representation
Johns Hopkins University · Lambda · University of Cambridge
HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models
May 11, 2026Qiuxuan Feng, Jiale Yu, Jiaming Liu +8World Models for Robotics Action-Conditioned World Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Models
May 8, 2026Bo-Kai Ruan, Teng-Fang Hsiao, Ling Lo +1World Models for Robotics World Model-Based Planning
National Yang Ming Chiao Tung University
LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
May 8, 2026Qixin Xiao, Maani GhaffariPhysical Consistency in Video Generation World Models for Robotics
University of Michigan

Jun 3, 2026J/K move · Enter open · S save

MAD: Mapping-Aware World Models for Agile Quadrotor Flight

Xinhong Zhang, Runqing Wang, Yunfan Ren, Ding Yu+5

State Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China · Zhongguancun Academy, Beijing 100094, China · School of Computer Science and Technology, Tongji University+2

Robot Policy Learning World Models for Robotics World Models 3D Occupancy Prediction Quadrotor Control Vision-Based Robot Control

Open paper AskPDF arXiv

Agile quadrotor flight in cluttered scenes requires more than a reactive mapping from a depth image to a control command: the vehicle must remember which regions have been observed, infer nearby occupied space, and act under partial visibility and tight latency. In this paper, we present Mapping-Aware Dreamer (MAD), a geometry-aware world model for vision-based quadrotor flight. Instead of using raw-image reconstruction as the main self-supervised objective, MAD learns recurrent latent dynamics that reconstruct robocentric occupancy and visibility grid maps together with proprioceptive states. This design forces the latent state to encode local geometry, visibility history, and ego-motion in a form that is directly relevant to collision avoidance. MAD is trained in DiffAero using a GPU-parallel map-construction module that provides high-throughput supervision for occupancy and visibility. The learned representation is used in three policy-learning modes: imagination-based MAD-Dreamer and feature-extractor variants based on PPO and SHAC. Across visual navigation and racing tasks, MAD-based agents achieve higher success rates, faster flight, and better cross-task transfer than corresponding vision-only baselines. The model also produces interpretable map predictions and accurate ego-motion estimates from depth observations. We further deploy the learned policy on a physical quadrotor with an Intel RealSense D435i and demonstrate safe indoor and outdoor flight under limited sensing, reaching 9.66 m/s in simulation and 5.05 m/s in real-world forest experiments. These results show that mapping-aware world models provide a practical middle ground between modular aerial navigation and end-to-end learning.