World Models for Autonomous Driving

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 91

All topics

IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving
May 29, 2026Chenghao Zhang, Timin Li, Dongmei LiWorld Model Learning End-to-End Autonomous Driving
Department of Electronic Engineering, Tsinghua University
DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
May 27, 2026Chen Shi, Jinrui Xu, Shaoshuai Shi +3Video Diffusion Models End-to-End Autonomous Driving
The Chinese University of Hong Kong, Shenzhen · Voyager Research, Didi Chuxing
SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation
May 23, 2026Ruoyu Wang, Jingke Wang, Yukai Ma +5World Model Learning End-to-End Autonomous Driving
Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China · Labs, Huawei
LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model
May 21, 2026Xiaodong Mei, Diankun Zhang, Hongwei Xie +3VLMs for Autonomous Driving World Models
The Hong Kong University of Science and Technology · Xiaomi EV
Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving
May 20, 2026Yang Wu, Qiang Meng, Zhaojiang Liu +3End-to-End Autonomous Driving World Models
SJTU · FDU · NJU
Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving
May 19, 2026Zhengqi Sun, Yiwen Sun, Boxuan Liu +3Language Model-Based Control World Models
Department of Information Management, Peking University, Beijing 100871, China · School of Intelligence Science and Technology, Peking University, Beijing 100871, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing 100080, China +3
Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving
May 18, 2026Lijun Zhou, Hongcheng Luo, Zhenxin Zhu +34Dynamic Scene Reconstruction World Models for Autonomous Driving
Xiaomi Auto World Model Team
GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning
May 17, 2026Cheng Chen, Hao Huang, Saurabh Bagchi3D Gaussian Splatting World Model Learning
Purdue University · New York University Abu Dhabi
EponaV2: Driving World Model with Comprehensive Future Reasoning
May 14, 2026Jiawei Xu, Zhizhou Zhong, Zhijian Shu +8World Model Learning World Model-Based Planning
1PCA Lab, VCIP, College of Computer Science, Nankai University · 2Horizon Robotics · 3HKUST +4
HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation
May 12, 2026Conglang Zhang, Yifan Zhan, Qingjie Wang +10World Model Learning Long-Horizon Video Generation
Wuhan University · The University of Tokyo · Horizon Robotics +3
The DAWN of World-Action Interactive Models
May 12, 2026Hongbo Lu, Liang Yao, Chenghao He +6Diffusion-Based Planning World Model-Based Planning
1COWARobot Co. Ltd · 2Shanghai Jiao Tong University · 3Hohai University
Is Your Driving World Model an All-Around Player?
May 11, 2026Lingdong Kong, Ao Liang, Tianyi Yan +20Autonomous Driving Benchmarks Physical Consistency in Video Generation
NUS · UM · USTC +10
DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
May 11, 2026Lingjun Zhang, Changjie Wu, Linzhe Shi +6World Model Learning End-to-End Autonomous Driving
Tsinghua University · Amap, Alibaba Group · Nanyang Technological University
CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving
May 11, 2026Minqing Huang, Yujiao Xiang, Zihan Liang +7VLMs for Autonomous Driving Diffusion-Based Planning
Afari Intelligent Drive · University of Electronic Science and Technology of China · Shanghai Jiao Tong University +2
Network-Efficient World Model Token Streaming
May 11, 2026Shatadal Mishra, Ahmadreza Moradipari, Nejib AmmarIntelligent Transportation Systems World Models
InfoTech Labs, Toyota Motor North America R&D, Mountain View, CA, USA
DriveFuture: Future-Aware Latent World Models for Autonomous Driving
May 10, 2026Yufeng Hong, Xiaotian Zhou, Yingyan Li +6World Models for Autonomous Driving Latent World Models
Beijing Institute of Technology · Institute of Automation, Chinese Academy of Sciences · Beihang University +5
GEM: Generating LiDAR World Model via Deformable Mamba
May 8, 2026Yang Wu, Zhaojiang Liu, Qiang Meng +5Mamba World Model Learning
SJTU · FDU · NTU +1
See Tomorrow, Act Today: Foresight-Driven Autonomous Driving
May 8, 2026Bozhou Zhang, Nan Song, Yuang Wang +3End-to-End Autonomous Driving World Models
School of Data Science, Fudan University · Shanghai Innovation Institute · Imperial College London +1
Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout
May 6, 2026Haozhuang Chi, Daosheng Qiu, Hao Su +4World Model Learning Causal World Models
Nanyang Technological University, Singapore · Hubei University, Wuhan, China · Osaka University, Osaka, Japan
HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation
Apr 30, 2026Xin Zhou, Dingkang Liang, Xiwu Chen +4World Model Learning 3D Scene Generation
Huazhong University of Science and Technology · Mach Drive · University of Hong Kong
ProDrive: Proactive Planning for Autonomous Driving via Ego-Environment Co-Evolution
Apr 28, 2026Chuyao Fu, Shengzhe Gan, Zhuoli Ouyang +5End-to-End Autonomous Driving World Models
1Southern University of Science and Technology · 2Hong Kong University of Science and Technology
Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
Apr 20, 2026Jinghui Lu, Jiayi Guan, Zhijian Huang +47Trajectory Generation VLMs for Autonomous Driving
Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception
Apr 19, 2026Siyuan Meng, Chengbo AiWorld Model Learning World Models
Department of Civil and Environmental Engineering, University of Massachusetts Amherst
DriveVA: Video Action Models are Zero-Shot Drivers
Apr 5, 2026Mengmeng Liu, Diankun Zhang, Jiuming Liu +7Action-Conditioned Video Generation World Models
University of Twente, The Netherlands · Xiaomi EV, China · University of Cambridge, United Kingdom +1
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving
Apr 3, 2026Zihao Sheng, Xin Ye, Jingru Luo +2World Model Learning VLMs for Autonomous Driving
Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI) · University of Wisconsin–Madison
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Jan 7, 2026Zhexiao Xiong, Xin Ye, Burhan Yaman +5Vision-Language Models World Model Learning
Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2
LaGen: Towards Autoregressive LiDAR Scene Generation
Nov 26, 2025Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang +73D Scene Generation Autoregressive Generation
University of Science and Technology of China, Hefei, Anhui 230026, China · Shanghai Innovation Institute, Shanghai 200231, China · Fudan University, Shanghai 200433, China +1
OmniNWM: Omniscient Driving Navigation World Models
Oct 21, 2025Bohan Li, Zhuang Ma, Dalong Du +10Action-Conditioned Video Generation World Model Learning
Shanghai Jiao Tong University · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · PhiGent +3
Auto-Regressive Models Need Structural Registers: Semantic Foresight Improves Coherent Driving Video Continuation
Dec 4, 2024Ruibo Ming, Jingwei Wu, Zhewei Huang +4Video Prediction Autoregressive Video Generation
Tsinghua University · Megvii Technology · University of the Chinese Academy of Sciences +2
Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
Sep 25, 2024Alexander Popov, Alperen Degirmenci, David Wehr +9End-to-End Autonomous Driving Imitation Learning
NVIDIA

May 29, 2026, cs.ROJ/K move · Enter open · S save

IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving

Chenghao Zhang, Timin Li, Dongmei Li

Department of Electronic Engineering, Tsinghua University

World Model Learning End-to-End Autonomous Driving World Models for Autonomous Driving Autonomous Driving Planning

Open paper AskPDF arXiv

End-to-end autonomous driving has emerged as a compelling paradigm for learning planning directly from sensor observations, while recent world-model-based approaches further enrich this paradigm by enabling explicit reasoning about how the scene may evolve in the future. Yet future prediction alone does not guarantee better planning unless the predicted evolution can be converted into planning-relevant trajectory updates. Many current methods still forecast future scene states without explicitly decoding the motion implications hidden in state transitions. As a result, future reasoning often remains descriptively useful but only weakly coupled to executable motion generation. To address this limitation, we propose \mathbf{IDOL}, an inverse-dynamics-guided future prediction framework for world-model-based end-to-end planning in latent BEV space, where inverse dynamics serves as the key bridge between future prediction and trajectory optimization. IDOL first predicts multiple future latent scene states with a BEV world model, then applies an inverse dynamics model to adjacent latent futures to decode transition-aware trajectory features and recover planning-relevant motion deltas that explain how the latent world evolves over time. These inverse-dynamics-derived signals are used to optimize the planned trajectory, turning future forecasting from passive scene anticipation into actionable planning guidance. A lightweight closed-loop refinement module further improves long-horizon consistency by reusing the optimized trajectory for another round of future-aware reasoning. By introducing inverse dynamics into latent future reasoning, IDOL tightens the coupling between world modeling and planning. Extensive experiments on the NAVSIM v1 and NAVSIM v2 benchmarks show that IDOL achieves state-of-the-art performance among comparable methods.

Momentum

Related topics

Latest papers 91