World Models for Autonomous Driving

Overview

World models learn to predict future driving scenes and agent behavior, often conditioned on the vehicle’s possible actions, so a driving system can anticipate outcomes before acting. Approaches range from video-generation models to compact latent or BEV representations, occupancy and 3D geometry forecasts, and vector-based traffic simulators; planners use these predictions directly or train through simulated rollouts. Work is evaluated on datasets and benchmarks such as nuScenes, NAVSIM, Bench2Drive, and CARLA, with increasing attention to closed-loop safety and interaction rather than prediction quality alone. A central challenge is making long-horizon predictions both visually or geometrically realistic and faithful to actions, while keeping rollouts stable, safe, and efficient.

Latest papers

CardsList

170 papers21 papers in the last four weeks, down 25% on the four weeks before. 0.2% of all new papers.

  1. SWIFT: Slow World, Instant Freshness, and Timely Action for Asynchronous Autonomous Driving

    Nov 18, 2026Jiazhe Wang, Tianyi Yang, Ruihang Li +5Autonomous Driving PlanningWorld Models for Autonomous Driving

  2. Learning Agent-Centric Vision-Language-Action World Models for Autonomous Driving

    Nov 17, 2026Guoqing Wang, Pin Tang, Xiangxuan Ren +1World Models for Autonomous DrivingWorld Models for VLAs

  3. Making the World React Realistically: A Closed-Loop Interactive World Model for Autonomous Driving

    Nov 17, 2026Jia Hu, Yang Chang, Panqu Wang +4World Models for Autonomous DrivingInteractive World Models

  4. Beyond the Camera: A Tri-Modal Driving World Model for Future RGB, Thermal, and LiDAR Forecasting in Adverse Weather

    Nov 17, 2026Faysal Mehrab Chowdhury, Md. Mahfuzur Rahman, Ahmed Rafi Hasan +3World Models for Autonomous DrivingVideo World Models

  5. AR-WAM: efficient streaming world-action modeling for autonomous driving

    Nov 16, 2026Baiting Luo, Shatadal Mishra, Ahmadreza Moradipari +1World Models for Autonomous DrivingEnd-to-End Autonomous Driving

  6. WTR-Drive: World-Task Decomposition with Structured Risk Supervision for End-to-End Driving

    Nov 16, 2026Haoyu Xing, Shaobing Ding, Guanglei Zhao +3End-to-End Autonomous DrivingAutonomous Driving Planning

  7. Value and Policy Calibration from Execution Feedback for Socially Compatible Driving

    Nov 15, 2026Sen Shen, Ning Liu, Xiangrui Kong +2Autonomous DrivingAutonomous Driving Planning

  8. PanDWM: Panoramic 4D Driving World Models

    Nov 15, 2026Jiajun Sun, Zeyu Zhang, Zeyu Ren +3World Models for Autonomous Driving4D Scene Generation

  9. ActWorld-JEPA: Action-Conditioned JEPA World Models Meet Latent Planning for End-to-End Autonomous Driving

    Nov 12, 2026Ziying Song, Shengkai Zhang, Caiyan Jia +10World Models for Autonomous DrivingAutonomous Driving Planning

  10. LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting

    Nov 9, 2026Zhimin Liao, Ping Wei, Kai Mao +1World Models for Autonomous Driving3D Occupancy Prediction

  11. Breaking survivorship bias in driving world models

    Nov 8, 2026Zihao Sheng, Zilin Huang, Yuhao Luo +5World Model LearningWorld Models for Autonomous Driving

  12. Latent-WAM++: Shaping World and Action Representations for End-to-End Autonomous Driving

    Nov 7, 2026Yichen Zhang, Yupeng Zheng, Linbo Wang +10End-to-End Autonomous DrivingWorld Models for Autonomous Driving

  13. JEPAhead: From Inpainting to Causal Forecasting for Urban Driving

    Nov 5, 2026Daniel G. P. Petrini, Davi Pincinato, Fabio Grassiotto +4Video PredictionSpatiotemporal Forecasting

  14. FreeOcc: Decoupling Ego-Motion for Efficient 4D Occupancy Forecasting via Continuous Flow Matching

    Nov 5, 2026Zeping Zhang, Zhuoya Zhao, Samy Metari +1World Models for Autonomous Driving3D Occupancy Prediction

  15. WAM-Sim0.1: World Anchored Memory Meets Online RL for Consistent and Realistic Driving Simulation

    Nov 5, 2026Jiahao Cui, Jiaye Li, Wenhao Zhang +7Memory-Augmented World ModelsWorld Models for Autonomous Driving

  16. GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving

    Nov 5, 2026Xiyan Su, Finn Rasmus Schäfer, Hojun Lim +3World Models for Autonomous DrivingInference-Time Guidance

  17. Jev with Foresight: Safe Language-Guided Autonomous Driving via a Language World Model

    Nov 3, 2026Yansong Qu, Jiancong Chen, Samuel LabiAutonomous DrivingAutonomous Driving Planning

  18. Trajectory Evaluation via Rollout-Free World Model for End-to-end Autonomous Driving

    Nov 3, 2026Junzhe Wang, Jinqing Zhang, Qingjie LiuAutonomous Driving PlanningEnd-to-End Autonomous Driving

  19. INTERACT: Learning Interaction-Aware World Dynamics from Driving Videos for Autonomous Driving

    Nov 2, 2026Zhan Chen, Songyan Zhang, Wenhui Huang +5World Models for Autonomous DrivingAutonomous Driving Planning

  20. World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning

    Oct 16, 2026Xiangteng Zhang, Haoyuan Liu, Yang Guan +7Action-Conditioned World ModelsWorld Models for Autonomous Driving