Autonomous Driving Planning

Follow this topic Ask about this topic

Autonomous driving planning turns sensor and map information, navigation goals, and predictions about other road users into decisions and vehicle trajectories. Approaches range from search and model predictive control (MPC), reachability and barrier methods, and game-theoretic planning to imitation learning, reinforcement learning, diffusion planners, and vision-language-action or world-action models. Systems increasingly combine prediction with planning to handle interactions, uncertainty, occlusion, and rare hazards, and are evaluated on benchmarks such as nuPlan, nuScenes, NAVSIM, Bench2Drive, and Waymo. A central challenge is that good open-loop trajectory scores do not guarantee safe, efficient closed-loop driving, while stronger safety checks and richer models must still meet real-time constraints and generalize to unfamiliar scenarios.

Latest papers

385 papers50 papers in the last four weeks, up 35% on the four weeks before. 0.3% of all new papers.

Feb 21, 2026

SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World

The end-to-end (E2E) paradigm, which maps sensor inputs directly to driving decisions, has recently attracted significant attention due to its unified modeling capability and scalability. However, ensuring safety in this unified framework remains one of the most critical challenges. In this work, we propose SafeDrive, an E2E planning framework designed to perform explicit and interpretable safety reasoning through a trajectory-conditioned Sparse World Model. SafeDrive comprises two complementary networks: the Sparse World Network (SWNet) and the Fine-grained Reasoning Network (FRNet). SWNet constructs trajectory-conditioned sparse worlds that simulate the future behaviors of critical dynamic agents and road entities, providing interaction-centric representations for downstream reasoning. FRNet then evaluates agent-specific collision risks and temporal adherence to drivable regions, enabling precise identification of safety-critical events across future timesteps. SafeDrive achieves state-of-the-art performance on both open-loop and closed-loop benchmarks. On NAVSIM, it records a PDMS of 91.6 and an EPDMS of 87.5, with only 61 collisions out of 12,146 scenarios (0.5%). On Bench2Drive, SafeDrive attains a 66.8% driving score.

Jungho Kim, Jiyong Oh, Seunghoon Yu +3

Seoul National University, Republic of Korea

Autonomous Driving Autonomous Driving Planning End-to-End Autonomous Driving Safe Motion Planning World Models for Autonomous Driving

Feb 20, 2026

Interacting safely with cyclists using Hamilton-Jacobi reachability and reinforcement learning

In this paper, we present a framework for enabling autonomous vehicles to interact with cyclists in a manner that balances safety and optimality. The approach integrates Hamilton-Jacobi reachability analysis with deep Q-learning to jointly address safety guarantees and time-efficient navigation. A value function is computed as the solution to a time-dependent Hamilton-Jacobi-Bellman inequality, providing a quantitative measure of safety for each system state. This safety metric is incorporated as a structured reward signal within a reinforcement learning framework. The method further models the cyclist's latent response to the vehicle, allowing disturbance inputs to reflect human comfort and behavioral adaptation. The proposed framework is evaluated through simulation and comparison with human driving behavior and an existing state-of-the-art method.

Aarati Andrea Noronha, Jean Oh

Autonomous Driving Planning Safe RL RL for Autonomous Driving

Feb 19, 2026

Hybrid System Planning using a Mixed-Integer ADMM Heuristic and Hybrid Zonotopes

Embedded optimization-based planning for hybrid systems is challenging due to the use of mixed-integer programming, which is computationally intensive and often sensitive to the specific numerical formulation. To address that challenge, this article proposes a framework for motion planning of hybrid systems that pairs hybrid zonotopes - an advanced set representation - with a new alternating direction method of multipliers (ADMM) mixed-integer programming heuristic. A general treatment of piecewise affine (PWA) system reachability analysis using hybrid zonotopes is presented and extended to formulate optimal planning problems. Sets produced using the proposed identities have lower memory complexity and tighter convex relaxations than equivalent sets produced from preexisting techniques. The proposed ADMM heuristic makes efficient use of the hybrid zonotope structure. For planning problems formulated as hybrid zonotopes, the proposed heuristic achieves improved convergence rates as compared to state-of-the-art mixed-integer programming heuristics. The proposed methods for hybrid system planning on embedded hardware are experimentally applied in a combined behavior and motion planning scenario for autonomous driving.

Joshua A. Robbins, Andrew F. Thompson, Jonah J. Glunt +1

Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802 USA

Motion Planning Alternating Direction Method of Multipliers Autonomous Driving Planning

Feb 19, 2026

Dodging the Moose: Experimental Insights in Real-Life Automated Collision Avoidance

The sudden appearance of a static obstacle on the road, i.e. the moose test, is a well-known emergency scenario in collision avoidance for automated driving. Model Predictive Control (MPC) has long been employed for planning and control of automated vehicles in the state of the art. However, real-time implementation of automated collision avoidance in emergency scenarios such as the moose test remains unaddressed due to the high computational demand of MPC for evasive action in such hazardous scenarios. This paper offers new insights into real-time collision avoidance via the experimental imple- mentation of MPC for motion planning after a sudden and unexpected appearance of a static obstacle. As the state-of-the-art nonlinear MPC shows limited capability to provide an acceptable solution in real-time, we propose a human-like feed-forward planner to assist when the MPC optimization problem is either infeasible or unable to find a suitable solution due to the poor quality of its initial guess. We introduce the concept of maximum steering maneuver to design the feed-forward planner and mimic a human-like reaction after detecting the static obstacle on the road. Real-life experiments are conducted across various speeds and level of emergency using FPEV2-Kanon electric vehicle. Moreover, we demonstrate the effectiveness of our planning strategy via comparison with the state-of- the-art MPC motion planner.

Leila Gharavi, Simone Baldi, Yuki Hosomi +4

Delft Center for Systems and Control, Delft University of Technology, Delft, the Netherlands. · School of Mathematics, Southeast University, Nanjing, China. · Department of Advanced Energy, Graduate School of Frontier Sciences, The University of Tokyo, Chiba 277-8561, Japan.

Collision Avoidance Autonomous Driving Planning Model Predictive Control

Feb 17, 2026

The Impact of Class Uncertainty Propagation in Perception-Based Motion Planning

Autonomous vehicles (AVs) are being increasingly deployed in urban environments. In order to operate safely and reliably, AVs need to account for the inherent uncertainty associated with perceiving the world through sensor data and incorporate that into their decision-making process. Uncertainty-aware planners have recently been developed to account for upstream perception and prediction uncertainty. However, such planners may be sensitive to prediction uncertainty miscalibration, the magnitude of which has not yet been characterized. Towards this end, we perform a detailed analysis on the impact that perceptual uncertainty propagation and calibration has on perception-based motion planning. We do so by comparing two novel prediction-planning pipelines with varying levels of uncertainty propagation on the recently-released nuPlan planning benchmark. We study the impact of upstream uncertainty calibration using closed-loop evaluation on the nuPlan challenge scenarios. We find that the method incorporating upstream uncertainty propagation demonstrates superior generalization to complex closed-loop scenarios.

Jibran Iqbal Shah, Andrei Ivanovic, Kelly Zhu +4

Department of Computer Science, University of Toronto. · Toyota Research Institute.

Autonomous Driving Planning Motion Planning Uncertainty Propagation Uncertainty Calibration

Feb 16, 2026

Multimodal Belief-Space Covariance Steering with Active Probing and Influence for Interactive Driving

Autonomous driving in complex traffic requires reasoning under uncertainty. Common approaches rely on prediction-based planning or risk-aware control, but these are typically treated in isolation, limiting their ability to capture the coupled nature of action and inference in interactive settings. This gap becomes especially critical in uncertain scenarios, where simply reacting to predictions can lead to unsafe maneuvers or overly conservative behavior. Our central insight is that safe interaction requires not only estimating human behavior but also shaping it when ambiguity poses risks. To this end, we introduce a hierarchical belief model that structures human behavior across coarse discrete intents and fine motion modes, updated via Bayesian inference for interpretable multi-resolution reasoning. On top of this, we develop an active probing strategy that identifies when multimodal ambiguity in human predictions may compromise safety and plans disambiguating actions that both reveal intent and gently steer human decisions toward safer outcomes. Finally, a runtime risk-evaluation layer based on Conditional Value-at-Risk (CVaR) ensures that all probing actions remain within human risk tolerance during influence. Our simulations in lane-merging and unsignaled intersection scenarios demonstrate that our approach achieves higher success rates and shorter completion times compared to existing methods. These results highlight the benefit of coupling belief inference, probing, and risk monitoring, yielding a principled and interpretable framework for planning under uncertainty.

Devodita Chakravarty, John Dolan, Yiwei Lyu

Department of Mechanical Engineering, Indian Institute of Technology Kharagpur, India · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA

Risk-Sensitive Planning Autonomous Driving Planning Decision-Making under Uncertainty Belief-Space Planning Active Perception

Feb 1, 2026

HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving

End-to-end autonomous driving models increasingly benefit from large vision-language models for semantic understanding, yet safe and reliable planning under long-tail conditions remains challenging, particularly in mixed-traffic environments involving heterogeneous road users and rare safety-critical interactions. This paper proposes HERMES, a holistic risk-aware end-to-end multimodal driving framework that explicitly incorporates long-tail semantic knowledge into trajectory planning. HERMES employs a foundation-model-assisted annotation pipeline to construct structured Long-Tail Scene Context and Long-Tail Planning Context, capturing hazard-centric scene information, maneuver intent, and risk-aware planning guidance. A Tri-Modal Driving Module then integrates multi-view visual observations, historical ego-motion, and long-tail semantic instructions through intent- and risk-aware conditioning for trajectory generation. Extensive experiments on a large-scale real-world long-tail driving benchmark demonstrate consistent improvements over representative recent baselines in overall planning performance and across diverse safety-critical scenarios. Ablation studies further validate the effectiveness and complementary roles of the major components within HERMES.

Weizhe Tang, Junwei You, Jiaxi Liu +7

Department of Civil and Environmental Engineering, University of Wisconsin–Madison, Madison, WI 53711 USA

Long-Tail Autonomous Driving VLMs for Autonomous Driving End-to-End Autonomous Driving Autonomous Driving Risk-Aware Path Planning Safe Motion Planning+1

Jan 29, 2026

Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving

End-to-end autonomous driving increasingly leverages self-supervised video pretraining to learn transferable planning representations. However, pretraining video world models for scene understanding has so far brought only limited improvements. This limitation is compounded by the inherent ambiguity of driving: each scene typically provides only a single human trajectory, making it difficult to learn multimodal behaviors. In this work, we propose Drive-JEPA, a framework that integrates Video Joint-Embedding Predictive Architecture (V-JEPA) with multimodal trajectory distillation for end-to-end driving. First, we adapt V-JEPA for end-to-end driving, pretraining a ViT encoder on large-scale driving videos to produce predictive representations aligned with trajectory planning. Second, we introduce a proposal-centric planner that distills diverse simulator-generated trajectories alongside human trajectories, with a momentum-aware selection mechanism to promote stable and safe behavior. When evaluated on NAVSIM, the V-JEPA representation combined with a simple transformer-based decoder outperforms prior methods by 3 PDMS in the perception-free setting. The complete Drive-JEPA framework achieves 93.3 PDMS on v1 and 87.8 EPDMS on v2, setting a new state-of-the-art.

Linhan Wang, Zichong Yang, Chen Bai +6

Virginia Tech · Purdue University · XPENG Motors +1

End-to-End Autonomous Driving NavSim Joint-Embedding Predictive Architecture Video Representation Learning Autonomous Driving Planning

Jan 7, 2026

UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.

Zhexiao Xiong, Xin Ye, Burhan Yaman +5

Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2

Vision-Language Models World Model Learning VLMs for Autonomous Driving Video Prediction World Models for Autonomous Driving Autonomous Driving Planning

Jan 5, 2026

AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving

Practical autonomous driving requires models that generalize by reasoning through spatial-temporal possibilities to exclude unsafe outcomes. While state-of-the-art (SOTA) methods use parallel planning architectures, they fail to explicitly couple speed decisions with agent behavior along the driving path, leading to suboptimal coordination. To address this, we propose a cascaded framework that transforms longitudinal planning from an independent prediction task into a path-conditioned reasoning process. On the model side, we introduce an anchor-based regression design that conditions longitudinal prediction on the lateral drive path, and reformulate longitudinal planning as 1D displacement prediction along the path. This reduces geometric uncertainty and sharpens the model's focus on interaction-driven dynamics. On the data side, we introduce a planning-oriented data augmentation strategy that simulates rare safety-critical events by programmatically inserting agents and relabeling longitudinal targets to enforce collision avoidance. Evaluated on the challenging Bench2Drive benchmark, our method achieves SOTA performance with a driving score of 89.07 and a success rate of 73.18%, demonstrating significantly improved coordination and safety. Further evaluation on Fail2Drive confirms strong generalization to rare edge cases where parallel formulations typically fail. Project page:https://yanhaowu.github.io/AlignDrive/.

Yanhao Wu, Haoyang Zhang, Fei He +6

School of Software Engineering, XJTU · Horizon Robotics · University of Chinese Academy of Sciences

Collision Avoidance End-to-End Autonomous Driving Autonomous Driving Planning

Dec 11, 2025

SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving

End-to-end autonomous driving methods built on vision language models (VLMs) have undergone rapid development driven by their universal visual understanding and strong reasoning capabilities obtained from the large-scale pretraining. However, we find that current VLMs struggle to understand fine-grained 3D spatial relationships which is a fundamental requirement for systems interacting with the physical world. To address this issue, we propose SpaceDrive, a spatial-aware VLM-based driving framework that treats spatial information as explicit positional encodings (PEs) instead of textual digit tokens, enabling joint reasoning over semantic and spatial representations. SpaceDrive employs a universal positional encoder to all 3D coordinates derived from multi-view depth estimation, historical ego-states, and text prompts. These 3D PEs are first superimposed to augment the corresponding 2D visual tokens. Meanwhile, they serve as a task-agnostic coordinate representation, replacing the digit-wise numerical tokens as both inputs and outputs for the VLM. This mechanism enables the model to better index specific visual semantics in spatial reasoning and directly regress trajectory coordinates rather than generating digit-by-digit, thereby enhancing planning accuracy. Extensive experiments validate that SpaceDrive achieves state-of-the-art open-loop performance on the nuScenes dataset and the second-best Driving Score of 78.02 on the Bench2Drive closed-loop benchmark over existing VLM-based methods. Code is available at: https://github.com/zhenghao2519/SpaceDrive.

Peizheng Li, Zhenghao Zhang, David Holtz +6

Mercedes-Benz AG · TU Munich · Karlsruhe Institute of Technology +4

Vision-Language Models VLMs for Autonomous Driving End-to-End Autonomous Driving 3D Spatial Reasoning Positional Encoding Autonomous Driving Planning

Dec 11, 2025

Closing the Navigation Compliance Gap in End-to-end Autonomous Driving

Trajectory-scoring planners achieve high navigation compliance when following the expert's original command, yet they struggle at intersections when presented with alternative commands; over 30 percent of such commands are ignored. We attribute this navigation compliance gap to two root causes: (1) existing metrics like Ego Progress do not explicitly measure navigation adherence, diluting the gap between on-route and off-route trajectories; and (2) current datasets pair each scenario with a single command, preventing models from learning command-dependent behavior. We address the metric gap by introducing the binary Navigation Compliance metric (NAVI) and the derived Controllability Measure (CM), and the data gap with the NavControl dataset, 14,918 intersection scenarios augmented with all feasible alternative commands and routing annotations, yielding over 34,000 direction samples. Building on these, we propose NaviHydra, a trajectory-scoring planner incorporating NAVI distillation and Bird's Eye View (BEV)-based trajectory gathering for context-position-aware trajectory feature extraction. NaviHydra achieves 92.7 PDM score on NAVSIM navtest split and 77.5 CM on NavControl test split. Training with NavControl improves controllability across diverse architectures, confirming it as a broadly effective augmentation for navigation compliance. Our project page is https://hanfeng-wu.github.io/closing/.

Hanfeng Wu, Marlon Steiner, Michael Schmidt +2

Karlsruhe Institute of Technology Karlsruhe, Germany · BMW Group Munich, Germany

Autonomous Driving Autonomous Driving Planning End-to-End Autonomous Driving Autonomous Driving Datasets

Dec 3, 2025

Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response

Autonomous driving planning systems perform nearly perfectly in routine scenarios using lightweight, rule-based methods but still struggle in dense urban traffic, where lane changes and merges require anticipating and influencing other agents. Modern motion predictors offer highly accurate forecasts, yet their integration into planning is mostly rudimental: discarding unsafe plans. Similarly, end-to-end models offer a one-way integration that avoids the challenges of joint prediction and planning modeling under uncertainty. In contrast, game-theoretic formulations offer a principled alternative but have seen limited adoption in autonomous driving. We present Bayesian Iterative Best Response (BIBeR), a framework that unifies motion prediction and game-theoretic planning into a single interaction-aware process. BIBeR is the first to integrate a state-of-the-art predictor into an Iterative Best Response (IBR) loop, repeatedly refining the strategies of the ego vehicle and surrounding agents. This repeated best-response process approximates a Nash equilibrium, enabling bidirectional adaptation where the ego both reacts to and shapes the behavior of others. In addition, our proposed Bayesian confidence estimation quantifies prediction reliability and modulates update strength, more conservative under low confidence and more decisive under high confidence. BIBeR is compatible with modern predictors and planners, combining the transparency of structured planning with the flexibility of learned models. Experiments show that BIBeR achieves an 11% improvement over state-of-the-art planners on highly interactive interPlan lane-change scenarios, while also outperforming existing approaches on standard nuPlan benchmarks.

Aron Distelzweig, Yiwei Wang, Faris Janjoš +5

University of Freiburg, Germany · Bosch Center for Artificial Intelligence, Germany · Technical University of Munich, Germany +2

Autonomous Driving Planning Multi-Agent Planning

Nov 14, 2025

GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

Vision-language models have recently emerged as promising planners for autonomous driving, where success hinges on reasoning over spatial structure and dynamic interactions from multimodal input. However, existing models are typically trained without supervision that explicitly encodes relational dependencies, requiring them to infer interactions between traffic entities implicitly from raw sensor data. In this work, we introduce GraphPilot, a model-agnostic approach that conditions language-based driving models on structured relational context in the form of traffic scene graphs. We serialize scene graphs at different abstraction levels and formats and inject them through structured prompts without modifying the underlying architecture or training objective. Extensive closed-loop evaluations on LangAuto and Bench2Drive show persistent improvements across LMDrive, BEVDriver, and SimLingo. Controlled ablations and hidden-state probing further indicate that relational supervision contributes substantially to these improvements while increasing the decodability of scene relations. Open-loop SpaceDrive results on nuScenes further demonstrate transfer to real-world data. These results indicate that diverse architectures can effectively internalize and ground relational priors through scene graph-conditioned training, even without requiring scene graph input at test-time. Code, fine-tuned models, and our scene graph dataset are publicly available at https://github.com/iis-esslingen/GraphPilot.

Fabian Schmidt, Markus Enzweiler, Abhinav Valada

Esslingen University of Applied Sciences · University of Freiburg

Vision-Language Models VLMs for Autonomous Driving Language Model-Based Control Autonomous Driving Autonomous Driving Planning Graph Reasoning

Nov 10, 2025

HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Driving

Three-dimensional feature extraction and multi-task perception are fundamental components of modern autonomous driving systems. Although large image encoders, high-resolution inputs, and long temporal contexts can substantially improve representation quality and overall performance, jointly leveraging these strategies remains challenging due to prohibitive computational costs during both training and inference. Furthermore, different perception tasks often require distinct feature representations, making it difficult for a unified architecture to achieve end-to-end multi-task performance comparable to specialized single-task systems. To address these challenges, we propose HENet++, a unified framework for multi-task 3D perception and end-to-end autonomous driving that employs a hybrid image encoding strategy, using a large encoder for short-term frames and a lightweight encoder for long-term temporal context to strike a favorable balance between accuracy and efficiency. The framework jointly extracts dense background features and sparse foreground features, enabling task-specific representations that reduce cumulative errors and provide richer information for downstream prediction and planning modules. HENet++ is compatible with diverse 3D feature extraction pipelines and supports multi-modal inputs, including camera and radar data. Extensive experiments demonstrate state-of-the-art performance on the nuScenes multi-task 3D perception benchmark, achieving the lowest collision rate on the nuScenes planning benchmark and higher PDMS on the NAVSIM benchmark.

Zhongyu Xia, Zhiwei Lin, Yongtao Wang +1

Wangxuan Institute of Computer Technology, Peking University, Beijing, China. · University of California, Merced, USA.

Autonomous Driving Perception End-to-End Autonomous Driving 3D Scene Understanding Multi-Task Learning Autonomous Driving Planning

Oct 28, 2025

Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer

Human demonstrations are widely considered the cornerstone of end-to-end (E2E) autonomous driving despite human demonstration's scarcity for long-tail and safety-critical scenarios. Nonetheless, current E2E autonomous driving (AD) training paradigms continue to rely on human demonstrations. Imitation learning (IL) requires human demonstrations for training, whereas reinforcement learning (RL) has emerged as a promising alternative to reduce this dependency. However, most existing RL methods for E2E AD still rely implicitly on human demonstrations. A pure rewards-based RL method can overcome the need for human demonstrations, but general RL policy gradient methods suffer from the cold-start problem. In this paper, we propose ZTRS (Zero-human demonstration end-to-end autonomous driving with TRajectory Scorer) - a complete RL-based E2E planning paradigm trained solely on real-world images and rule-based rewards, entirely without human demonstration. Through our proposed Exhaustive Policy Optimization (EPO), a policy gradient variant tailored for enumerable trajectory actions and dense supervision, ZTRS enables the model to generalize better to long-tail driving scenarios. We demonstrate this generalization through our SOTA performance against IL approaches on both long-tail Navhard and closed-loop HUGSIM datasets. Project page: https://zhenxinli.net/ZTRS/.

Zhenxin Li, Nadine Chang, Wenhao Yao +9

Institute of Trustworthy Embodied AI, Fudan University · *Work done during an internship at NVIDIA. · NVIDIA +3

Long-Tail Autonomous Driving Reinforcement Learning End-to-End Autonomous Driving Autonomous Driving Autonomous Driving Planning RL for Autonomous Driving

Oct 6, 2025

More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models

Vision-Language Model (VLM) driving agents promise explainable end-to-end autonomy by first producing natural-language reasoning and then predicting trajectory planning. However, whether planning is causally driven by this reasoning remains a critical but unverified assumption. To investigate this, we build DriveMind, a large-scale driving Visual Question Answering corpus with plan-aligned Chain-of-Thought (CoT), automatically generated from nuPlan. Our data generation process converts sensors and annotations into structured inputs and, crucially, separates priors from to-be-reasoned signals, enabling clean information ablations. Using DriveMind, we train representative VLM agents with Supervised Fine-Tuning and Group Relative Policy Optimization and evaluate them with nuPlan's metrics. Our results, unfortunately, indicate a consistent causal disconnect in reasoning-planning: removing ego/navigation priors causes large drops in planning scores, whereas removing CoT produces only minor changes. Attention analysis further shows that planning primarily focuses on priors rather than the CoT. Based on this evidence, we propose the Reasoning-Planning Decoupling Hypothesis, positing that the training-yielded reasoning is an ancillary byproduct rather than a causal mediator. To enable efficient diagnosis, we introduce a novel, training-free probe that measures an agent's reliance on priors by evaluating its planning robustness against minor input perturbations. In summary, we provide the community with a new dataset and a diagnostic tool to evaluate the causal fidelity of future models.

Xurui Song, Shuo Huai, JingJing Jiang +2

S-Lab, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore

VLMs for Autonomous Driving Reasoning for VLA Models Autonomous Driving Planning Faithfulness in VLMs

Aug 23, 2025

Drive As You Like: Multi-Head Diffusion with Reinforcement Learning for Personalized Driving

Personalized autonomous driving requires understanding the driving needs implicit in user requests. This is challenging when driving demonstrations lack style annotations and behavioral adaptation risks degrading planning quality. We propose M-Diffusion Planner with a dual-decoupling design: separating intent understanding from trajectory planning, and general planning capability learning from strategy alignment. A large language model infers driving needs and selects a strategy, which is executed by the corresponding head of a shared diffusion planner. Following imitation pretraining, strategy-specific rewards specialize individual heads without style annotations. We innovatively adapt Group Relative Policy Optimization using surrogate policy-ratio estimation and constrained head-specific updates to mitigate degradation of pretrained planning capabilities. Open-loop and closed-loop evaluations on nuPlan demonstrate distinct strategy-aligned behaviors and competitive planning performance. Further experiments assess intent-to-strategy mapping and collision risk, and analyze the effects of reward design and optimization on strategy alignment and planning quality.

Ding Fan, Xuewen Luo, Hwa Hui Tew +3

Monash University, Malaysia · University of Utah, USA · Monash University, Australia

Autonomous Driving Planning Autonomous Driving Diffusion-Based Planning RL for Autonomous Driving

Aug 15, 2025

OEL-AD: An Online Evolutive Learning Framework for Cross-Region Adaptive Autonomous Driving

Autonomous driving models have achieved great progress in recent years, yet most existing approaches remain static, following a train-then-deploy paradigm that lacks the ability to adjust internal parameters when encountering new environments. This limitation is particularly critical in cross-region scenarios, which are common in real-world deployments in autonomous driving where differences in road layouts, traffic patterns, and environmental conditions introduce distribution shifts. In this paper, we argue for a paradigm shift towards Online Evolutive Learning (OEL), a learning paradigm designed to enable adaptive, online model optimization through the interaction and coordination of intelligent components in dynamic environments. We introduce OEL-AD, a novel framework that instantiates this paradigm for autonomous driving. Specifically, OEL-AD leverages trajectory uncertainty from the planning module to determine whether adaptive updates are necessary. When triggered, the framework uses discrepancies between predicted agent behaviors and subsequent observations as a self-supervised learning signal to refine perception and prediction modules, thereby indirectly improving planning accuracy and safety. Experiments on the nuScenes dataset with a cross-region split between Singapore and Boston demonstrate that OEL-AD consistently improves perception and prediction performance. This enhancement in perception and prediction consequently results in improved planning capabilities, thereby confirming the promise of online evolutive learning as a fundamental approach for building adaptive and robust autonomous driving systems.

Jiayue Jin, Lang Qian, Jingyu Zhang +2

College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China

Autonomous Driving Adaptive Learning Autonomous Driving Perception Domain Adaptation Autonomous Driving Planning

May 27, 2025

CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Driving

While end-to-end autonomous driving has advanced significantly, prevailing methods remain fundamentally misaligned with human cognitive principles in both perception and planning. In this paper, we propose CogAD, a novel end-to-end autonomous driving model that emulates the hierarchical cognition mechanisms of human drivers. CogAD implements dual hierarchical mechanisms: global-to-local context processing for human-like perception and intent-conditioned multi-mode trajectory generation for cognitively-inspired planning. The proposed method demonstrates three principal advantages: comprehensive environmental understanding through hierarchical perception, robust planning exploration enabled by multi-level planning, and diverse yet reasonable multi-modal trajectory generation facilitated by dual-level uncertainty modeling. Extensive experiments on nuScenes and Bench2Drive demonstrate that CogAD achieves state-of-the-art performance in end-to-end planning, exhibiting particular superiority in long-tail scenarios and robust generalization to complex real-world driving conditions.

Zhennan Wang, Jianing Teng, Canqun Xiang +4

HAOMO.AI Technology Co., Ltd

End-to-End Autonomous Driving Autonomous Driving Perception Autonomous Driving Planning

May 21, 2025

Generative AI for Autonomous Driving: A Review

Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle motion planning. By examining multiple generative approaches ranging from Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) to Generative Transformers (GTs) and Diffusion Models (DMs), we highlight and compare their capabilities and limitations for AD-specific applications. Additionally, we discuss hybrid methods integrating conventional techniques with generative approaches, and emphasize their improved adaptability and robustness. We also identify relevant datasets and outline open research questions to guide future developments in GenAI. Finally, we discuss five core challenges: safety, interpretability, realtime capabilities, training and benchmarking. We also present recommendations for image generation, dynamic scenario generation, and planning.

Katharina Winter, Abhishek Vivekanandan, Rupert Polley +17

Munich University of Applied Sciences, Intelligent Vehicles Lab (IVL), 80335 Munich, Germany · FZI Research Center for Information Technology, 76131 Karlsruhe, Germany · Continental Automotive Technologies GmbH, Guericke Str.7, 60488 Frankfurt a. M., Germany +6

Autonomous Driving Autonomous Driving Planning Autonomous Driving Scenario Generation Trajectory Prediction

Mar 31, 2025

A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective

Reinforcement learning (RL), with its ability to explore and optimize policies in complex, dynamic decision-making tasks, has emerged as a promising approach to addressing motion planning (MoP) challenges in autonomous driving (AD). Despite rapid advancements in RL and AD, a systematic description and interpretation of the RL design process tailored to diverse driving tasks remains underdeveloped. This survey provides a comprehensive review of RL-based MoP for AD, focusing on lessons from task-specific perspectives. We first outline the fundamentals of RL methodologies, and then survey their applications in MoP, analyzing scenario-specific features and task requirements to shed light on their influence on RL design choices. Building on this analysis, we summarize key design experiences, extract insights from various driving task applications, and provide guidance for future implementations. Additionally, we examine the frontier challenges in RL-based MoP, review recent efforts to addresse these challenges, and propose strategies for overcoming unresolved issues.

Zhuoren Li, Guizhe Jin, Ran Yu +8

College of Automotive and Energy Engineering, Tongji University, Shanghai 201804, China

Autonomous Driving Planning RL for Autonomous Driving

Feb 17, 2025

Does Knowledge About Perceptual Uncertainty Help an Agent in Automated Driving?

Agents in real-world scenarios like automated driving deal with uncertainty in their environment, in particular due to perceptual uncertainty. Although, reinforcement learning is dedicated to autonomous decision-making under uncertainty these algorithms are typically not informed about the uncertainty currently contained in their environment. On the other hand, uncertainty estimation for perception itself is typically directly evaluated in the perception domain, e.g., in terms of false positive detection rates or calibration errors based on camera images. Its use for deciding on goal-oriented actions remains largely unstudied. In this paper, we investigate how an agent's behavior is influenced by an uncertain perception and how this behavior changes if information about this uncertainty is available. Therefore, we consider a proxy task, where the agent is rewarded for driving a route as fast as possible without colliding with other road users. For controlled experiments, we introduce uncertainty in the observation space by perturbing the perception of the given agent while informing the latter. Our experiments show that an unreliable observation space modeled by a perturbed perception leads to a defensive driving behavior of the agent. Furthermore, when adding the information about the current uncertainty directly to the observation space, the agent adapts to the specific situation and in general accomplishes its task faster while, at the same time, accounting for risks.

Natalie Grabowsky, Annika Mütze, Joshua Wendland +2

Institute of Mathematics Technical University of Berlin Berlin, Germany · Department of Mathematics University of Wuppertal Wuppertal, Germany · Faculty of Computer Science Ruhr University Bochum Bochum, Germany

Autonomous Driving Planning RL for Autonomous Driving Reinforcement Learning

Feb 10, 2025

Occlusion-Aware Contingency Safety-Critical Planning for Autonomous Driving

Ensuring safe driving while maintaining travel efficiency for autonomous vehicles in dynamic and occluded environments is a critical challenge. This paper proposes an occlusion-aware contingency safety-critical planning approach for real-time autonomous driving. Leveraging reachability analysis for risk assessment, forward reachable sets of phantom vehicles are used to derive risk-aware dynamic velocity boundaries. These velocity boundaries are incorporated into a biconvex nonlinear programming (NLP) formulation that formally enforces safety using spatiotemporal barrier constraints, while simultaneously optimizing exploration and fallback trajectories within a receding horizon planning framework. To enable real-time computation and coordination between trajectories, we employ the consensus alternating direction method of multipliers (ADMM) to decompose the biconvex NLP problem into low-dimensional convex subproblems. The effectiveness of the proposed approach is validated through simulations and real-world experiments in occluded intersections. Experimental results demonstrate enhanced safety and improved travel efficiency, enabling real-time safe trajectory generation in dynamic occluded intersections under varying obstacle conditions. The project page is available at https://zack4417.github.io/oacp-website/.

Lei Zheng, Rui Yang, Minzhe Zheng +3

Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · School of Engineering, Great Bay University, Dongguan 523000, China · Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR, China

Safe Motion Planning Motion Planning Autonomous Driving Planning

Aug 18, 2023

Fading Expert Guidance: Bridging Model-Based and Learning-Based Control for Abortable Autonomous Overtaking

Overtaking on two-lane roads is a safety-critical decision-making problem for autonomous vehicles, since oncoming traffic may force the ego vehicle to abort and merge back to its original lane. Deep reinforcement learning (DRL) is promising for such continuous-control tasks, but it requires substantial interaction data and may produce unsafe exploratory actions during training. Model-based controllers provide structured behavior, but depend on model fidelity and hand-designed logic. This paper proposes a fading expert-guidance framework that uses a model-based controller to guide DRL training for autonomous overtaking. The expert combines a constrained iterative LQR (CiLQR) planner with PID-based auxiliary controllers, activated when the optimized plan becomes infeasible. The expert action enters the actor objective through a fading term, interpreted as a time-varying soft trust region around the expert policy. This term first biases the policy toward the expert and then vanishes, allowing optimization according to the reinforcement-learning objective. The method is evaluated with PPO, TD3, and SAC. Simulations with bidirectional traffic show improved sample efficiency and final task performance. The safety effects are algorithm-dependent: guidance reduces vehicle collisions for SAC, eliminates boundary collisions for TD3, and leaves the PPO collision profile unchanged. Overall, the results support fading expert guidance as a practical mechanism for transferring model-based driving knowledge to learning-based controllers without restricting the final policy to imitation in an abortable overtaking task with bidirectional traffic.

Jinxiong Lu, Gokhan Alcan, Ville Kyrki

Intelligent Robotics Group, Department of Electrical Engineering and Automation, Aalto University, 02150 Espoo, Finland

Autonomous Driving Planning RL for Autonomous Driving