cs.AIMar 11, 2026

A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms

Authors: Kejin Yu, Yuhan Sun, Taiqiang Wu, Ruixu Zhang, Zhiqiang Lin, Yuxin Meng, Junjie Wang, Yujiu Yang

Organizations: 1Tsinghua University · East China Normal University · 3The University of Hong Kong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The development of high-level autonomous driving (AD) is shifting from perception-centric limitations to a more fundamental bottleneck, namely, a deficit in robust and generalizable reasoning. Although current AD systems manage structured environments, they consistently falter in long-tail scenarios and complex social interactions that require human-like judgment. Meanwhile, the advent of large language and multimodal models (LLMs and MLLMs) presents a transformative opportunity to integrate a powerful cognitive engine into AD systems, moving beyond pattern matching toward genuine comprehension. However, a systematic framework to guide this integration is critically lacking. To bridge this gap, we provide a comprehensive review of this emerging field and argue that reasoning should be elevated from a modular component to the system's cognitive core. Specifically, we first propose a novel Cognitive Hierarchy to decompose the monolithic driving task according to its cognitive and interactive complexity. Building on this, we further derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social-game reasoning. Furthermore, we conduct a dual-perspective review of the state-of-the-art, analyzing both system-centric approaches to architecting intelligent agents and evaluation-centric practices for their validation. Our analysis reveals a clear trend toward holistic and interpretable "glass-box" agents. In conclusion, we identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control. For future work, a primary objective is to bridge the symbolic-to-physical gap by developing verifiable neuro-symbolic architectures, robust reasoning under uncertainty, and scalable models for implicit social negotiation.

Figures & tables

Figure 1: Motivation: why explicit reasoning matters in autonomous driving. The left panel summarizes seven recurring reasoning challenges in our taxonomy. The right panel presents three illustrative scenarios (E1–E3) that contrast a representative current AD system driven by rule-based heuristics with a reasoning-in-AD approach that integrates contextual signals, traffic rules, and multi-agent interaction cues via explicit inference (dashed boxes). The comparison highlights how brittle policies can yield unsafe or overly conservative actions (red), while structured reasoning supports context-appropriate decisions (green).
Figure 3: The proposed Cognitive Hierarchy for reasoning in autonomous driving. This framework deconstructs the monolithic “driving” task into three distinct levels of increasing cognitive and interactive complexity: (1) the Sensorimotor Level, (2) the Egocentric Reasoning Level, and (3) the Social-Cognitive Level.
Figure 4: The taxonomy of seven core reasoning challenges in autonomous driving. These challenges are categorized by their corresponding cognitive level: C1–C4 at the Egocentric Reasoning level, while C5–C7 are at the Social-Cognitive level. Each numbered scenario illustrates a specific challenge analyzed in the text.
Figure 5: The chronological evolution of major methods in autonomous driving. This timeline highlights the rapid progression of the field and provides historical context for the thematic complexity taxonomy introduced in Sec. 5.2 .
Figure 6: Among various existing methods, we select and present five cases that enhance capabilities across different dimensions in an AD system: (a) HiLM-D ( Ding et al., 2025a ) focuses on Reasoning-Enhanced Perception; (b) Trace ( Puthumanaillam et al., 2025 ) focuses on Reasoning-Informed Prediction; (c) Driving with Regulation ( Cai et al., 2024 ) focuses on Reasoning in Planning and Decision-Making; (d) ORION ( Fu et al., 2025 ) focuses on Holistic Architectures; (e) Tell-Drive ( Xu et al., 2025a ) focuses on Reasoning in Supporting and Auxiliary Tasks.
Figure 7: The chronological evolution of benchmarks and datasets for autonomous driving reasoning. This timeline illustrates the rapid acceleration of evaluation-centric research and provides the historical context for the thematic taxonomy discussed in Sec. 6.1 .
Figure 8: Among various existing benchmarks, we highlight four widely used cases: (a) NuScenes-QA for scene-level spatial reasoning ( Qian et al., 2024 ) ; (b) OmniDrive for holistic reasoning and planning ( Wang et al., 2025b ) ; (c) DrivingDojo for interactive behavior modeling ( Wang et al., 2024e ) ; (d) DriveBench for safety-critical reliability evaluation ( Xie et al., 2025 ) .
Figure 9: A genealogical chart of the autonomous driving benchmark ecosystem. We illustrate the derivative and inheritance relationships among key datasets, highlighting how foundational platforms (e.g., NuScenes) have inspired a “family” of specialized benchmarks designed to evaluate specific reasoning capabilities.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

DateMethodsAffiliationChallengesKeywords
2023.07Drive like a humanReasoning in Planning and Decision-MakingC5, C7Human-like driving; Social reasoning
( Fu et al., 2024 )  
2023.10Driving with LLMsHolistic Architectures: Integrated and End-to-End AgentsC1, C4Object–language fusion; Explainable planning
( Chen et al., 2024 )  
2023.12DolphinsReasoning in Planning and Decision-MakingC1, C4, C5Multimodal CoT; Adaptive behavior
( Ma et al., 2024a )  
Appendix
Table 1: A comprehensive overview and comparison of key methods for autonomous driving reasoning. This table summarizes the primary attributes of each method, including its publication Date , Affiliation , core reasoning Challenges (C1–C7) it addresses, associated Keywords .
More in this topic

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

Explore similar work

CardsList
Mar 2, 2026

DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving

Multimodal Large Language Models (MLLMs) are rapidly becoming the intelligence brain of end-to-end autonomous driving systems. A key challenge is to assess whether MLLMs can truly understand and follow complex real-world traffic rules. However, existing benchmarks mainly focus on single-rule scenarios like traffic sign recognition, neglecting the complexity of multi-rule concurrency and conflicts in real driving. Consequently, models perform well on simple tasks but often fail or violate rules in real world complex situations. To bridge this gap, we propose DriveCombo, a text and vision-based benchmark for compositional traffic rule reasoning. Inspired by human drivers' cognitive development, we propose a systematic Five-Level Cognitive Ladder that evaluates reasoning from single-rule understanding to multi-rule integration and conflict resolution, enabling quantitative assessment across cognitive stages. We further propose a Rule2Scene Agent that maps language-based traffic rules to dynamic driving scenes through rule crafting and scene generation, enabling scene-level traffic rule visual reasoning. Evaluations of 14 mainstream MLLMs reveal performance drops as task complexity grows, particularly during rule conflicts. After splitting the dataset and fine-tuning on the training set, we further observe substantial improvements in both traffic rule reasoning and downstream planning capabilities. These results highlight the effectiveness of DriveCombo in advancing compliant and intelligent autonomous driving systems.

Enhui Ma, Jiahuan Zhang, Guantian Zheng +10

Autolab, Westlake University · Li Auto Inc · Tsinghua University +1

Autonomous Driving BenchmarksVLMs for Autonomous DrivingCompositional Reasoning
AskPDF
Jun 7, 2026

LUNA-AD: Lightweight Uncertainty-Aware Language Model with Lifelong Learning for Autonomous Driving

While large language models (LLMs) offer promising reasoning capabilities, their integration into safety-critical driving systems is hindered by limited reasoning diversity, high computational overhead, and static learning paradigms. To address these challenges, we propose LUNA-AD, a lightweight uncertainty-aware language model with lifelong learning for autonomous driving (AD). LUNA-AD features a tri-system architecture that reconciles complex multimodal behavioral reasoning, efficient deployment, and continual refinement. We design a multi-agent analytical system to generate uncertainty-aware decision-making demonstrations through diverse hypothesis exploration. A dual-head lightweight heuristic model is distilled to unify the inference of decision distributions and textual explanations while enabling efficient deployment. Furthermore, a reflection-driven lifelong learning mechanism operates on multimodal decision outputs and preserves strategic diversity, allowing for the refinement of candidate decisions and rationales via closed-loop feedback to enhance driving robustness. Extensive experiments on nuPlan benchmarks demonstrate that LUNA-AD achieves state-of-the-art success rates under both non-reactive and reactive modes, with drastically reduced inference latency compared to existing knowledge-driven AD frameworks.

Ruoyu Yao, Pei Liu, Ruiguo Zhong +3

The Hong Kong University of Science and Technology (Guangzhou), China

Continual Learning for LLMsLanguage Model-Based ControlContinual LearningAutonomous Driving PlanningDecision-Making under Uncertainty
AskPDF
May 29, 2026

nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving

Reasoning is essential for autonomous driving (AD) in long-tail scenarios, where vehicles must apply commonsense knowledge, understand spatial relations, infer agent interactions, and make safe decisions. However, existing AD datasets and benchmarks mainly target perception, prediction, or planning, and provide limited supervision for reasoning over realistic long-tail driving scenes. We introduce nuReasoning, a large-scale real-world dataset and benchmark for reasoning-centric AD. Following the lineage of nuScenes and nuPlan, nuReasoning advances real-world AD datasets and benchmarks toward reasoning in long-tail driving scenarios. The dataset contains 20,000 clips, each 20 seconds long, collected across multiple cities, with synchronized multi-camera images, LiDAR data, HD maps, object annotations, and human-verified reasoning annotations spanning Spatial Reasoning, Decision Reasoning, and Counterfactual Reasoning. Unlike prior datasets that focus primarily on visual question answering, nuReasoning supports both reasoning evaluation and planning evaluation, enabling a direct study of how reasoning supervision affects driving performance. Experiments show that fine-tuning VLMs on nuReasoning substantially improves driving-specific question answering, while incorporating reasoning supervision into VLA training improves planning performance even when textual reasoning outputs are disabled at inference time. These results establish nuReasoning as a foundation for evaluating and improving robust, interpretable, reasoning-driven AD systems in realistic long-tail settings.

Zhiyu Huang, Johnson Liu, Rui Song +13

Xuewei · University of California, Los Angeles · 2Motional

Autonomous Driving DatasetsAutonomous Driving BenchmarksLong-Tail Autonomous DrivingAutonomous DrivingVLM ReasoningAutonomous Driving Planning
AskPDF