cs.ROMar 17, 2026

The Era of End-to-End Autonomy: Transitioning from Rule-Based Driving to Large Driving Models

Authors: Eduardo Nebot, Julie Stephany Berrio Perez

Organizations: The Australian Centre for Robotics, School of Aeronautical, Mechanical and Mechatronic Engineering, The University of Sydney, Australia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Autonomous driving is undergoing a shift from modular rule based pipelines toward end to end (E2E) learning systems. This paper examines this transition by tracing the evolution from classical sense perceive plan control architectures to large driving models (LDMs) capable of mapping raw sensor input directly to driving actions. We analyze recent developments including Tesla's Full Self Driving (FSD) V12 V14, Rivian's Unified Intelligence platform, NVIDIA Cosmos, and emerging commercial robotaxi deployments, focusing on architectural design, deployment strategies, safety considerations and industry implications. A key emerging product category is supervised E2E driving, often referred to as FSD (Supervised) or L2 plus plus, which several manufacturers plan to deploy from 2026 onwards. These systems can perform most of the Dynamic Driving Task (DDT) in complex environments while requiring human supervision, shifting the driver's role to safety oversight. Early operational evidence suggests E2E learning handles the long tail distribution of real world driving scenarios and is becoming a dominant commercial strategy. We also discuss how similar architectural advances may extend beyond autonomous vehicles (AV) to other embodied AI systems, including humanoid robotics.

Figures & tables

Fig. 1: Authors in [ 16 ] define (a) the classical modular approach separates perception, prediction, and planning through intermediate representations such as bounding boxes and trajectories. (b) The end-to-end paradigm jointly learns interconnected modules, allowing information flow and backpropagation across perception, mapping, prediction, and planning components.
Fig. 2: Training Process for large Driving Models. Phase one use a curated dataset based on good driving behaviors to obtain a baseline model. In Phase 2, additional data, comprising fleet-collected edge cases and synthetically generated scenarios, are used to further train the policy via reinforcement learning, yielding an updated model with improved safety.
Fig. 3: FSD Supervised operation: The driver select the destination and initiate the system; the vehicle will then perform 100% of the Dynamic Driving Task (DDT) for the entire journey to the destination.
Fig. 4: Visualization Interface: The system provides the driver with clear situational awareness and communicates the vehicle’s immediate intended action and trajectory. (a) Top left: the vehicle will exit the roundabout. (b) Top right: the vehicle will follow the road while turning right. (c) Bottom right: the vehicle will turn right at an intersection. (d) The vehicle will proceed straight, while the interface will display road infrastructure and nearby vehicles to indicate the vehicle’s intended path and surrounding context.
Fig. 5: Visual–Haptic–Dynamic interface: (a) Top left: the vehicle is waiting for a gap in traffic to merge. (b) Top right: Haptic—the steering wheel begins to rotate, indicating the vehicle has detected an opening. (c) Top left: Visual—the planned trajectory starts to appear; Haptic—the steering wheel moves; Dynamic—the vehicle accelerates slowly to alert the driver that it is about to move. (d) Visual—the full trajectory is displayed in blue; Dynamic—the vehicle begins the merge.
Fig. 6: Miles driven before major/ minor collisions (non-highway, worldwide), showing higher distances between collisions for Tesla vehicles using FSD (Supervised) compared to manual driving and the U.S. average. [ 30 ]
More in this topic

End-to-End Autonomous Driving

End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.

See all 227 papers

Explore similar work

CardsList

  1. A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform

    Oct 1, 2026Chengkai Xu, Yiming Cui, Jiaqi Liu +7End-to-End Autonomous Driving

    College of Transportation, Tongji University, Shanghai 201804, China

  2. End-to-End Learning vs. Modular Architectures: Comparative Insights into Autonomous Driving Systems

    Oct 1, 2026Kartik B. KapseEnd-to-End Autonomous DrivingAutonomous Driving

    AI Engineering of Autonomous Systems, Dept. Electrical Engineering and IT, Technische Hochschule Ingolstadt

  3. Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms

    Aug 20, 2026Yanchen Guan, Xingcheng Liu, Bin Rao +7End-to-End Autonomous DrivingAutonomous Driving Planning

    State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau SAR, 999078, China · College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing, 400044, China · Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong, 999077, China +2