AutoE2E: Don't speak - Language free end-to-end autonomy A Real-Time Driving Model for L2++ ADAS and L4 Robotaxis
Organizations: Autoware Foundation Robotaxi Working Group
Abstract
AutoE2E is an open-source end-to-end driving model developed by the Autoware Foundation Robotaxi Working Group. It introduces a temporal bird's-eye-view (BEV) architecture that combines multi-camera perception, HD-map information and route conditioning to predict future vehicle motion. The evaluated model uses a pretrained camera-BEV backbone, temporal feature fusion and a lightweight recurrent trajectory planner. It contains approximately 79.9 million parameters, of which approximately 1.45 million are trainable in the evaluated configuration. The v1.0 release provides model checkpoints, training and evaluation code, and DataModelConsole, a dashboard for visualizing and inspecting model experiments. On the KITScenes validation benchmark, the fine-tuned checkpoint achieves an average displacement error (ADE) of 0.745 m and a final displacement error (FDE) of 2.020 m at a three-second prediction horizon. The paper also reports preliminary real-vehicle research conducted with the Technical University of Munich's EDGAR platform, alongside ongoing collaboration with the Karlsruhe Institute of Technology. AutoE2E v1.0 is an early open research release. Its quantitative evaluation is open-loop, and the model has not yet demonstrated validated closed-loop autonomous driving. The release is intended to provide the autonomous-driving community with an inspectable baseline for further research, experimentation and collaborative development.
Figures & tables
| ch | Name | Source primitive (Lanelet2 [ 13 ] ) | Value | Mean occ. [%] |
|---|---|---|---|---|
| 0 | Drivable area | Lanelet polygons (excluding crosswalks) | binary | 15.55 |
| 1 | Lane boundary | Left/right boundary lines (width 1.0 m) | binary | 8.29 |
| 2 | Lane centerline | Centerline (width 1.0 m) | binary | 5.52 |
| 3 | Intersection | Lanelet polygons with a turn_direction attribute | binary | 1.46 |
| 4 | Crosswalk | Polygons of subtype crosswalk | binary | 0.96 |
| 5 | Stop line | Stop-line polylines (width 1.0 m) | binary | 0.18 |
| Component | Parameters | State |
|---|---|---|
| ResNet-50 backbone | 23,508,032 | frozen |
| Feature pyramid (4 levels) | 3,278,592 | frozen |
| BEV queries 300 200 256 | 15,360,000 | frozen |
| Row, column, level, and camera embeddings | 66,560 | frozen |
| Encoder, 6 layers | 4,940,928 | frozen |
| Front cross-attention (copy) | 328,960 | frozen |
| System | Camera-only prim. eval. | HD map input | Route input | Temp. BEV | Unified det./track./occ. | Closed-loop eval. | Evaluation data |
|---|---|---|---|---|---|---|---|
| UniAD [ 3 ] | — (online map est.) | Navigation command | — | nuScenes | |||
| VAD [ 7 ] | — (vector map est.) | Navigation command | Partial | (CARLA) | nuScenes / CARLA | ||
| TransFuser [ 8 ] | — (camera + LiDAR) | — | Target point | — | Aux. tasks | (CARLA) | CARLA |
| Qwen-Drive-1.0 [ 1 ] | — (online est.) | n/a | — | (3D perc.) | — | nuScenes | |
| Alpamayo 1.5 | — (camera + nav.) | n/a | (nav. command) | n/a | — | (AlpaSim) | PhysicalAI-AV, AlpaSim |
| METEOR | n/a | n/a | n/a | n/a | n/a | n/a | internal, unpublished |
| Model | ADE 1 s | ADE 2 s | ADE 3 s | ADE 5 s | FDE 1 s | FDE 2 s | FDE 3 s | FDE 5 s |
|---|---|---|---|---|---|---|---|---|
| AutoE2E (fine-tuned) | 0.1472 | 0.3729 | 0.7449 | 1.9405 | 0.2850 | 0.9239 | 2.0196 | 5.5645 |
| AutoE2E (baseline) | 0.1951 | 0.4723 | 0.9147 | 2.2951 | 0.3779 | 1.1331 | 2.4226 | 6.3885 |
| Model | ADE 1 s | ADE 2 s | ADE 3 s | ADE 5 s | FDE 1 s | FDE 2 s | FDE 3 s | FDE 5 s |
|---|---|---|---|---|---|---|---|---|
| AutoE2E (fine-tuned) | 0.1463 | 0.4121 | 0.8263 | 2.1042 | 0.3107 | 1.0453 | 2.2199 | 5.9422 |
| AutoE2E (baseline) | 0.1602 | 0.3979 | 0.7924 | 2.1120 | 0.3134 | 0.9727 | 2.1621 | 6.1751 |
| Model | Setting | Evaluated on | ADE/FDE@1s | ADE/FDE@2s | ADE/FDE@3s | ADE/FDE@5s |
|---|---|---|---|---|---|---|
| Qwen-Drive-1.0 [ 1 ] | first of 6 trajectories | 202 kf | 0.173/0.349 | 0.422/0.981 | 0.764/1.862 | 1.685/4.331 |
| AutoE2E (fine-tuned) | single | 202 kf | 0.147/0.296 | 0.380/0.933 | 0.744/1.980 | 1.883/5.477 |
| SimForge (7Cam-VT) | first of 6 | 202 kf | 0.159/0.336 | 0.427/1.047 | 0.831/2.209 | 2.077/5.756 |
| AutoE2E (baseline) | single, zero-shot | 202 kf | 0.176/0.339 | 0.431/1.055 | 0.854/2.302 | 2.232/6.318 |
| Alpamayo 1.5 | first of 6 trajectories | 202 kf | 0.187/0.409 | 0.551/1.441 | 1.124/2.988 | 2.564/6.307 |
| METEOR | top-1 of 3 paths | 202 kf | 0.219/0.484 | 0.630/1.574 | 1.240/3.282 | — |
End-to-End Autonomous Driving
End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.
See all 249 papers