BEVCast: Path-Adaptive BEV for Efficient End-to-End Autonomous Driving
Organizations: Connected and Autonomous Research Laboratory, University of Delaware, Newark, USA
Abstract
Camera-based driving models build a dense bird's-eye-view (BEV) grid and spend the same computation on every cell of it, although the vehicle crosses only a few of those cells within the planning horizon. We present BEVCast, a two-stage BEV whose resolution adapts to the predicted driving path. A coarse pass encodes the whole scene on a grid of 4.096 m cells. A heat head trained on the recorded path assigns a heat to every coarse cell and the cells with the highest heat are refined into 1.024 m cells under a fixed budget. The remaining cells keep their coarse resolution, so the trajectory head reads a mixed-resolution BEV that still covers the entire scene and needs neither an HD map nor a navigation command. Against a full grid of 1.024 m cells with the same architecture and training recipe, BEVCast at a 10% refinement budget encodes 28% of the BEV queries and needs 66% of the BEV encoder time at batch size 1. On nuScenes its mean displacement error is 1.078 m against 1.149 m and its drivable area violation rate is 1.46% against 2.29%. On NAVSIM navtest its mean displacement error is 0.577 m against 0.598 m and its extended predictive driver model score is 82.51 against 81.28. Next to published planners BEVCast reaches a comparable error on both benchmarks without a navigation command. The resolution of a driving BEV can therefore follow the task instead of the grid.
Figures & tables
| nuScenes planners | Backbone | Sensors | Planner | Cmd | Ego status | Map | L2 1 s | 2 s | 3 s | Mean | Col 1 s | 2 s | 3 s | Mean | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BEV | Planner | ||||||||||||||
| AD-MLP [ 34 ] | - | ego state | regression | ✓ | ✓ | 0.20 | 0.26 | 0.41 | 0.29 | 0.17% | 0.18% | 0.24% | 0.19% | ||
| ST-P3 [ 10 ] | EfficientNet-B4 | C | sampling | ✓ | ✓ | 1.33 | 2.11 | 2.90 | 2.11 | 0.23% | 0.62% | 1.27% | 0.71% | ||
| VAD-Base [ 13 ] | ResNet-50 | C | regression | ✓ | ✓ | ✓ | 0.41 | 0.70 | 1.05 | 0.72 | 0.07% | 0.17% | 0.41% | 0.22% | |
| BEV-Planner [ 18 ] | ResNet-50 | C | regression | ✓ | 0.30 | 0.52 | 0.83 | 0.55 | 0.10% | 0.37% | 1.30% | 0.59% | |||
| PARA-Drive [ 30 ] | ResNet-50 | C | regression | ✓ | ✓ | 0.25 | 0.46 | 0.74 | 0.48 | 0.14% | 0.23% | 0.39% | 0.25% | ||
| NAVSIM navtest | Backbone | Sensors | Planner | Map | NC | DAC | DDC | TLC | EP | TTC | LK | HC | EC | EPDMS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TransFuser [ 6 , 7 ] | ResNet-34 | C+L | regression | ✓ | 96.9 | 89.9 | 97.8 | 99.7 | 87.1 | 95.4 | 92.7 | 98.3 | 87.2 | 76.7 |
| Hydra-MDP++ [ 15 , 32 ] | ResNet-34 | C | vocabulary | ✓ | 97.2 | 97.5 | 99.4 | 99.6 | 83.1 | 96.5 | 94.4 | 98.2 | 70.9 | 81.4 |
| DriveSuprim [ 32 ] | ResNet-34 | C | vocabulary | ✓ | 97.5 | 96.5 | 99.4 | 99.6 | 88.4 | 96.6 | 95.5 | 98.3 | 77.0 | 83.1 |
| BEVCast 10% (ours) | ResNet-50 | C | regression | 98.0 | 94.3 | 99.5 | 99.8 | 87.1 | 97.4 | 97.1 | 98.3 | 84.6 | 82.5 |
| nuScenes 3 s | Queries | L2 | Lat | Lon | Col | NM | DAV | Hard |
| Full grid 1.024 m | 10,000 | 1.149 | 0.206 | 1.083 | 0.44% | 2.32% | 2.29% | 2.32 |
| Coarse only 4.096 m | 625 | 1.170 | 0.201 | 1.108 | 0.48% | 2.41% | 1.70% | 2.32 |
| BEVCast 20% | 3,750 | 1.069 | 0.207 | 1.002 | 0.42% | 2.26% | 2.21% | 2.38 |
| BEVCast 10% | 2,805 | 1.078 | 0.191 | 1.022 | 0.40% | 2.08% | 1.46% | 2.32 |
| BEVCast 5% | 2,340 | 1.047 | 0.175 | 0.996 | 0.37% | 1.52% | 1.25% | 2.24 |
| Refined cells only 10% | 2,242 | 1.069 | 0.186 | 1.012 | 0.47% | 2.05% | 1.60% | 2.27 |
| Violation rate | Static | Start | Stop | Left | Right | Turns | Straight | All |
|---|---|---|---|---|---|---|---|---|
| Frames | 728 | 207 | 100 | 240 | 339 | 579 | 3,355 | 4,969 |
| Full grid 1.024 m | 1.92% | 3.06% | 7.33% | 5.14% | 9.24% | 7.54% | 1.27% | 2.29% |
| Coarse only 4.096 m | 0.82% | 4.03% | 3.67% | 3.89% | 7.28% | 5.87% | 0.96% | 1.70% |
| BEVCast 20% | 1.83% | 2.74% | 3.33% | 7.36% | 8.46% | 8.00% | 1.22% | 2.21% |
| BEVCast 10% | 0.73% | 3.22% | 4.67% | 4.86% | 4.72% | 4.78% | 0.83% | 1.46% |
| BEVCast 5% | 0.60% | 2.09% | 3.33% | 4.86% | 4.72% | 4.78% | 0.67% | 1.25% |
| NAVSIM navtest | L2 1 to 3 s | 1 to 4 s | Col | EPDMS | DAC | EC |
|---|---|---|---|---|---|---|
| Full grid 1.024 m | 0.598 | 0.907 | 0.09% | 81.28 | 93.4 | 83.6 |
| Coarse only 4.096 m | 0.585 | 0.887 | 0.07% | 81.85 | 94.0 | 84.0 |
| BEVCast 20% | 0.590 | 0.894 | 0.05% | 81.64 | 93.5 | 83.9 |
| BEVCast 10% | 0.577 | 0.876 | 0.04% | 82.51 | 94.3 | 84.6 |
| BEVCast 5% | 0.580 | 0.881 | 0.06% | 82.22 | 94.0 | 84.1 |
| Queries | Time per sample (ms) | BEV | ||||
| Batch size 1 | frame | sample | Backbone | BEV | Model | vs full |
| Full grid 1.024 m | 10,000 | 30,000 | 33.4 | 23.6 | 57.0 | 100% |
| Coarse only 4.096 m | 625 | 1,875 | 33.3 | 9.5 | 42.8 | 40% |
| BEVCast 20% | 3,750 | 5,000 | 33.3 | 15.7 | 49.1 | 67% |
| BEVCast 10% | 2,805 | 4,055 | 33.4 | 15.6 | 48.9 | 66% |
| BEVCast 5% | 2,340 | 3,590 | 33.4 | 15.5 | 48.9 | 66% |
End-to-End Autonomous Driving
End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.
See all 249 papers