Vision-centric autonomous driving models encode thousands of image tokens per frame, yet most depict sky, buildings, or empty road---content irrelevant to planning. Worse, these tokens dilute the planner's attention: we show that randomly discarding 75% of tokens already outperforms the dense model, because the planner can concentrate on the remaining driving-critical content. Building on this finding, we present Geometry-Aware Token Selection (GATS), which uses LiDAR and radar point projections to select a small fraction of image tokens before the vision backbone runs. Because the selector operates on measured 3D geometry rather than visual features, the backbone encodes only the selected patches, cutting end-to-end FLOPs by up to 18×. Each kept token receives a 3D anchor from its LiDAR point and is fed to a self-supervised latent world model that predicts future features and 3D displacements without box annotations. At 25% budget, GATS achieves a 3-sec L2 displacement error of 0.520 m on nuScenes and an EPDMS of 88.9 on NAVSIM---surpassing the dense baseline (0.573 m / 85.9) at 6.6× fewer total FLOPs. At an extreme 10% budget (24 GFLOPs, 18× reduction), it still beats the dense baseline on both nuScenes (0.537 m vs. 0.573 m L2) and NAVSIM (87.9 vs. 85.9 EPDMS). On 739 intent-critical scenarios, GATS covers 95.8% of hazardous vehicles and achieves a 0.09% collision rate---73.5% lower than the dense model (0.34%).
Figures & tables
Figure 2: Method overview. LiDAR and radar measurements provide both patch-routing statistics and metric metadata. GATS selects image patches before visual encoding; Sparse ViT encodes only the selected patches. Their projected LiDAR anchors and geometric statistics are embedded as 3D positional metadata and added to the visual tokens before the world-model encoder. The same anchors provide metric references for displacement prediction and self-supervised feature-geometry matching. At 10% budget the full pipeline costs 24 GFLOPs.
Method
β
Tok.
L2 avg↓
Col% ↓
GFLOPs
Published E2E (box+map sup., different backbone):
UniAD [ 12 ]
100%
—
1.02 ‡
0.29 ‡
—
VAD-Base [ 14 ]
100%
—
0.54 ‡
0.07 ‡
—
SparseDrive [ 27 ]
100%
—
0.58 ‡
0.05 ‡
—
Our models (DINOv2-S, self-supervised, no box/map ann.):
Dense fusion
100%
6913
0.573
0.09
248
Table 1: Planning quality vs. compute on nuScenes val. L2: avg. displacement error (m) over 3 s. Col: collision rate (%, excluding GT-collision timesteps, following [ 12 ] ). GFLOPs: world model + planner only.
Figure 3: Token selection comparison (front camera, 25% global budget). Dense: all 576 tokens. For controlled visual comparison, Random and DINOv2-Attn are shown with the same front-camera token count as GATS at this scene. GATS concentrates on road users and drivable surface near the ego path.
Method
β
EPDMS ↑
GFLOPs
Published (different backbone / supervision):
WoTE [ 18 ]
100%
87.7
—
DiffusionDrive [ 21 ]
100%
85.2
—
DiffusionDrive-v2 [ 30 ]
100%
85.5
—
Our models (DINOv2-S, self-supervised):
Dense fusion
100%
85.9
722
Table 2: NAVSIM v2 navtest results (EPDMS). All our models use the same codebase and v2 scoring function. GFLOPs: world model + planner only (NAVSIM uses 8 cameras, hence higher than nuScenes).
Selector ( β=25% )
Cov. ↑
Col. ↓
GATS
95.8%
0.09%
DINOv2-Attn
93.9%
0.13%
Random
81.5%
0.25%
Grid
80.5%
0.43%
Learned [ 13 ]
70.0%
0.61%
Distance (depth) [ 23 ]
22.1%
0.50%
Table 3: Intent-critical analysis on 739 mined samples at 25% budget. Cov.: hazardous vehicles with ≥ 1 selected token. Col.: collision rate.
Input
Ntok
Entropy ↓
log2Ntok
Dense fusion
3,456
7.0
11.8
Random (25%)
864
6.5
9.8
GATS (25%)
864
5.7
9.8
Table 4: Planner cross-attention entropy (bits, lower = more concentrated). Measured over 800 nuScenes val samples. Ntok : per-frame token count (the world model ingests Th=2 frames, but entropy is measured per frame). Max. possible entropy: log2Ntok .
Figure 4: Qualitative result. Six camera views with GATS selection at 10% budget; bright patches are retained tokens, dark regions are pruned. Most driving-relevant structure is covered while sky and irrelevant background elements remain pruned.
Method
EPE mov @2s ↓
Zero@2s ↑
Rel.% ↑
EPE sta @2s ↓
GATS (25%)
5.76
12.94
55.5%
0.54
Random (25%)
6.81
11.93
42.9%
0.58
DINOv2-Attn (25%)
6.65
11.55
42.4%
0.56
Dense fusion
6.62
11.92
44.5%
0.59
Table 5: Motion prediction on nuScenes val (EPE in m at 2 s, no box supervision). Zero@2s is the mean GT travel distance of each selector’s moving tokens (higher = harder targets). Rel.% measures reduction from this zero-motion baseline. EPE mov is not directly comparable across methods because each selects different points.
Variant
L2 avg↓
ΔL2
Col% ↓
Full (GATS 25%)
0.520
–
0.04
w/o 3D anchoring
0.958
+84.2%
0.43
w/o road stream
0.609
+17.1%
0.13
w/o learned refine.
0.606
+16.5%
0.09
w/o world model
0.574
+10.4%
0.14
w/o past flow
0.569
+9.4%
0.11
Table 6: Ablation on nuScenes val (baseline: GATS 25%). Each row removes one component; ΔL2 is the relative planning degradation.
Method
Clean L2 ↓
Corrupt L2 ↓
Rel. degrad. ↓
Mot. blur ↓
Dense camera
0.668
0.924
+38.3%
+130%
Dense fusion
0.573
0.723
+26.2%
+88%
GATS (25%)
0.520
0.597
+14.8%
+36%
Random (25%)
0.550
0.599
+8.8%
+20%
Grid (25%)
0.604
0.678
+12.2%
+32%
DINOv2-Attn (25%)
0.584
0.690
+18.0%
+44%
Table 7: Robustness to image corruption (severity 3, 25% budget). Clean L2 and corrupted L2 (avg. over 5 types: Gaussian noise, shot noise, motion blur, defocus blur, brightness) in meters.
Configuration
BB
WM
Total
Dense (6 cams)
185
248
433
Sparse 25% (sparse BB)
40
25.9
66
Sparse 10% (sparse BB)
16
7.6
24
Table 8: End-to-end FLOPs (GFLOPs). BB: vision backbone; WM: world model + planner.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : GATS at 25% budget on three intent-critical scenes (6 surround cameras). Per-camera token counts (in parentheses) vary because GATS allocates globally: cameras facing busy road areas receive more tokens. At 25%, most driving-relevant structure is covered while sky and distant buildings remain pruned.
Figure A2 : GATS at 10% budget: urban intersection with FedEx truck, multiple vehicles, and traffic signals. Most driving-relevant structure is covered while sky and distant buildings remain pruned.
Figure A3 : GATS at 10% budget: dense urban traffic with pedestrians on crosswalk, vehicles, and traffic lights. GATS concentrates tokens on all road users while discarding irrelevant background.
Figure A4 : GATS at 10% budget: braking scenario with multiple pedestrians crossing under backlit conditions. GATS selects tokens on pedestrians and nearby vehicles despite challenging lighting.
Figure A5 : Front camera: Dense vs. Random vs. GATS (10% budget) on the braking scenario from Fig. A4 . Random scatters tokens uniformly; GATS concentrates on pedestrians and the lead vehicle.
Figure A6 : Front camera: Dense vs. Random vs. GATS (10% budget) on a scene with a bus and multiple large vehicles. GATS covers all large vehicles; Random misses portions of them.
Figure A7 : Full qualitative result on the urban traffic scene from Fig. A3 (6 cameras). GATS (10% budget) concentrates tokens on all road users while pruning sky and distant buildings.
Backbone
Selector
L2 avg↓
Coverage ↑
DINOv2-S/14
GATS
0.537
86.3%
DINOv2-S/14
DINOv2-Attn
0.628
84.7%
DINOv2-S/14
Grid
0.563
64.4%
DINOv2-S/14
Random
0.560
60.2%
DINOv3-S
GATS
0.550
85.1%
DINOv3-S
Random
0.568
60.2%
Appendix
Table A1: Backbone comparison at 10% budget. GATS consistently outperforms random and other selectors with different backbones.
Selector ( β=10% )
Cov. ↑
Col. ↓
GATS
86.3%
0.09%
DINOv2-Attn
84.7%
0.18%
Random
60.2%
0.38%
Grid
64.4%
0.26%
Learned [ 13 ]
16.5%
0.52%
Distance [ 23 ]
10.7%
0.64%
Appendix
Table A2: Intent-critical analysis at 10% budget on 739 mined samples.
Method
EPE mov @2s ↓
Zero@2s ↑
Rel.% ↑
EPE sta @2s ↓
GATS (10%)
5.10
13.29
61.6%
0.48
Random (10%)
7.22
11.93
39.5%
0.60
Grid (10%)
7.16
11.88
39.8%
0.61
Dense fusion
6.62
11.92
44.5%
0.59
DINOv2-Attn (10%)
5.46
11.40
52.1%
0.50
Distance (10%)
6.18
12.22
49.4%
0.59
Appendix
Table A3: Motion prediction at 10% budget (EPE in m at 2 s).
Variant
L2 avg↓
ΔL2
Col% ↓
Full (GATS 10%)
0.537
–
0.09
w/o radar velocity
0.602
+12.1%
0.15
w/o motion residuals
0.558
+3.9%
0.14
w/o 3D anchoring
0.601
+11.9%
0.10
w/o past flow
0.592
+10.2%
0.10
w/o learned refine.
0.565
+5.1%
0.10
Appendix
Table A4: Ablation at 10% budget on nuScenes val.
Variant
PDMS ↑
EPDMS ↑
ΔEPDMS
Full (GATS 10%)
88.9
87.9
–
w/o motion residuals
86.0
85.0
− 3.3%
w/o 3D anchoring
84.6
83.4
− 5.1%
w/o past flow
84.9
83.9
− 4.6%
w/o learned refine.
84.5
83.5
− 5.0%
w/o road stream
81.7
80.7
− 8.2%
Appendix
Table A5: Ablation at 10% budget on NAVSIM navtest.
Method
Clean L2 ↓
Corrupt L2 ↓
Rel. degrad. ↓
Mot. blur ↓
Dense camera
0.668
0.924
+38.3%
+130%
Dense fusion
0.573
0.723
+26.2%
+88%
GATS (10%)
0.537
0.610
+13.6%
+31%
Random (10%)
0.560
0.624
+11.4%
+27%
Grid (10%)
0.563
0.637
+13.2%
+29%
DINOv2-Attn (10%)
0.628
0.717
+14.2%
+31%
Appendix
Table A6: Robustness at 10% budget (severity 3; same 5 corruption types as Tab. 7 ).