Organizations: Institute for AI Industry Research (AIR), Tsinghua University · HiThink Research · The Hong Kong University of Science and Technology (Guangzhou) · Agency for Science, Technology and Research (A*STAR), Singapore · University of Washington · Renmin University of China · Beihang University
Autonomous driving is turning to vision-language-action models (VLAs). They answer and plan by decoding, one step at a time. Each step is another forward pass, so a plan can take over a second of GPU time. Embodied AI often pairs such a slow, reasoning System 2 with a fast System 1 that acts. Human drivers plan at a glance, with System 1. Faster designs usually put System 1 outside the vision-language model (VLM), in a planner without its abilities. Yet before a VLM writes its first token, it has already read the scene into one hidden state. We show that, once trained, this hidden state is enough to drive and to answer questions about what the model sees. SuperGlance reads it to turn a decoder-only VLM into an encoder-style driving policy. Its linear head scores a fixed set of 5,273 candidate trajectories. With the set fixed, rewards are computed offline, so supervised targets can be reward-weighted and reinforcement learning maximizes the exact expected reward. Before and during policy training, multiple-choice training keeps the VLM's visual abilities in one pass, so the policy still answers questions about driving scenes. Training questions are built against shortcuts. Each must be answered right in two option orders. Against drift, a KL term keeps each stage near its starting model. One pass offers VLAs a more plausible path to System 1. SuperGlance answers visual multiple-choice questions as well as its base VLM does by decoding. It plans in 84 ms of GPU time, six times faster than Qwen-Drive-1.0, and at least as well on NAVSIM, PAI-AV and WOD-E2E. In AlpaSim closed loop, km per at-fault incident holds for both planners up to a 300-ms delay and drops beyond. With GPU time charged as delay, SuperGlance's lead over Qwen-Drive-1.0 grows from 2.8 to 3.9 times. Delayed by 1.2 s, our model falls below Qwen-Drive-1.0 at its own 518 ms.
Figures & tables
Figure 1: Three ways a vision-language model produces a plan. Only (c) has no loop. (a) Autoregressive decoding writes a trajectory one token per forward pass (text waypoints, AutoVLA [ 2 ] ). (b) Diffusion decoding runs the VLM once, then a planning expert refines a noisy trajectory over several steps (Qwen-Drive-1.0 [ 1 ] ). (c) SuperGlance reads the last hidden state h once, at [ANS] , the prompt’s last position. The Policy Head gives a probability (outline width and opacity) to each of 5,273 fixed candidates, and for questions, the LM head gives one to each option letter. Each bottom strip shows one plan’s measured GPU time to scale; in (a), our VLM writing text runs off its axis.
Planning
Sequential
Planner
Keeps
RL
Method
paradigm
steps
in VLM
language
objective
Decoding or denoising VLAs
AutoVLA [ 2 ]
Autoregressive
10 + text
✓
✓
GRPO
Qwen-Drive-1.0 [ 1 ]
Flow matching
11 + text
×
✓
GRPO
One-pass or fast VLM planners
SimLingo [ 10 ]
Regression
1 + text
✓
✓
×
Table 1: Related driving policies. Text adds a step per token.
Figure 2: Training data of each stage. Every stage trains an answer read in one forward pass: a distribution over option letters, and in stage 3 also over candidates. Left: held-out samples where Qwen3.5-9B is wrong and the model after that stage is right. A tick marks our pick, a cross Qwen3.5-9B’s, and questions are lightly abridged. In the stage-3 row, the four paths are our supervised model’s pick among its 5,121 candidates and three low-reward candidates, each with its score. Right: each stage’s sources with shares (rings) and counts, and the 5,273 candidates seen from above, coloured by origin ( Sec. 3.1 ).
Figure 3: From rewards to a policy on one navtest scene. (a) Each candidate is colored by its reward; unsafe ones are gray ( Sec. 3.2 ). (b) In the supervised policy, width and opacity grow with probability, and unsafe candidates hold nearly a quarter of it. (c) The RL optimum, computed from (a) and (b) by Eq. 4 without the RL model, gives them none. Bars give the unsafe share of candidates (a) and of probability (b, c).
Figure 4: Questions built against shortcuts in stages 1 and 2. A one-pass letter can be right by position alone. (a) A shortcut answers a question right (left) and its twin wrong (right); gold is dark. (b) Rows use the stored order (gold at C) and columns the shifted one (gold at D); bars show illustrative letter probabilities, shades pair probabilities. (c) Top: letter shares on 3DSRBench’s 4-option questions, with gold spread evenly (dashed: 25% each); Qwen3.5-9B picks C or D for 75%, by position, and after stage 2 each letter takes 23–27%.
Qwen3.5-9B
SuperGlance after stage
Benchmark
letter
decoded
Qwen-Drive-1.0
1
2
3
Vision
CV-Bench [ 64 ]
86.36
86.47
85.29
87.60
87.66
87.40
BLINK [ 65 ]
79.37
76.62
77.60
84.68
81.34
79.37
3DSRBench [ 66 ]
45.39
57.09
46.81
60.27
60.50
58.29
Language
Table 2: Question answering. Accuracy (%) after each training stage of SuperGlance; all models are scored by us with the same prompts and option-letter readout. Shading deepens from the worst to the best score, by row here and by column below. The decoded column reads Qwen3.5-9B’s written replies (thinking off). Stage 3 is our final RL model. For BLINK we use counting, object localization, relative depth and spatial relation. DriveLM (our scene-split test questions) and 3DSRBench count an answer only if it holds in every cyclic option order. Only DriveLM is in our training data.
Figure 5: Time per plan and distance in closed loop. Top: a plan takes 84 ms of GPU time in our single pass, 518 ms in Qwen-Drive-1.0 with 10 flow steps and 5,558 ms when our VLM writes it as text. Bottom: in AlpaSim, each time (dotted) becomes the next 100-ms lag, and both planners keep their distance per at-fault incident within 8% up to 300 ms. At 600 ms, Qwen-Drive-1.0 falls to 0.317 km and ours to 1.061 km; at 1.2 s ours falls to 0.259 km, below Qwen-Drive-1.0 at its own 518 ms. At 5 s every plan arrives after its last pose (hollow): both planners drive more slowly and vehicles behind hit them, which is not at fault and ends the scenario as a pass.
Method
RL
Coll. ↓
Off-road ↓
Prog. ↑
km / incident ↑
Pass ↑
Each plan’s GPU time as delay
Qwen-Drive-1.0, 518 ms
32.6 / 18.1
19.3
67.9
0.229 / 0.317
36.6
+ reasoning, 1.2 s
26.7 / 21.0
36.3
51.8
0.145 / 0.158
9.3
SuperGlance, 84 ms
✓
10.9 / 6.8
7.5
88.5
0.970 / 1.248
72.2
No delay (simulator default)
Qwen-Drive-1.0 [ 1 ]
32.8 / 14.4
15.4
71.1
0.284 / 0.461
58.7
Table 3: AlpaSim closed loop. Top: each plan’s GPU time acts as its delay, rounded up to a lag of 600 ms, 1.2 s or 100 ms (appendix Appendix D ); bottom: no delay. Qwen-Drive-1.0 plans with reasoning except in its 518 ms row. Coll. and km / incident: all / at-fault; Coll., off-road, progress and pass in %.
Method
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
Supervised
TransFuser [ 76 ]
97.7
92.8
79.2
92.8
100.0
84.0
DRAMA [ 77 ]
98.0
93.1
80.1
94.8
100.0
85.5
Hydra-MDP [ 33 ]
98.3
96.0
78.7
94.6
100.0
86.5
DiffusionDrive [ 78 ]
98.2
96.2
82.2
94.7
100.0
88.1
Epona [ 18 ]
97.9
95.1
80.4
93.8
99.9
86.2
Table 4: NAVSIM v1.1 navtest. One plan per scene; shading ranks the methods within each group, and Comf. is unshaded.
Method
RL
3-s ADE ↓
5-s ADE ↓
RFS ↑
MindVLA-U1 [ 80 ]
1.16
2.67
7.77
AutoVLA [ 2 ]
✓
1.35
2.96
7.56
MindVLA-U1
✓
1.09
2.66
7.87
Qwen-Drive-1.0 + reasoning [ 1 ]
1.19
2.65
7.78
Qwen-Drive-1.0 + reasoning
✓
1.19
2.67
7.91
SuperGlance
✓
1.190
2.546
7.987
Table 5: WOD-E2E test split. ADE in metres; our 4-s plans are extended to 5 s at constant velocity.
One plan
Best of six
Method
RL
3 s
5 s
3 s
5 s
DriveWAM [ 20 ] †
0.69
–
0.38
–
Qwen-Drive-1.0 + reasoning [ 1 ]
0.414
1.205
0.385
1.095
Qwen-Drive-1.0 + reasoning
✓
0.454
1.231
0.415
1.102
SuperGlance
0.394
1.109
0.284
0.821
SuperGlance
✓
0.373
1.076
0.271
0.777
Table 6: PAI-AV open loop. ADE (m) ↓ on 700 held-out frames. One plan: our top-64 mean and Qwen-Drive-1.0’s first sample; best of six: our six most probable candidates and its six samples. We extend our 4-s plans to 5 s at constant velocity; † : reproduced on these frames in the Qwen-Drive-1.0 paper; – : not evaluated.
Figure 6: Language steers the plan. Each instruction selects its own plan from the same candidates. Left: two instructions on a training scene, each with the candidates that follow it (red); line weight is the target q∝exp(−d/τ) , τ=1 m, with d the distance to the logged future in what the instruction leaves free (across the path for a speed instruction). Right: plans for both instructions (light, dark) in three Language2Drive scenes, boxes at 4 s; ours (red) change with the instruction, Qwen-Drive-1.0’s (purple) barely.
Method
SemSISR ↑
SemCFPS ↑
SCS ↑
PCS ↑
Qwen-Drive-1.0 (4B) [ 1 ]
15.8
0.0
25.3
2.8
AutoVLA [ 2 ]
18.8
0.0
30.2
5.4
DriveMA-2B [ 81 ]
20.4
0.0
30.6
6.7
ReCogDrive [ 5 ]
13.2
0.0
22.5
2.8
OneVL [ 82 ]
20.7
0.0
30.5
4.6
SuperGlance, no instruction
11.8
0.0
26.2
6.2
Table 7: Language2Drive, NAVSIM track. 38 instruction pairs in four wordings each, scored with the benchmark’s current scorer, others from their predictions. SemSISR, SemCFPS: single instructions and pairs followed (%); SCS, PCS: graded scores.
Open loop
Closed loop (AlpaSim)
Output
Target
PDMS ↑
km / incident ↑
Pass (%) ↑
Text, decoded
Imitation
87.23
–
–
Linear regression
Imitation
87.82
1.360
70.3
Policy Head
Imitation
82.03
1.199
71.0
Policy Head
Reward
91.30
1.486
73.4
Qwen-Drive-1.0 + reasoning [ 1 ]
88.2
0.461
58.7
Table 8: Output and target. Variants of our supervised model before its last phase, with Qwen-Drive-1.0 for reference. Imitation: the target follows the human trajectory; – : not evaluated (writing the plan as text takes 5,558 ms, longer than the 4 s it covers). PDMS is on navtest; AlpaSim without delay, km per at-fault incident.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Held-out question samples of stages 1 and 2. Each prompt is shown beside its image as the model reads it, verbatim apart from the cut marked [...] , with the gold option highlighted and its letter as the target at [ANS] ; the grey tag gives each question’s ID and split.
Source
Questions about
Options
Questions
Drawn (%)
Stage 1: 1,158,150 questions, 500,000 drawn
GQA [ 52 ]
Relations, attributes and objects (yes/no); colour, material, category
Table 9: Question sources of stages 1 and 2. Questions: training questions in the pool; drawn: share of the draws in training. Indented rows are built from the source above, and GQA’s flipped questions count toward its share. Text: CSQA [ 89 ] , SIQA [ 90 ] , CosmosQA [ 91 ] , PIQA [ 92 ] , MedMCQA [ 93 ] , MedQA [ 94 ] , QASC [ 95 ] , OpenBookQA [ 96 ] , AQuA [ 97 ] , WANLI [ 98 ] , α NLI [ 99 ] , StrategyQA [ 100 ] and RuleTaker [ 101 ] , at most 40,000 questions each.
Source
Rows
Drawn (%)
Target q or reward r
Rows from
Supervised: 2,191,689 rows, 20,900 steps of 128 rows, 5,121 candidates
NAVSIM scenes
92,339
12.17
q∝exp((PDMS⋅DDC−0.1ADE)/0.1)
One per navtrain scene of 1,083 logs
NAVSIM frames
44,243
5.83
q∝exp(−ADE/1m)
Other frames of 1,076 of these logs that pass the navtrain filter
PAI-AV logged
1,044,771
36
q∝exp(−ADE/0.5m)
149,253 training clips, every 2 s
PAI-AV rollouts
46,583
4.5
As above, with the ADE to a recovery trajectory
AlpaSim rollouts of earlier models in 492 scenes
WOD-E2E
406,205
18
q∝exp(−ADE/1m)
Every 10-Hz frame of 1,991 training segments
Appendix
Table 10: Training pools of stage 3 , by phase. Drawn: share of the rows drawn in the phase; ADE: metres to the logged future over the eight poses at 0.5–4 s; R~ : the RFS with the logged future as the only rated trajectory. None of the phases trains on the 10,949 scenes of NAVSIM’s 109 held-out training logs.
NAVSIM
PAI-AV
WOD-E2E
Camera
nuPlan’s [ 102 ] front camera, CAM_F0, as logged
The 120 ∘ f-theta front-wide camera, re-projected into CAM_F0
FRONT, FRONT_LEFT and FRONT_RIGHT, re-projected into CAM_F0 by rotation, each pixel from the camera with the nearest optical axis
Current frame
1920 × 1080 with CAM_F0’s intrinsics and distortion; re-projected frames are saved at JPEG quality 92. The processor resizes the frame to 1024 × 576, its limit of 589,824 pixels, which gives 576 visual tokens.
Past frames
1.5, 1.0 and 0.5 s earlier, from the 2-Hz log
The same times, nearest frame within 0.1 s; clips with an even hash, about half
None
556 × 313 (at most 174,080 pixels), 544 × 320 in the processor, 170 visual tokens each. Rows without all three, rollout rows and question rows stay single-frame; 237,246 of the 391,322 RL rows have past frames.
Sampling
One row per navtrain scene (4 past and 10 future frames, a route), and other frames of the same logs that pass this filter
Every 2 s from 1.9 s after a clip’s start to 4.4 s before its end; RL: up to 2 per clip
Every 10-Hz frame; 46 segments held out by hash
Speed and acceleration
Current ego status
Egomotion at about 10 Hz, central differences over ± 0.2 s
Last past state, at 4 Hz
Appendix
Table 12: Driving inputs by source. All sources share the prompt of Tab. 11 and the camera model of CAM_F0; positions are in the current ego frame, speed is ∣v∣ and acceleration longitudinal.
Stage 1
Stage 2
Stage 3, supervised
Last phase
RL
Starts from
Qwen3.5-9B, new LoRA
Stage-1 model, merged; new LoRA
Stage-2 model, merged; new LoRA; Policy Head at zero
The supervised model
The last-phase model; Policy Head refit to 5,273 rows
Trainable
80,216,064
80,216,064
80,216,064 + 20,980,737
80,216,064 + 20,980,737
80,216,064 + 21,603,481
LoRA
Rank 32, α 64, dropout 0, on 200 projections: q, k, v and o of the 8 full-attention layers; in_proj_qkv, in_proj_z and out_proj of the 24 Gated DeltaNet layers; gate, up and down of all 32 MLPs. The vision tower, the embeddings and the language-model head stay frozen.
Objective
Paired Brier, 32 draws per option order
As stage 1
Cross-entropy to q ; questions as in stage 2
As stage 3
Exact expected reward
KL term
0.1 to Qwen3.5-9B
0.1 to the stage-1 model
0.1 to the stage-2 model, on question rows
As stage 3
β=0.3 to pSFT
Batch
64
64
128
128
64
Appendix
Table 13: Training settings. Every stage trains a LoRA of the same shape on the language model, and stage 3 adds the Policy Head. The last phase continues the supervised model of stage 3, and RL the last-phase model, without merging. Batches and draws count questions in stages 1 and 2 and rows in stage 3. Candidate rewards take about 1,600 single-thread CPU process-hours, and rendering the frames of one AlpaSim rollout for the rollout rows takes about 1.1 H200 GPU minutes.
Candidate set
K
Spacing (m)
Best PDMS
Nearest PDMS
k-means 256 + stationary
257
0.98
99.21
77.88
k-means 1,024 + stationary
1,025
0.61
99.66
82.76
k-means 4,096 + stationary
4,097
0.40
99.84
85.69
Appendix
Table 14: Coverage of NAVSIM scenes by k-means sets of human trajectories. On 10,549 of the 10,949 held-out training scenes, the columns give the mean distance between neighbouring candidates and the mean PDMS of the best candidate and of the candidate nearest to the human trajectory. More centres mainly lift the nearest candidate; the best one is near the PDMS ceiling at every size. Shading deepens from worst to best in each PDMS column.
Benchmark
What it is
Loop
Split and size
Metrics
Horizon
NAVSIM [ 50 ] v1.1
Scenes of nuPlan [ 102 ] logs; the PDM scorer simulates each plan
Open
navtest: 12,146 scenes from 136 logs
PDMS: NC × DAC × a weighted sum of EP, TTC and Comf.
4 s at 2 Hz
WOD-E2E [ 59 ]
Long-tail scenes; RFS scores a plan against rated trajectories
Open
Test split, scored by the official server; the 479 rated validation frames provide RL rewards
RFS; 3- and 5-s ADE
5 s at 4 Hz
PAI-AV [ 58 ]
Driving clips with released egomotion
Open
Qwen-Drive-1.0’s leakage-free subset: 700 frames, five in each of 140 held-out test clips
3- and 5-s ADE of one plan and of the best of six
5 s at 10 Hz
AlpaSim [ 13 ]
Simulator that renders the camera along the car’s own path in PAI-AV-NuRec [ 74 , 75 ] v26.01 scenes
Closed
916 scenarios; one whose route check fails counts as failed, with score zero
Collision and off-road rates, progress, km per at-fault incident, pass rate and mean scenario score
Whole scenario, a call every 100 ms
Language2Drive [ 83 ]
Two instructions per scene; checks whether each plan follows its own (NAVSIM track)
Open
38 instruction pairs in four wordings each
SemSISR, SemCFPS: single instructions and pairs followed; SCS, PCS: graded scores
4 s
Appendix
Table 15: The five benchmarks. Four score each plan once (open loop); in AlpaSim, the car drives its plans (closed loop). Where a benchmark scores 5 s, our 4-s plans are extended at constant speed.
Method
Sequential passes
GPU (ms)
End to end (ms)
SuperGlance
1
83.8
145.5
ReCogDrive [ 5 ]
6
198.6
296.5
Qwen-Drive-1.0 [ 1 ]
11
518.3
1,014.1
+ reasoning
28
1,213.2
1,710.1
AutoVLA [ 2 ]
45
1,529.3
1,900.3
Text waypoints
∼ 155
5,557.7
5,622.4
Appendix
Table 16: Time per plan. GPU time grows with the number of sequential passes. Times are medians of 100 calls, two on each of 50 navtest scenes, on one H200 at batch size one in eager PyTorch; end to end is the median of each call’s total, which adds input processing (image decoding, resizing and tokenization) to the GPU time. With reasoning and for text waypoints, the number of passes varies by call, and the table gives its median. Shading deepens from the slowest to the fastest time in both time columns.
Model
Collision (%)
Off-road (%)
Progress (%)
km / incident
Mean score
Pass (%)
Qwen-Drive-1.0 + reasoning
32.8 / 14.4
15.4
71.1
0.284 / 0.461
0.516
58.7
Qwen-Drive-1.0 + reasoning, RL
43.6 / 15.6
13.1
57.3
0.194 / 0.383
0.391
53.6
Imitation, current frame
15.4 / 9.1
9.8
84.5
0.684 / 0.914
0.603
64.4
Imitation, history frames
10.3 / 5.1
9.8
87.4
0.888 / 1.199
0.682
71.0
Reward target, current frame
15.1 / 8.1
9.3
84.5
0.708 / 0.993
0.608
65.3
Reward target, current frame, extended set
15.3 / 8.1
8.9
84.5
0.714 / 1.018
0.611
65.6
Appendix
Table 17: AlpaSim results by training recipe. All rows run the 916 scenarios without delay, with the same simulator configuration and scorer, and our rows read the weighted top-64 mean at each call. Blocks: Qwen-Drive-1.0’s planners, our supervised models and our RL models; the indented row runs the model above it without the PDM simulator. Collision and km per incident: all / at-fault. Shading deepens from the worst to the best value in each column. Last phase: the last supervised phase of stage 3, with WOD-E2E rows around the rated frames ( Tab. 10 ); the other supervised rows stop before it.
Figure 8: Closed loop in AlpaSim, without delay. The plans of Qwen-Drive-1.0’s RL planner lean left of the human trajectory, by a median of about 0.4 m at 2–3 s ahead, while ours stay centred. (a) Lateral offset of the car and of its plan from the human trajectory, over the calls in the 916 scenarios made while the car moves and before the first incident: lines give the median, bands the middle 50%. (b, c) Front-camera frames with the plan returned at each frame, and a bird’s-eye view at the moment of contact: Qwen-Drive-1.0’s plans bend over the centre line into an oncoming car (b) and, at a red light, through the car standing ahead, while ours waits (c).
Qwen3.5-9B
Final model
Benchmark
Q
Letter
Decoded
Thinking
Capped (%)
Letter
Decoded
CV-Bench
1,320
87.68
87.05
85.36
6.36
88.55
88.61
BLINK
509
79.37
76.62
75.44
10.41
79.37
79.17
3DSRBench
1,000
43.50
58.20
55.00
9.17
58.10
57.60
DriveLM
1,000
61.30
61.50
54.70
13.05
86.90
86.20
MMLU
1,800
76.78
85.50
83.11
6.89
78.94
78.61
Appendix
Table 18: Letter and decoded replies. Each row scores a fixed subset of a benchmark by the option letter and by written replies, and shading deepens along the row from lowest to highest accuracy. Q: questions; Decoded, Thinking: written replies, thinking off and on; Capped: share of thinking replies at the token cap. Table 2 gives Qwen3.5-9B’s decoded replies on full benchmarks.
Figure 9: Manoeuvre and choice on NAVSIM navtest. The most probable candidate takes the driver’s manoeuvre in 89% of the scenes; RL narrows its gap to the best of the six. Top: front camera and NAVSIM map for three of 20 scenes where Qwen-Drive-1.0’s plan [ 1 ] runs into the vehicle ahead while our final RL model’s stays behind it, within 1 m of the human’s at 4 s; both plans as the NAVSIM scorer simulates them, in 0.5-s steps, with all vehicles at 4 s. Bottom, over all 12,146 scenes, for our supervised (SFT) and final RL models: (a) manoeuvre agreement of the most probable candidates; (b) PDMS of the best of the k most probable; (c) share of the scenes and of the total gap between the best of the six and the most probable one, by gap size. Manoeuvres, Qwen-Drive-1.0 runs and scene selection: Appendix F .
Change
PDMS ↑
ADE ↓
km ↑
Pass ↑
Score change (95% interval)
No change
91.30
0.400
1.486
73.4
–
No history frames
90.83
0.397
0.993
65.3
−0.104 ( −0.134 , −0.076 )
No rollout rows
91.35
0.393
0.958
71.3
−0.027 ( −0.058 , +0.002 )
Imitation target
82.03
0.395
1.199
71.0
−0.030 ( −0.052 , −0.010 )
Linear regression
87.82
0.320
1.360
70.3
−0.026 ( −0.055 , +0.004 )
No PDM simulator
–
–
0.929
64.3
−0.090 ( −0.114 , −0.064 )
Appendix
Table 19: One change at a time. Each row changes one thing in the supervised model before its last phase, read at the top-64 mean. Columns: navtest PDMS, PAI-AV 3-s ADE (m) and, in AlpaSim without delay, km per at-fault incident, share passed (%) and the paired change in the mean scenario score with its 95% interval; – : not applicable. Navtest PDMS moves by under a point except with the imitation target or linear regression, while km per at-fault incident falls most without history frames, rollout rows, PAI-AV rows or the PDM simulator. Shading deepens from the worst to the best value in the PDMS, ADE, km and pass columns.
navtest
AlpaSim
RL step
NC ↑
TTC ↑
EP ↑
PDMS ↑
km ↑
From the reward target, current frame only
None
98.48
95.03
85.93
90.83
0.993
Sampled, normalized
97.87
93.45
89.12
91.30
1.103
Exact, normalized
98.93
96.50
85.55
91.60
1.113
Exact
99.11
97.16
83.68
91.16
1.074
Appendix
Table 20: RL step. Sampling with a normalized advantage trades NC and TTC for progress; the exact expectation raises both. Scores are on navtest and in closed loop on the 916 AlpaSim scenarios; km: km per at-fault incident. Shading compares the rows of the first block, from the worst to the best value in each column. RL runs use 2,000 steps and β=0.3 unless noted; “sampled” draws 2 × 32 candidates per scene, and “normalized” divides the advantage by its standard deviation in each scene, as in GRPO [ 31 ] .
Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual question answering and chain-of-thought reasoning data, which emphasizes linguistic reasoning rather than action-grounded planning. As a result, the learned representations capture semantic knowledge but lack spatial dependencies crucial for reliable trajectory prediction. We propose DriveTeach-VLA, a framework that explicitly teaches VLAs what to see and where to look. Driving-aware Vision Distillation (DVD) injects driving-specific perceptual priors into the vision encoder, while 2D Trajectory-Guided Prompts (2D-TGP) provide spatial conditioning aligned with feasible driving trajectories. Together, they form a vision-guided learning pipeline: what to see (DVD pretraining) - where to look (TGP-guided SFT) - how to act (TGP-guided GRPO). DriveTeach-VLA achieves the state-of-the-art performance on NAVSIM and nuScenes. Our code is available at: https://github.com/ShivaTeam/DriveTeach-VLA.
Yuguang Yang, Canyu Chen, Zhewen Tan +10
School of Electronic Information Engineering, Beihang University · Institute for AI Industry Research (AIR), Tsinghua University · National College for Excellent Engineers, Beihang University +5
Vision-Language-Action (VLA) models promise to bring end-to-end reasoning to autonomous driving, but their computational cost remains far too high for real-time control. The core challenge is structural: VLA inference is not a single bottleneck but a cascade of four. Visual encoding wastes compute on overlapping video frames; language-model prefill recomputes context that could be carried over from the previous timestep; reasoning tokens are generated serially despite low entropy; and flow-matching denoising applies uniform compute to a non-uniform velocity field. Addressing any one stage in isolation leaves the others untouched. We propose FlashDrive, an algorithm-system co-design framework that targets all four stages simultaneously. Our key insight is that each bottleneck admits a distinct, lightweight algorithmic shortcut: temporal overlap enables streaming KV-cache reuse across frames; the low per-token entropy and strong intra-block correlations of driving-domain reasoning make a non-autoregressive diffusion drafter highly effective for speculative decoding; and the velocity field's structure---sharp at the endpoints, flat in the middle---permits adaptive step caching that concentrates compute where it matters. Layered on system-level CUDA Graph compilation and kernel fusion, these techniques compound. Applied to Alpamayo 1.5-10B with W4A8 quantization, FlashDrive reduces end-to-end latency from 717ms to 151ms (4.7x) while leaving accuracy essentially unchanged: [email protected] shifts by only 0.08m, minADE1 improves, and closed-loop collision and off-road rates improve in simulation. By raising a 10B-parameter reasoning VLA from 1.4Hz to 6.6Hz on a single GPU, FlashDrive moves end-to-end autonomous driving substantially closer to real-time deployment.
Zekai Li, Yihao Liang, Hongfei Zhang +3
1UC San Diego · 2Princeton · 3Independent Researcher
Driving Vision-Language-Action Models (Driving VLAs) aim to use language to improve end-to-end planning, but the language-action gap limits this promise. We propose DriveMA, a Driving VLA framework built on verifiable meta-actions, which summarize future ego motion into compact language-domain intentions and can be constructed from expert trajectories with a trajectory-grounded annotation pipeline and can be verified against generated trajectories through rule-based projection. DriveMA exploits this verifiability with action-centric supervised training and a data-efficient turn-level credit assignment reinforcement learning framework, explicitly aligning high-level decisions with low-level trajectory planning through dense rewards and precise credit assignment. DriveMA sets a new state of the art on the Waymo Open Dataset Vision-based E2E Driving, achieving a Rater Feedback Score of 8.060 with a 2B model and further improving it to 8.079 with a 4B model; it also obtains competitive closed-loop planning performance on NAVSIM. These results show that even a simple meta-action interface can achieve state-of-the-art planning when made verifiable and optimized for language-action alignment. Code, data, and models are available at https://tsinghua-mars-lab.github.io/DriveMA.
Weicheng Zheng, Yixin Huang, Qiao Sun +2
1Shanghai Qi Zhi Institute · 3Tongji University · 2IIIS Tsinghua University