SuperGlance: Driving at a Glance with a Vision-Language Model
Organizations: Institute for AI Industry Research (AIR), Tsinghua University · HiThink Research · The Hong Kong University of Science and Technology (Guangzhou) · Agency for Science, Technology and Research (A*STAR), Singapore ·
Abstract
Autonomous driving is turning to vision-language-action models (VLAs). They answer and plan by decoding, one step at a time. Each step is another forward pass, so a plan can take over a second of GPU time. Embodied AI often pairs such a slow, reasoning System 2 with a fast System 1 that acts. Human drivers plan at a glance, with System 1. Faster designs usually put System 1 outside the vision-language model (VLM), in a planner without its abilities. Yet before a VLM writes its first token, it has already read the scene into one hidden state. We show that, once trained, this hidden state is enough to drive and to answer questions about what the model sees. SuperGlance reads it to turn a decoder-only VLM into an encoder-style driving policy. Its linear head scores a fixed set of 5,273 candidate trajectories. With the set fixed, rewards are computed offline, so supervised targets can be reward-weighted and reinforcement learning maximizes the exact expected reward. Before and during policy training, multiple-choice training keeps the VLM's visual abilities in one pass, so the policy still answers questions about driving scenes. Training questions are built against shortcuts. Each must be answered right in two option orders. Against drift, a KL term keeps each stage near its starting model. One pass offers VLAs a more plausible path to System 1. SuperGlance answers visual multiple-choice questions as well as its base VLM does by decoding. It plans in 84 ms of GPU time, six times faster than Qwen-Drive-1.0, and at least as well on NAVSIM, PAI-AV and WOD-E2E. In AlpaSim closed loop, km per at-fault incident holds for both planners up to a 300-ms delay and drops beyond. With GPU time charged as delay, SuperGlance's lead over Qwen-Drive-1.0 grows from 2.8 to 3.9 times. Delayed by 1.2 s, our model falls below Qwen-Drive-1.0 at its own 518 ms.
Figures & tables
| Planning | Sequential | Planner | Keeps | RL | |
| Method | paradigm | steps | in VLM | language | objective |
| Decoding or denoising VLAs | |||||
| AutoVLA [ 2 ] | Autoregressive | 10 + text | ✓ | ✓ | GRPO |
| Qwen-Drive-1.0 [ 1 ] | Flow matching | 11 + text | ✓ | GRPO | |
| One-pass or fast VLM planners | |||||
| SimLingo [ 10 ] | Regression | 1 + text | ✓ | ✓ | |
| Qwen3.5-9B | SuperGlance after stage | |||||
|---|---|---|---|---|---|---|
| Benchmark | letter | decoded | Qwen-Drive-1.0 | 1 | 2 | 3 |
| Vision | ||||||
| CV-Bench [ 64 ] | 86.36 | 86.47 | 85.29 | 87.60 | 87.66 | 87.40 |
| BLINK [ 65 ] | 79.37 | 76.62 | 77.60 | 84.68 | 81.34 | 79.37 |
| 3DSRBench [ 66 ] | 45.39 | 57.09 | 46.81 | 60.27 | 60.50 | 58.29 |
| Language | ||||||
| Method | RL | Coll. | Off-road | Prog. | km / incident | Pass |
|---|---|---|---|---|---|---|
| Each plan’s GPU time as delay | ||||||
| Qwen-Drive-1.0, 518 ms | 32.6 / 18.1 | 19.3 | 67.9 | 0.229 / 0.317 | 36.6 | |
| + reasoning, 1.2 s | 26.7 / 21.0 | 36.3 | 51.8 | 0.145 / 0.158 | 9.3 | |
| SuperGlance, 84 ms | ✓ | 10.9 / 6.8 | 7.5 | 88.5 | 0.970 / 1.248 | 72.2 |
| No delay (simulator default) | ||||||
| Qwen-Drive-1.0 [ 1 ] | 32.8 / 14.4 | 15.4 | 71.1 | 0.284 / 0.461 | 58.7 | |
| Method | NC | DAC | EP | TTC | Comf. | PDMS |
|---|---|---|---|---|---|---|
| Supervised | ||||||
| TransFuser [ 76 ] | 97.7 | 92.8 | 79.2 | 92.8 | 100.0 | 84.0 |
| DRAMA [ 77 ] | 98.0 | 93.1 | 80.1 | 94.8 | 100.0 | 85.5 |
| Hydra-MDP [ 33 ] | 98.3 | 96.0 | 78.7 | 94.6 | 100.0 | 86.5 |
| DiffusionDrive [ 78 ] | 98.2 | 96.2 | 82.2 | 94.7 | 100.0 | 88.1 |
| Epona [ 18 ] | 97.9 | 95.1 | 80.4 | 93.8 | 99.9 | 86.2 |
| Method | RL | 3-s ADE | 5-s ADE | RFS |
|---|---|---|---|---|
| MindVLA-U1 [ 80 ] | 1.16 | 2.67 | 7.77 | |
| AutoVLA [ 2 ] | ✓ | 1.35 | 2.96 | 7.56 |
| MindVLA-U1 | ✓ | 1.09 | 2.66 | 7.87 |
| Qwen-Drive-1.0 + reasoning [ 1 ] | 1.19 | 2.65 | 7.78 | |
| Qwen-Drive-1.0 + reasoning | ✓ | 1.19 | 2.67 | 7.91 |
| SuperGlance | ✓ |
| One plan | Best of six | ||||
| Method | RL | 3 s | 5 s | 3 s | 5 s |
| DriveWAM [ 20 ] † | 0.69 | – | 0.38 | – | |
| Qwen-Drive-1.0 + reasoning [ 1 ] | 0.414 | 1.205 | 0.385 | 1.095 | |
| Qwen-Drive-1.0 + reasoning | ✓ | 0.454 | 1.231 | 0.415 | 1.102 |
| SuperGlance | 0.394 | 1.109 | 0.284 | 0.821 | |
| SuperGlance | ✓ | 0.373 | 1.076 | 0.271 | 0.777 |
| Method | SemSISR | SemCFPS | SCS | PCS |
|---|---|---|---|---|
| Qwen-Drive-1.0 (4B) [ 1 ] | 15.8 | 0.0 | 25.3 | 2.8 |
| AutoVLA [ 2 ] | 18.8 | 0.0 | 30.2 | 5.4 |
| DriveMA-2B [ 81 ] | 20.4 | 0.0 | 30.6 | 6.7 |
| ReCogDrive [ 5 ] | 13.2 | 0.0 | 22.5 | 2.8 |
| OneVL [ 82 ] | 20.7 | 0.0 | 30.5 | 4.6 |
| SuperGlance, no instruction | 11.8 | 0.0 | 26.2 | 6.2 |
| Open loop | Closed loop (AlpaSim) | |||
|---|---|---|---|---|
| Output | Target | PDMS | km / incident | Pass (%) |
| Text, decoded | Imitation | – | – | |
| Linear regression | Imitation | 1.360 | 70.3 | |
| Policy Head | Imitation | 1.199 | 71.0 | |
| Policy Head | Reward | 1.486 | 73.4 | |
| Qwen-Drive-1.0 + reasoning [ 1 ] | 88.2 | 0.461 | 58.7 | |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Questions about | Options | Questions | Drawn (%) |
| Stage 1: 1,158,150 questions, 500,000 drawn | ||||
| GQA [ 52 ] | Relations, attributes and objects (yes/no); colour, material, category | Yes/no, or the gold and 3 others of its kind | 435,825 | 19.91 |
| flipped | Spatial relations, turned into their opposite | Yes/no; the gold is ‘no’ | 12,158 | |
| SAT [ 53 ] | Spatial relations, counts, action consequences, goals, perspective | SAT’s own: 2 for relations, 5 for counts | 139,632 | 19.91 |
| TallyQA [ 54 ] | Counts | 5 consecutive integers around the gold | 119,859 | 16.59 |
| ChartQA [ 56 ] | Chart values; yes/no | 5 values or 5 years around the gold | 22,136 | 5.31 |
| Source | Rows | Drawn (%) | Target or reward | Rows from |
| Supervised: 2,191,689 rows, 20,900 steps of 128 rows, 5,121 candidates | ||||
| NAVSIM scenes | 92,339 | 12.17 | One per navtrain scene of 1,083 logs | |
| NAVSIM frames | 44,243 | 5.83 | Other frames of 1,076 of these logs that pass the navtrain filter | |
| PAI-AV logged | 1,044,771 | 36 | 149,253 training clips, every 2 s | |
| PAI-AV rollouts | 46,583 | 4.5 | As above, with the ADE to a recovery trajectory | AlpaSim rollouts of earlier models in 492 scenes |
| WOD-E2E | 406,205 | 18 | Every 10-Hz frame of 1,991 training segments | |
| NAVSIM | PAI-AV | WOD-E2E | |
|---|---|---|---|
| Camera | nuPlan’s [ 102 ] front camera, CAM_F0, as logged | The 120 ∘ f-theta front-wide camera, re-projected into CAM_F0 | FRONT, FRONT_LEFT and FRONT_RIGHT, re-projected into CAM_F0 by rotation, each pixel from the camera with the nearest optical axis |
| Current frame | 1920 1080 with CAM_F0’s intrinsics and distortion; re-projected frames are saved at JPEG quality 92. The processor resizes the frame to 1024 576, its limit of 589,824 pixels, which gives 576 visual tokens. | ||
| Past frames | 1.5, 1.0 and 0.5 s earlier, from the 2-Hz log | The same times, nearest frame within 0.1 s; clips with an even hash, about half | None |
| 556 313 (at most 174,080 pixels), 544 320 in the processor, 170 visual tokens each. Rows without all three, rollout rows and question rows stay single-frame; 237,246 of the 391,322 RL rows have past frames. | |||
| Sampling | One row per navtrain scene (4 past and 10 future frames, a route), and other frames of the same logs that pass this filter | Every 2 s from 1.9 s after a clip’s start to 4.4 s before its end; RL: up to 2 per clip | Every 10-Hz frame; 46 segments held out by hash |
| Speed and acceleration | Current ego status | Egomotion at about 10 Hz, central differences over 0.2 s | Last past state, at 4 Hz |
| Stage 1 | Stage 2 | Stage 3, supervised | Last phase | RL | |
|---|---|---|---|---|---|
| Starts from | Qwen3.5-9B, new LoRA | Stage-1 model, merged; new LoRA | Stage-2 model, merged; new LoRA; Policy Head at zero | The supervised model | The last-phase model; Policy Head refit to 5,273 rows |
| Trainable | 80,216,064 | 80,216,064 | 80,216,064 + 20,980,737 | 80,216,064 + 20,980,737 | 80,216,064 + 21,603,481 |
| LoRA | Rank 32, 64, dropout 0, on 200 projections: q, k, v and o of the 8 full-attention layers; in_proj_qkv, in_proj_z and out_proj of the 24 Gated DeltaNet layers; gate, up and down of all 32 MLPs. The vision tower, the embeddings and the language-model head stay frozen. | ||||
| Objective | Paired Brier, 32 draws per option order | As stage 1 | Cross-entropy to ; questions as in stage 2 | As stage 3 | Exact expected reward |
| KL term | 0.1 to Qwen3.5-9B | 0.1 to the stage-1 model | 0.1 to the stage-2 model, on question rows | As stage 3 | to |
| Batch | 64 | 64 | 128 | 128 | 64 |
| Candidate set | Spacing (m) | Best PDMS | Nearest PDMS | |
|---|---|---|---|---|
| k-means 256 + stationary | 257 | 0.98 | 99.21 | 77.88 |
| k-means 1,024 + stationary | 1,025 | 0.61 | 99.66 | 82.76 |
| k-means 4,096 + stationary | 4,097 | 0.40 | 99.84 | 85.69 |
| Benchmark | What it is | Loop | Split and size | Metrics | Horizon |
|---|---|---|---|---|---|
| NAVSIM [ 50 ] v1.1 | Scenes of nuPlan [ 102 ] logs; the PDM scorer simulates each plan | Open | navtest: 12,146 scenes from 136 logs | PDMS: NC DAC a weighted sum of EP, TTC and Comf. | 4 s at 2 Hz |
| WOD-E2E [ 59 ] | Long-tail scenes; RFS scores a plan against rated trajectories | Open | Test split, scored by the official server; the 479 rated validation frames provide RL rewards | RFS; 3- and 5-s ADE | 5 s at 4 Hz |
| PAI-AV [ 58 ] | Driving clips with released egomotion | Open | Qwen-Drive-1.0’s leakage-free subset: 700 frames, five in each of 140 held-out test clips | 3- and 5-s ADE of one plan and of the best of six | 5 s at 10 Hz |
| AlpaSim [ 13 ] | Simulator that renders the camera along the car’s own path in PAI-AV-NuRec [ 74 , 75 ] v26.01 scenes | Closed | 916 scenarios; one whose route check fails counts as failed, with score zero | Collision and off-road rates, progress, km per at-fault incident, pass rate and mean scenario score | Whole scenario, a call every 100 ms |
| Language2Drive [ 83 ] | Two instructions per scene; checks whether each plan follows its own (NAVSIM track) | Open | 38 instruction pairs in four wordings each | SemSISR, SemCFPS: single instructions and pairs followed; SCS, PCS: graded scores | 4 s |
| Method | Sequential passes | GPU (ms) | End to end (ms) |
|---|---|---|---|
| SuperGlance | 1 | 83.8 | 145.5 |
| ReCogDrive [ 5 ] | 6 | 198.6 | 296.5 |
| Qwen-Drive-1.0 [ 1 ] | 11 | 518.3 | 1,014.1 |
| + reasoning | 28 | 1,213.2 | 1,710.1 |
| AutoVLA [ 2 ] | 45 | 1,529.3 | 1,900.3 |
| Text waypoints | 155 | 5,557.7 | 5,622.4 |
| Model | Collision (%) | Off-road (%) | Progress (%) | km / incident | Mean score | Pass (%) |
|---|---|---|---|---|---|---|
| Qwen-Drive-1.0 + reasoning | 32.8 / 14.4 | 15.4 | 71.1 | 0.284 / 0.461 | 0.516 | 58.7 |
| Qwen-Drive-1.0 + reasoning, RL | 43.6 / 15.6 | 13.1 | 57.3 | 0.194 / 0.383 | 0.391 | 53.6 |
| Imitation, current frame | 15.4 / 9.1 | 9.8 | 84.5 | 0.684 / 0.914 | 0.603 | 64.4 |
| Imitation, history frames | 10.3 / 5.1 | 9.8 | 87.4 | 0.888 / 1.199 | 0.682 | 71.0 |
| Reward target, current frame | 15.1 / 8.1 | 9.3 | 84.5 | 0.708 / 0.993 | 0.608 | 65.3 |
| Reward target, current frame, extended set | 15.3 / 8.1 | 8.9 | 84.5 | 0.714 / 1.018 | 0.611 | 65.6 |
| Qwen3.5-9B | Final model | ||||||
| Benchmark | Q | Letter | Decoded | Thinking | Capped (%) | Letter | Decoded |
| CV-Bench | 1,320 | 87.68 | 87.05 | 85.36 | 6.36 | 88.55 | 88.61 |
| BLINK | 509 | 79.37 | 76.62 | 75.44 | 10.41 | 79.37 | 79.17 |
| 3DSRBench | 1,000 | 43.50 | 58.20 | 55.00 | 9.17 | 58.10 | 57.60 |
| DriveLM | 1,000 | 61.30 | 61.50 | 54.70 | 13.05 | 86.90 | 86.20 |
| MMLU | 1,800 | 76.78 | 85.50 | 83.11 | 6.89 | 78.94 | 78.61 |
| Change | PDMS | ADE | km | Pass | Score change (95% interval) |
|---|---|---|---|---|---|
| No change | 91.30 | 0.400 | 1.486 | 73.4 | – |
| No history frames | 90.83 | 0.397 | 0.993 | 65.3 | ( , ) |
| No rollout rows | 91.35 | 0.393 | 0.958 | 71.3 | ( , ) |
| Imitation target | 82.03 | 0.395 | 1.199 | 71.0 | ( , ) |
| Linear regression | 87.82 | 0.320 | 1.360 | 70.3 | ( , ) |
| No PDM simulator | – | – | 0.929 | 64.3 | ( , ) |
| navtest | AlpaSim | ||||
| RL step | NC | TTC | EP | PDMS | km |
| From the reward target, current frame only | |||||
| None | 98.48 | 95.03 | 85.93 | 90.83 | 0.993 |
| Sampled, normalized | 97.87 | 93.45 | 89.12 | 91.30 | 1.103 |
| Exact, normalized | 98.93 | 96.50 | 85.55 | 91.60 | 1.113 |
| Exact | 99.11 | 97.16 | 83.68 | 91.16 | 1.074 |
VLMs for Autonomous Driving
Vision-language models for autonomous driving connect camera and sensor data with language to interpret scenes, predict hazards, and plan actions.
See all 221 papers