Dense bird's-eye-view (BEV) construction accounts for 60% of the parameters in a strong recent driving planner. This paper presents LOWP, a planner that looks only where it plans: two sets of sixteen learnable queries read the cameras through the known calibration, one by sampling image features at projected 3D anchors and one by attending to features labeled with their viewing rays, so no dense BEV is built and no object or map is supervised. Evaluations on open-loop nuScenes, pseudo-simulated NAVSIM and closed-loop Bench2Drive show competitive planning with a compact representation. On nuScenes, average L2 changes from 0.302 to 0.296 m with 53% of the dense counterpart's parameters and 1.7 times its speed on the same GPU; the anchor-based reader performs better on turns than the global reader. On NAVSIM, scoring the planner's own trajectories alongside a vocabulary gives 85.9 PDMS without retraining. The closed-loop control configuration scores 65.5 on 218 routes, with a brake-induced standstill failure whose release rule has an inconclusive aggregate benefit. We distinguish measured latency from hardware-transferred estimates, which do not establish a common-hardware ranking.
Figures & tables
Figure 2: LOWP. Left: a ResNet trunk turns the camera images into stride-16 features F , and no dense BEV is built. Two experts plan from F , each a reader of 16 queries (three layers) and a planning head with the ego status. The deformable reader moves one metric anchor per query, lifts it to four heights of eight points, projects them through the calibration and aggregates the samples with a null slot; the global reader cross-attends to every cell of F , tagged with the encoding of its viewing ray. The command mixes the two trajectories, and a light head adapts the output to each benchmark. Right: both readers on a real nuScenes sample (red: the anchors, and where one anchor’s 32 samples land in the front camera; teal: viewing rays, dotted at depths of the ray encoding, which spans 1 to 60 m in eight steps); LiDAR points and vehicles are context only.
Oct. 2
Oct. 18
Model
min
med.
min
med.
Pub.
nuScenes
ResWorld
41.35
57.22
31.29
32.28 †
15.4 FPS ∗
LOWP
24.42
37.67
17.55
17.75 †
LOWP-S (global only)
16.02
22.15
–
–
NAVSIM
Table 1: Direct network latency in milliseconds (minimum and median) on an RTX PRO 6000 Blackwell, eager FP32, batch one, using CUDA events and GPU-resident inputs. Preprocessing, data transfer, PID/recovery and simulator time are excluded. The anchor script uses three interleaved rounds of 50 warm-up and 300 timed forwards; medians are over round medians. † The footprint script uses 20/100 forwards per round and one nuScenes sample instead of 32, so sessions are not a controlled timing ablation. Other inputs are synthetic; nuScenes uses trained checkpoints, and the footprint run also loads Bench2Drive checkpoints. The Oct. 18 log records PyTorch 2.11.0+cu128 and an idle start. Pub.: published anchor speeds on an RTX 4090 ( ∗ H100), separate from the local measurements; their precision, batch and exact timing scope remain unverified.
Figure 3: Descriptive accuracy–speed comparison on (a) nuScenes (average L2, lower is better), (b) NAVSIM (PDMS) and (c) Bench2Drive (driving score). Red squares and bars are LOWP’s transferred estimates and sensitivity ranges; circles use competitors’ published speeds on heterogeneous hardware (area: parameters; open ring: not stated). These are not common-hardware Pareto fronts. Methods without a configuration-matched speed, including FlowAD, remain in the accuracy tables. LOWP-S uses the global reader alone.
nuScenes (open loop)
NAVSIM (pseudo-simulation)
Bench2Drive (closed loop)
Method
FPS ↑
L2 ↓
Par.
Method
FPS ↑
PDMS ↑
Par.
Method
FPS ↑
DS ↑
Par.
LOWP-S (global)
29–42 E
0.309
35
LOWP regr. (ours)
47–86 E
85.18
31
AD-MLP
333
18.05
0
LOWP (ours)
19–26 E
0.296
43
TransFuser
60
84.00
56
CIL++
185
59.53
-
EchoP
22
0.360
35 e
PRIX
57
87.80
37
LOWP (ours)
77–111 E
65.51
34
ResWorld
15
0.300
81
LOWP (ours)
36–60 E
85.89
34
iPad
23
65.02
29
HiP-AD
9.1
0.560
90
DiffusionDrive
45
88.10
60
ETA (async)
20
69.53
-
Table 2: Descriptive comparison with published baselines; further rows appear in Tabs. 7 , 5 and 8 . Unmarked FPS: published on heterogeneous hardware; E : transferred estimates for LOWP (shaded), spanning anchor/statistic choices (and two sessions for NAVSIM), not confidence intervals. Direct measurements are in Tab. 1 . FlowAD, NAVSIM iPad and TCP-traj have no verified configuration-matched speed here (–). These speeds and the unmatched closed-loop protocols do not establish a global Pareto ordering. Par.: parameters in millions ( e estimated); bold: best listed accuracy.
Figure 4: nuScenes against dense ResWorld on four samples where plans differ (titles: average L2 of LOWP and ResWorld, or an evaluator-flagged collision, marked by a cross). Bird’s-eye view shows LiDAR (context only), map, annotated agents, LOWP’s anchors (diamonds; arrows point to off-window anchors) and 3 s plans.
Figure 5: NAVSIM against the official TransFuser baseline (retrained), on four selected scenes where TransFuser scores zero. Top: the camera showing the agent that is hit. Bottom: current agents and their next 4 s (grey; hit agent dashed magenta), expert future (corridor and dashed line) and plans. Insets magnify first contact (cross); footprints follow the plans without the simulator’s tracking controller.
Straight (4,436)
Turn (683)
Reading
AVG
MAX
Col.
AVG
MAX
Col.
Deformable reader alone
0.293
0.576
15
0.414
0.806
7
Global reader alone
0.291
0.585
13
0.438
0.909
24
Mix (LOWP)
0.280
0.553
11
0.401
0.784
7
LOWP-S (global model)
0.290
0.578
13
0.435
0.894
18
ResWorld (dense)
0.287
0.562
14
0.397
0.760
8
Table 3: Which reader carries which regime on nuScenes: the two heads of one trained LOWP alone and mixed, the separately trained global-reader model and the dense ResWorld, split by the driving command. L2 (m) under both conventions; Col.: collision events at 3 s. On turns the deformable head beats the global head by 0.10 m MAX L2 (95% scene-bootstrap interval [0.05,0.16] ; AVG [0.005,0.047] ); on straight driving the two are level.
Component
Metric
Before → after
Gain
Two experts
nuS. L2
0.309 → 0.296
−0.013
Vocab. scorer
PDMS
85.18 → 85.35
+0.17
Own candidates
PDMS
85.35 → 85.89
+0.54[0.35,0.74]
Re-timed cand.
PDMS
85.89 → 85.90
+0.01[−0.03,0.05]
Control config.
B2D DS
60.56 → 65.72
+5.16±2.37
No host syncs
time (ms)
9.44 → 6.09
same output
Table 4: Configuration comparisons (nuS.: nuScenes; B2D: Bench2Drive). Two experts compares separately trained LOWP-S with the two-reader model, not a matched-capacity ablation. NAVSIM gains include 95% scene-bootstrap intervals. B2D uses 217 common routes and mean ± standard error; the control model also receives additional training ( Tab. 6 ). Host-sync timing is the NAVSIM scorer, with identical outputs.
Method
NC
DAC
TTC
Comf.
EP
PDMS
Ego-status MLP (NeurIPS2024)
93.0
77.3
83.6
100.0
62.8
65.60
VADv2 (ICLR2026)
97.2
89.1
91.6
100.0
76.0
80.90
UniAD (CVPR2023)
97.8
91.9
92.9
100.0
78.8
83.40
LTF (TPAMI2023)
97.4
92.8
92.4
100.0
79.0
83.80
TransFuser † (TPAMI2023)
97.7
92.8
92.8
100.0
79.2
84.00
Hydra-MDP † (arXiv2024)
98.3
96.0
94.6
100.0
78.7
86.50
Table 5: NAVSIM navtest sub-scores and PDMS. † camera and LiDAR; every LOWP row is camera-only. Shaded: LOWP; bold: best per column.
Controller
Reverse
Release
Routes
DS
PID
off
off
218
59.01
PID
on
off
217
60.56
PID + control
on
off
218
65.51
PID + control
on
on
216
67.10
Table 6: Bench2Drive controller and recovery configurations, one seed. Reverse: after 3 s under throttle without motion, reverse for 1 s, at most three times per route. Release: after 20 s stopped, temporarily use PID longitudinal control when the head brakes against a moving plan. Paired DS changes use common routes: reverse +1.30 (217); control configuration +5.16±2.37 (217); release +1.59±2.06 (216), mean ± standard error where recorded. The control configuration is additionally trained; the release run also changes simulator process placement. These comparisons do not isolate either change from run variability.
Figure 6: Bench2Drive route 152 against TCP-traj (released weights in our harness), from selected traced runs, each row at its own instants. The ego is outlined; bands show driven paths and lines show plans (bars mark stationary plans). Simulated LiDAR is context only. Grey paths show other vehicles, and crosses mark contact. LOWP stops about 5 m behind a braking car and resumes when the queue moves; TCP-traj drifts left and rear-ends a stopped car (17.2 s).
L2 (m)
CR (%)
Method
1s
2s
3s
Avg
1s
2s
3s
Avg
ST-P3
1.33
2.11
2.90
2.110
0.23
0.62
1.27
0.710
UniAD-Base
0.44
0.67
0.96
0.690
0.04
0.08
0.23
0.120
VAD-Base
0.17
0.34
0.60
0.370
0.07
0.10
0.24
0.140
BEV-Planner
0.16
0.32
0.57
0.350
0.00
0.29
0.73
0.340
PARA-Drive
0.25
0.46
0.74
0.480
0.14
0.23
0.39
0.250
Table 7: nuScenes, temporal-average protocol with ego status: L2 and collision rate (CR) at 1, 2 and 3 s and their average. Shaded: LOWP; bold: best per column. FlowAD reports only the averages.
Method
DS
SR (%)
Effic.
Comf.
AD-MLP (arXiv2023)
18.05
0.00
48.45
22.63
UniAD-Tiny (CVPR2023)
40.73
13.18
123.92
47.04
UniAD-Base (CVPR2023)
45.81
16.36
129.21
43.58
VAD (ICCV2023)
42.35
15.00
157.94
46.01
TCP (NeurIPS2022)
40.70
15.00
54.26
47.80
TCP-ctrl (NeurIPS2022)
30.47
7.27
55.97
51.51
Table 8: Bench2Drive, base split unless marked: driving score, success rate, efficiency and comfortness. External rows use their published protocols; shaded LOWP rows use 217 recorded routes for PID and 218 with the control branch, both with standstill reversal ( Tab. 6 ). These are not matched-harness comparisons. ‡ full 2 M-frame training set. LOWP efficiency and comfortness are unavailable (-); bold: best listed value per column.
End-to-end driving requires planning-relevant bird's-eye-view (BEV) representations, but existing pretraining approaches often rely on task annotations or dense scene reconstruction. We introduce PAVER, Planning-Aligned BEV Encoder Pretraining. From a single LiDAR sweep, PAVER constructs sparse risk and unknown targets describing occupied and unobserved evidence along rule-based ego motions. A 10K-parameter head predicts these targets from masked BEV features conditioned on the action state, directing supervision toward geometric constraints on candidate motions. Pretraining requires no driving-task annotations or dense reconstruction. Only the BEV encoder is transferred, preserving the downstream architecture and camera-only inference. On nuScenes, PAVER reduces VAD-Tiny's average collision rate from 0.51% to 0.19%, while improving planning L2, motion prediction, detection, and mapping. The selected VAD-Tiny and VAD-Base schedules use about 36% less estimated total training time than scratch training, including pretraining. On Bench2Drive Town05 Long, PAVER improves UniAD-Tiny's closed-loop Driving Score from 48.45 to 58.79. The project page is available at https://archiiive99.github.io/PAVER.
We present a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles. The model is conditioned on a bird's-eye-view (BEV) raster of the surrounding scene and generates control sequences in a small number of Ordinary Differential Equations (ODE) integration steps, enabling low-latency inference suitable for real-time closed-loop re-planning. We train exclusively on urban scenarios (real urban city streets, intersections and roundabouts of the city of Parma, Italy) collected from a 2D traffic simulator with reactive agents, and evaluate in closed-loop on both in-distribution and markedly out-of-distribution environments, including multi-lane highways and unseen urban scenarios. Our results show that the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution. We attribute this to the BEV representation, which provides a geometry-centric view of the scene that is inherently less sensitive to distributional shifts, and to the flow-matching formulation, which learns a smooth vector field that degrades gracefully under distribution shift. We provide video demonstrations of closed-loop behavior at https://marcelloceresini.github.io/DirectControlFlowMatching.
Marcello Ceresini, Federico Pirazzoli, Andrea Bertogalli +5
1Università degli Studi di Parma · 2Alma Mater Studiorum - Università di Bologna · 3VisLab (an Ambarella Inc. company)
End-to-end planners based on waypoint regression achieve strong open-loop accuracy, but they primarily learn to mimic expert geometry and remain difficult to adapt to deployment-time safety constraints. We propose a query-based cost-learning framework that estimates bounded costs for dynamically reachable ego trajectory queries, rather than dense BEV cells or a small regressed trajectory set. Compact joint scene tokens capture coherent multimodal agent futures, while contingency-aware cost aggregation and cost-guided intra-cluster MPPI mixing convert the learned cost topology into feasible ego plans. On nuScenes, our method improves over prior cost-estimation planners such as ST-P3 and NMP, outperforms most regression baselines in collision rate, while remaining competitive in L2, and retaining an interpretable cost interface. On real-world driving logs, the proposed planner reduces collision rates compared with SparseDrive and Alpamayo without fine-tuning, while maintaining a diverse set of candidate trajectories.
Ahmed Abouelazm, Rupert Polley, Qingyuan Zhang +4
FZI Research Center for Information Technology · Karlsruhe Institute of Technology · CARIAD SE