Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning. State-of-the-art models have achieved remarkable accuracy in such prediction tasks. Besides accuracy, diversity is also crucial for safe planning because human behaviors are inherently uncertain and multimodal. However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes. Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance sampling-based generative models.
Figures & tables
Fig. 1: Motivation for controllable prediction. The prediction that is most likely across population is not always the one that materializes. A predictor whose latent variable controls an interpretable attribute (illustrated as driving style; our experiments control average speed, a coarse proxy) lets the planner reason about each variant explicitly.
Fig. 2: The two capabilities PrefCVAE targets. (1) Semantic encoding: the posterior encoder maps a trajectory to a concentrated latent that orders it by a predefined attribute m(⋅) given the context. (2) Controllable generation: assigning latent values produces predictions whose attribute is ordered accordingly (line colors match the latent samples on the left; ≻ : is preferred to, i.e., has the smaller metric).
Fig. 3: (a) Controllable predictions from the PrefCVAE decoder on a test scene. Columns: base model, PrefCVAE controlling all agents, and PrefCVAE controlling only one agent (pink) while the latents of the others are drawn once and held fixed. Rows: predictions colored by the assigned zS ; average speed versus zS (dashed lines: ground truth); displacement error of the first sample versus zS (solid: 12× the root-mean-square error over the 12 predicted steps; dashed: final displacement error). With PrefCVAE ( ν=0.125 ), a larger zS yields a higher average speed, and the error is lowest near the latent value whose speed matches the ground truth. (b) Posterior distributions of zS encoded from test-set predictions generated with nine assigned values zS∈{0.1,…,0.9} (one color each). Ideally they are nine narrow peaks at the assigned values.
Use rate ν
0 (Base)
0.125
0.5
minADE5 (m)
2.62
2.66
2.64
minFDE5 (m)
5.62
5.76
5.74
Speed min/max (m/s)
6.32/6.34
3.74/8.76
3.26/9.72
AW VR (%)
82.28
0.10
0.12
TABLE I: Accuracy (latent sampled from the prior), speed range over the nine assigned values of zS , and agent-wise VR of the base model ( M=32 , learned prior) and of PrefCVAE ( M=2 , fixed prior) with different use rates ν (best of three seeds).
Base
ν =0.125
ν =0.5
Avg. JSD
0.0352
0.4874
0.4578
logL
3.19
13.23
11.76
Avg. mean deviation
0.1660
0.0090
0.0163
TABLE II: Posterior encoding of the assigned values of zS (base: M=32 ; PrefCVAE: M=2 ). Avg. JSD: mean over the 36 pairs of the nine fitted distributions; logL at the assigned values (both higher is better); deviation of the fitted mean from the assigned value (lower is better).
Fig. 4: Violation rates (solid: mean over three seeds; shaded: one standard deviation; dashed: best seed) and best-seed minADE5 / minFDE5 (right axis) versus use rate ν ( λ=16 , M=2 ).
SW VR
MBW VR
AW VR
ν
λ =8
λ =32
λ =8
λ =32
λ =8
λ =32
0.025
91.30
96.38
64.56
64.11
45.99
38.70
0.050
93.48
10.14
65.08
0.62
46.09
0.21
0.075
91.30
13.04
63.30
1.07
44.62
0.38
0.100
95.65
11.59
77.44
0.94
56.89
0.32
0.125
38.41
15.22
6.05
0.94
2.18
0.32
TABLE III: Violation rates (%, best of three seeds) for weights λ=8 and 32 at small use rates.
M=8
M=16
M=32
λ =16
λ =32
λ =16
λ =32
λ =16
λ =32
SW VR (%)
3.62
0.72
5.07
0
0.72
5.80
MBW VR (%)
0.20
0.03
0.23
0
0.03
0.26
AW VR (%)
0.07
0.01
0.08
0
0.01
0.09
Speed min (m/s)
3.48
3.10
3.48
3.44
3.60
3.36
Speed max (m/s)
10.06
12.00
10.16
11.54
9.92
9.82
TABLE IV: Effect of the latent dimension M on violation rate, speed range, and accuracy (best of three seeds, ν=0.125 ).
Diffusion models for multi-agent trajectory prediction are limited by iterative denoising, which causes inference latency that hinders their use in time-critical settings like autonomous driving. Fast-sampling variants using DDIM and informed initial noise distributions partially alleviate this issue, but they either fail to achieve true single-step generation or are constrained by the chosen noise distribution. Consistency Models (CMs) offer high-quality one-step generation by mapping noise directly to data, but are difficult to train from scratch. We propose ECTraj, an enhanced CM pipeline with improved training and conditional generation for trajectory prediction. Our framework extends the student-teacher consistency training scheme: the student produces standard outputs, while the teacher explicitly fuses its predictions with parts of the ground truth to give stronger supervision. We also exploit CMs' direct denoising for top-K multi-shot generation during training. Combining conditional generation with this enhanced consistency objective yields faster inference and improved prediction accuracy, establishing competitive new benchmarks on the large-scale Argoverse 2 dataset.
Alen Mrdovic, Qingze, Liu +6
Tony · Rutgers University, New Brunswick · The College of New Jersey
Accurate and uncertainty-aware trajectory prediction remains a core challenge for autonomous driving, driven by complex multi-agent interactions, diverse scene contexts and the inherently stochastic nature of future motion. Diffusion-based generative models have recently shown strong potential for capturing multimodal futures, yet existing approaches such as cVMD suffer from slow sampling, limited exploitation of generative diversity and brittle scenario encodings. This work introduces cVMDx, an enhanced diffusion-based trajectory prediction framework that improves efficiency, robustness and multimodal predictive capability. Through DDIM sampling, cVMDx achieves up to a 100x reduction in inference time, enabling practical multi-sample generation for uncertainty estimation. A fitted Gaussian Mixture Model further provides tractable multimodal predictions from the generated trajectories. In addition, a CVQ-VAE variant is evaluated for scenario encoding. Experiments on the publicly available highD dataset show that cVMDx achieves higher accuracy and significantly improved efficiency over cVMD, enabling fully stochastic, multimodal trajectory prediction.
Marion Neumeier, Niklas Roßberg, Michael Botsch +1
Autoregressive VLA planning usually predicts continuous trajectories as the final output, but driving instructions often describe actions rather than ego states, which weakens direct conditioning on the final continuous trajectory. We address this gap with TrajCond, which introduces a discrete trajectory-geometric condition for VLA planning. Specifically, we represent TrajCond as a 1D-Brohan differential action trajectory and learn it with a two-stage training framework before final continuous trajectory prediction. With the simultaneous autoregressive generation of discrete and numeric trajectories, we name our method AutoVLA-D (Double). Experiments on NAVSIM navtest, nuScenes, and NeuroNCAP show that discrete trajectories are weaker as direct outputs but more effective as TrajCond, consistently improving final continuous trajectory prediction across closed-loop and open-loop driving benchmarks. Code is available at https://github.com/ShivaTeam/AutoVLA-D.
Yuguang Yang, Zhewen Tan, Canyu Chen +9
School of Electronic and Information Engineering, Beihang University · Institute for AI Industry Research (AIR), Tsinghua University · School of Computer Science and Engineering, Beihang University +6