Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning. State-of-the-art models have achieved remarkable accuracy in such prediction tasks. Besides accuracy, diversity is also crucial for safe planning because human behaviors are inherently uncertain and multimodal. However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes. Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance sampling-based generative models.
Figures & tables
Fig. 1: Motivation for controllable prediction. The prediction that is most likely across population is not always the one that materializes. A predictor whose latent variable controls an interpretable attribute (illustrated as driving style; our experiments control average speed, a coarse proxy) lets the planner reason about each variant explicitly.
Fig. 2: The two capabilities PrefCVAE targets. (1) Semantic encoding: the posterior encoder maps a trajectory to a concentrated latent that orders it by a predefined attribute m(⋅) given the context. (2) Controllable generation: assigning latent values produces predictions whose attribute is ordered accordingly (line colors match the latent samples on the left; ≻ : is preferred to, i.e., has the smaller metric).
Fig. 3: (a) Controllable predictions from the PrefCVAE decoder on a test scene. Columns: base model, PrefCVAE controlling all agents, and PrefCVAE controlling only one agent (pink) while the latents of the others are drawn once and held fixed. Rows: predictions colored by the assigned zS ; average speed versus zS (dashed lines: ground truth); displacement error of the first sample versus zS (solid: 12× the root-mean-square error over the 12 predicted steps; dashed: final displacement error). With PrefCVAE ( ν=0.125 ), a larger zS yields a higher average speed, and the error is lowest near the latent value whose speed matches the ground truth. (b) Posterior distributions of zS encoded from test-set predictions generated with nine assigned values zS∈{0.1,…,0.9} (one color each). Ideally they are nine narrow peaks at the assigned values.
Use rate ν
0 (Base)
0.125
0.5
minADE5 (m)
2.62
2.66
2.64
minFDE5 (m)
5.62
5.76
5.74
Speed min/max (m/s)
6.32/6.34
3.74/8.76
3.26/9.72
AW VR (%)
82.28
0.10
0.12
TABLE I: Accuracy (latent sampled from the prior), speed range over the nine assigned values of zS , and agent-wise VR of the base model ( M=32 , learned prior) and of PrefCVAE ( M=2 , fixed prior) with different use rates ν (best of three seeds).
Base
ν =0.125
ν =0.5
Avg. JSD
0.0352
0.4874
0.4578
logL
3.19
13.23
11.76
Avg. mean deviation
0.1660
0.0090
0.0163
TABLE II: Posterior encoding of the assigned values of zS (base: M=32 ; PrefCVAE: M=2 ). Avg. JSD: mean over the 36 pairs of the nine fitted distributions; logL at the assigned values (both higher is better); deviation of the fitted mean from the assigned value (lower is better).
Fig. 4: Violation rates (solid: mean over three seeds; shaded: one standard deviation; dashed: best seed) and best-seed minADE5 / minFDE5 (right axis) versus use rate ν ( λ=16 , M=2 ).
SW VR
MBW VR
AW VR
ν
λ =8
λ =32
λ =8
λ =32
λ =8
λ =32
0.025
91.30
96.38
64.56
64.11
45.99
38.70
0.050
93.48
10.14
65.08
0.62
46.09
0.21
0.075
91.30
13.04
63.30
1.07
44.62
0.38
0.100
95.65
11.59
77.44
0.94
56.89
0.32
0.125
38.41
15.22
6.05
0.94
2.18
0.32
TABLE III: Violation rates (%, best of three seeds) for weights λ=8 and 32 at small use rates.
M=8
M=16
M=32
λ =16
λ =32
λ =16
λ =32
λ =16
λ =32
SW VR (%)
3.62
0.72
5.07
0
0.72
5.80
MBW VR (%)
0.20
0.03
0.23
0
0.03
0.26
AW VR (%)
0.07
0.01
0.08
0
0.01
0.09
Speed min (m/s)
3.48
3.10
3.48
3.44
3.60
3.36
Speed max (m/s)
10.06
12.00
10.16
11.54
9.92
9.82
TABLE IV: Effect of the latent dimension M on violation rate, speed range, and accuracy (best of three seeds, ν=0.125 ).
School of Electronic and Information Engineering, Beihang University · Institute for AI Industry Research (AIR), Tsinghua University · School of Computer Science and Engineering, Beihang University +6