Learning robust driving policies that generalize beyond their training domain remains a central challenge in motion planning. We propose an uncertainty-driven self-play scheme, in which half of the agents are rewarded for increasing their neighbors' epistemic uncertainty, while the other half learn to drive safely as the exploring agents continuously seek out their weaknesses. Epistemic uncertainty indicates where the policy is least confident and serves as an automatic proxy for scenario difficulty. Exploring agents use this signal to seek out hard cases, generating a challenging curriculum without manual scenario design. The resulting policy outperforms standard self-play, achieving best-in-class zero-shot transfer to nuplan Val14 (99.12% vs. 97.97%) and Argoverse 2 (82.39% vs. 78.03%) without fine-tuning, while also improving performance over standard self-play on Waymo. Our diversity-driven approach achieves 85% higher LSTM hidden-state coverage than the baseline, indicating a much richer set of behaviors. Further analysis shows that epistemic uncertainty serves as a calibrated difficulty detector, with highest-uncertain agents exhibiting 14.8 times the combined failure rate across collision, off-road, and DNF, of lowest-uncertain agents, enabling automatic prioritization of challenging scenarios without ground-truth labels.
Figures & tables
Fig. 1 : Probing agents (red), assigned the role in each episode, are rewarded for increasing the epistemic uncertainty of their nearest neighbors, while neutral agents (blue) receive only the task reward. As neutral agents adapt, probing agents must discover increasingly novel confusing situations, generating an automatic diversity-driven curriculum. At evaluation, all agents use the neutral role, benefiting from the broader training distribution.
Fig. 2 : Architecture overview. Observations are processed by ego, partner, and road encoders with max-pooling over variable-count entities. The concatenated embedding passes through an LSTM with MC-Dropout and is modulated by FiLM conditioning on a scalar role signal (probing or neutral). At evaluation, all agents use the neutral role.
Method
Conditioning Mechanism
Score (%) ↑
Goal (%) ↑
Coll. (%) ↓
Offroad (%) ↓
DNF (%) ↓
Baseline (PPO)
None (unconditional self-play)
99.29 ± 0.08
99.64 ± 0.04
1.92 ± 0.19
0.87 ± 0.14
0.24 ± 0.01
UncArms (Ours)
Scalar role signal ( Δ unc arms-race)
99.41 ± 0.08 †
99.72 ± 0.02 †
2.25 ± 0.17
1.13 ± 0.09
0.20 ± 0.02
GIGAFLOW ∗
Task-weight vector (3D: goal, coll, off)
94.70 ± 0.70 †
96.60 ± 0.50 †
4.60 ± 0.40
4.30 ± 0.70
1.81 ± 0.31
DIAYN
Discrete skill index ( z∈{1..5} )
98.36 ± 0.18 †
98.89 ± 0.08 †
4.65 ± 0.16
2.29 ± 0.19
0.72 ± 0.01
Ent-PPO
Target entropy level
98.33 ± 0.25 †
98.98 ± 0.14 †
5.12 ± 0.36
2.99 ± 0.32
0.62 ± 0.09
TABLE I : Main performance on WOMD (10K validation maps, 4 seeds, all methods trained for 1.5B steps). Score = goal reached ∧ no collision. Goal = fraction reaching goal. † : p<0.05 vs. Baseline (Welch’s t -test). UncArms is the only conditioned policy to surpass the unconditional baseline.
Method
Score (%) ↑
Goal (%) ↑
Coll. (%) ↓
Offroad (%) ↓
DNF (%) ↓
UncArms (Ours)
99.12 ± 0.32
99.24 ± 0.30
9.53 ± 0.14
1.15 ± 0.10
0.44 ± 0.18
Baseline (PPO)
97.97 ± 0.57
98.27 ± 0.58
9.61 ± 0.31
1.36 ± 0.26
1.27 ± 0.45
DIAYN
97.16 ± 0.75
97.35 ± 0.62
13.12 ± 0.50
2.31 ± 0.46
1.08 ± 0.23
Ent-PPO
95.67 ± 0.25
95.98 ± 0.39
12.40 ± 0.54
2.48 ± 0.37
2.47 ± 0.49
GIGAFLOW ∗
91.82 ± 1.91
92.02 ± 1.82
10.95 ± 0.66
2.31 ± 0.53
3.97 ± 1.33
TABLE II : Zero-shot transfer to nuplan Val14 (no fine-tuning). SDC-only: ego agent controlled by policy, background agents replay logged trajectories. UncArms achieves best transfer on all metrics.
Method
Score (%) ↑
Goal (%) ↑
Coll. (%) ↓
Offroad (%) ↓
DNF (%) ↓
UncArms (Ours)
82.39 ± 0.68
93.05 ± 0.49
9.84 ± 0.13
26.67 ± 1.56
0.69 ± 0.08
Baseline (PPO)
78.03 ± 2.79
89.33 ± 2.78
9.66 ± 0.46
30.07 ± 2.61
1.25 ± 0.17
DIAYN
75.00 ± 2.88
82.77 ± 2.90
12.02 ± 0.83
28.28 ± 2.93
3.09 ± 0.09
Ent-PPO
76.92 ± 0.93
85.18 ± 0.91
12.82 ± 0.72
27.70 ± 0.79
3.26 ± 0.99
GIGAFLOW ∗
69.44 ± 2.30
76.71 ± 1.74
11.21 ± 0.43
30.18 ± 2.09
4.51 ± 0.81
TABLE III : Zero-shot transfer to Argoverse 2 validation (24,988 scenarios, no fine-tuning). SDC-only control. UncArms leads on score and goal rate.
Fig. 3 : Epistemic uncertainty as failure predictor and calibration analysis (UncArms, 113K agents across 10K WOMD validation maps, wunc=0 ). (a) Combined failure rate (collision + offroad + DNF) by uncertainty decile. (b) Reliability diagram: logistic mapping fitted on training data, evaluated on held-out validation.
Method
Coverage ↑
Eff. Dim. ↑
Spread
UncArms (Ours)
0.357
110
34.33
Baseline (PPO)
0.193
94
33.25
TABLE IV : LSTM hidden-state diversity.
Method
Arms-Race
Score (%)
Baseline (PPO, no dropout)
×
99.29 ± 0.08
Baseline + MC-Dropout
×
98.50 ± 0.08
UncArms (Ours)
✓
99.41 ± 0.08
TABLE V : Ablation isolating the adversarial training contribution (10K WOMD maps, 4 seeds). MC-Dropout alone affects the performance while the arms-race mechanism more than compensates.
(a) Neighborhood size K
Score (%) ↑
K=1
98.97 ± 0.24
K=3 (Ours)
99.41 ± 0.08
K=5
99.22 ± 0.09
K=10
98.99 ± 0.20
(b) Warmup schedule
Score (%) ↑
With warmup (Ours)
99.41 ± 0.08
TABLE VI : Neighborhood size ( K ) and warmup ablations.
Method
Score (%) ↑
Baseline (PPO)
99.29 ± 0.08
UncArms (no role assignment)
99.18 ± 0.30
UncArms (Concatentation)
99.35 ± 0.07
UncArms (FiLM, Ours)
99.41 ± 0.08
TABLE VII : Effect of conditioning mechanism on WOMD score. All variants use the same arms-race training. Only the role-signal modulation differs.
Neutral (%)
Probing (%)
Score (%) ↑
25
75
99.04 ± 0.53
50
50
99.41 ± 0.08
75
25
99.40 ± 0.03
TABLE VIII : Effect of probing fraction of UncArm on WOMD score.
Fig. 4 : Qualitative safety comparison under neutral conditioning ( wunc=0 ). For each scenario, the left panel shows the initial state, and the 2×2 matrix shows the controlled ego (blue, red on collision) for Baseline (top) and UncArms (bottom) at two timesteps. Other agents replay their logged trajectories (green vehicles, amber pedestrians/cyclists), the yellow star marks the ego goal, and the red ring marks the collision partner.
Fig. 5 : Role-conditioned behavior from a single UncArms checkpoint on three WOMD scenarios. Left: initial scene with the goal visible. 2×2 matrix: ego at two timesteps under neutral conditioning ( wunc=0 , top) and probing conditioning ( wunc=0.5 , bottom). Blue: controlled ego. red: ego at collision. green: replay vehicles. amber: pedestrians/cyclists. red ring: collision partner.