ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving

Tong Nie; Yihong Tang; Junlin He; Yuewen Mei; Jie Sun; Lijun Sun; Wei Ma; Jian Sun

cs.LGMar 16, 2026

ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving

Authors: Tong Nie, Yihong Tang, Junlin He, Yuewen Mei, Jie Sun, Lijun Sun, Wei Ma, Jian Sun

Organizations: The Hong Kong Polytechnic University, Hong Kong SAR, China · Tongji University, Shanghai, China · McGill University, Montreal, QC, Canada

Abstract

Deploying autonomous driving systems requires robustness against long-tail scenarios that are rare but safety-critical. While adversarial training offers a promising solution, existing methods typically decouple scenario generation from policy optimization and rely on heuristic surrogates. This leads to objective misalignment and fails to capture the shifting failure modes of evolving policies. This paper presents ADV-0, a closed-loop min-max optimization framework that treats the interaction between driving policy (defender) and adversarial agent (attacker) as a zero-sum Markov game. By aligning the attacker's utility directly with the defender's objective, we reveal the optimal adversary distribution. To make this tractable, we cast dynamic adversary evolution as iterative preference learning, efficiently approximating this optimum and offering an algorithm-agnostic solution to the game. Theoretically, ADV-0 converges to a Nash Equilibrium and maximizes a certified lower bound on real-world performance. Experiments indicate that it effectively exposes diverse safety-critical failures and greatly enhances the generalizability of both learned policies and motion planners against unseen long-tail risks.

Figures & tables

Figure 1: Illustration of the ADV-0 framework. It alternates between an Inner Loop where the adversary

\psi

evolves via IPL to track the failure frontier, and an Outer Loop where the ego

\pi_{\theta}

optimizes policy gradients against the induced adversarial distribution.

Adversary	Replay		IDM		RL Agent
Adversary	CR $\uparrow$	ER $\downarrow$	CR $\uparrow$	ER $\downarrow$	CR $\uparrow$	ER $\downarrow$
Replay	-	-	19.03%	51.75	16.80%	51.65
Heuristic	100.00%	0.00	74.70%	32.12	69.64%	24.90
CAT	90.08%	1.03	43.13%	43.47	36.84%	42.26
KING	23.28%	47.67	24.49%	49.41	21.26%	49.32
AdvTrajOpt	69.64%	3.12	26.92%	45.36	28.95%	45.10

Table 1: Safety-critical scenario generation . CR denotes the collision rate of the ego in generated scenarios, and ER denotes the ego’s cumulative return. Ego is controlled by Replay, IDM, and trained PPO policies, respectively. Metrics of ADV-0 are mean values across all training methods. See full results in Table 6 .

Figure 4: Examples of generated reward-reduced adversarial scenarios from ADV-0 . Additional cases are shown in Figure 15 .

	Val. Env.	RC $\uparrow$	Crash $\downarrow$	Reward $\uparrow$	Cost $\downarrow$
ADV-0 (w/ IPL)	Replay	0.742 $\pm$ 0.011	0.159 $\pm$ 0.019	49.56 $\pm$ 1.05	0.480 $\pm$ 0.013
	ADV-0	0.695 $\pm$ 0.011	0.289 $\pm$ 0.029	44.60 $\pm$ 0.83	0.598 $\pm$ 0.022
	CAT	0.704 $\pm$ 0.011	0.271 $\pm$ 0.025	45.68 $\pm$ 1.16	0.585 $\pm$ 0.025
	SAGE	0.699 $\pm$ 0.013	0.263 $\pm$ 0.025	45.19 $\pm$ 1.54	0.567 $\pm$ 0.028
	Heuristic	0.710 $\pm$ 0.016	0.217 $\pm$ 0.021	45.41 $\pm$ 2.08	0.552 $\pm$ 0.035
	Avg.	0.710 $\pm$ 0.012	0.240 $\pm$ 0.024	46.09 $\pm$ 1.33	0.556 $\pm$ 0.027

Table 2: Performance validation of learned policies . Results are averaged across 6 RL methods (GRPO, PPO, PPO-Lag, SAC, SAC-Lag, TD3). This table compares the performances of agents learned by different adversarial methods and evaluate on different generated scenarios. Full results are shown in Tables 8 , 8 , 10 , 10 , 12 , 12 .

Agent/Adversary	Reward ( $\uparrow$ )
Agent/Adversary	Adversary (w/o IPL)	Adversary (w/ IPL)	Decrease
Agent w/o IPL	$41.03\pm 1.87$	$39.01\pm 0.97$	$-4.92\%$
Agent w/ IPL	$44.60\pm 0.83$	$43.47\pm 1.40$	$-2.53\%$
Improvement	$+8.70\%$	$+11.43\%$
Agent/Adversary	Cost ( $\downarrow$ )
Agent/Adversary	Adversary (w/o IPL)	Adversary (w/ IPL)	Decrease

Table 3: Cross-validation of ADV-0 . Performances of agents and adversaries with/without IPL. Decrease indicates the percentage change in performance when facing an IPL-enhanced adversary compared to the baseline. Improvement indicates the percentage change when the agent is trained with IPL compared to the other.

Figure 5: Generalization gap over different training budgets.

	Reactive Traffic	Safety Margin		Stability & Comfort		Defensive Driving
	Reactive Traffic	Avg Min-TTC ( $\uparrow$ )	Avg Min-PET ( $\uparrow$ )	Mean Abs Jerk ( $\downarrow$ )	95% Jerk ( $\downarrow$ )	Near-Miss Rate ( $\downarrow$ )	RDP Violation Rate ( $\downarrow$ )
ADV-0 (w/ IPL)	$\checkmark$	$\pagecolor{myblue!20}{\boldsymbol{0.993\pm 0.189}}$	${\boldsymbol{1.150\pm 0.317}}$	${\boldsymbol{1.653\pm 0.169}}$	${\boldsymbol{5.053\pm 0.885}}$	${\boldsymbol{63.45\%\pm 4.34\%}}$	${\boldsymbol{33.70\%\pm 3.77\%}}$
ADV-0 (w/ IPL)	$\times$	${{1.527\pm 0.182}}$	${\boldsymbol{1.103\pm 0.313}}$	${\boldsymbol{1.617\pm 0.204}}$	${\boldsymbol{5.177\pm 0.953}}$	${\boldsymbol{60.97\%\pm 3.94\%}}$	${\boldsymbol{36.45\%\pm 2.85\%}}$
CAT	$\checkmark$	${0.825\pm 0.192}$	${1.017\pm 0.317}$	${1.875\pm 0.229}$	${5.422\pm 0.833}$	${83.04\%\pm 5.99\%}$	${44.80\%\pm 6.12\%}$
CAT	$\times$	${1.415\pm 0.462}$	${0.980\pm 0.327}$	${1.979\pm 0.193}$	${6.002\pm 0.931}$	${75.36\%\pm 7.86\%}$	${51.20\%\pm 4.74\%}$
Heuristic	$\checkmark$	${0.864\pm 0.200}$	$\boldsymbol{1.150\pm 0.467}$	${2.129\pm 0.237}$	${6.542\pm 0.942}$	${74.46\%\pm 5.98\%}$	${48.79\%\pm 3.65\%}$

Table 4: Robustness on mined real-world long-tailed sets. Average agent performance on four long-tail scenario categories filtered by criteria: Critical TTC (

\min\text{TTC}<0.4s

), Critical PET (

\text{PET}<1.0s

), Hard Dynamics (Acc.

<-4.0m/s^{2}

|\text{Jerk}|>4.0m/s^{3}

), and Rare Cluster (topologically sparse trajectory clusters). Metrics assess average Safety Margin (higher values indicate earlier risk detection), Stability & Comfort (lower Jerk indicates smoother control), and Defensive Driving , quantified by Near-Miss Rate (proximity without collision) and RDP Violation (percentage of time requiring deceleration

>6m/s^{2}

to avoid collision). Full results are shown in Table 15 .

Figure 6: Visualization of improved safe driving ability after being trained with ADV-0 . More examples are shown in Figure 14 .

Model	RC $\uparrow$	Crash $\downarrow$	Reward $\uparrow$	Cost $\downarrow$
PlanTF	0.628 $\pm$ 0.025	0.357 $\pm$ 0.031	35.85 $\pm$ 2.51	1.04 $\pm$ 0.04
+ ADV-0	0.674 $\pm$ 0.016	0.263 $\pm$ 0.025	41.98 $\pm$ 1.62	0.77 $\pm$ 0.02
Rel. Change	+7.46%	-26.23%	+17.11%	-25.68%
SMART	0.587 $\pm$ 0.029	0.396 $\pm$ 0.034	32.66 $\pm$ 2.85	1.15 $\pm$ 0.05
+ ADV-0	0.631 $\pm$ 0.015	0.305 $\pm$ 0.023	37.85 $\pm$ 1.70	0.92 $\pm$ 0.02
Rel. Change	+7.57%	-22.88%	+15.86%	-20.31%

Table 5: Application to learning-based planners . Performance comparisons of two SOTA trajectory planning models before and after fine-tuning using ADV-0 (GRPO). See Table 13 for details.

Figure 7: Impacts of the temperature parameter in sampling.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

ADV-0 Variation	Replay		IDM		RL Agent
ADV-0 Variation	CR $\uparrow$	Reward $\downarrow$	CR $\uparrow$	Reward $\downarrow$	CR $\uparrow$	Reward $\downarrow$
Pretrained (logit-based sampling)	$31.72\%\pm 1.06\%$	$43.31\pm 1.10$	$18.84\%\pm 1.48\%$	$50.66\pm 2.22$	$12.49\%\pm 1.06\%$	$52.12\pm 2.03$
Pretrained (energy-based sampling)	$85.09\%\pm 1.13\%$	$1.89\pm 0.20$	$40.14\%\pm 1.06\%$	$45.72\pm 0.47$	$36.30\%\pm 0.77\%$	$41.99\pm 0.33$
GRPO Finetuned	$92.61\pm 0.62\%$	$0.87\pm 0.06$	$46.34\pm 0.73\%$	$39.60\pm 0.13$	$41.95\pm 0.47\%$	$38.45\pm 0.42$
PPO Finetuned	$90.93\pm 0.69\%$	$1.01\pm 0.07$	$45.09\pm 0.63\%$	$40.69\pm 0.22$	$39.12\pm 0.38\%$	$39.63\pm 0.43$
SAC Finetuned	$91.88\pm 0.86\%$	$1.03\pm 0.08$	$46.09\pm 0.19\%$	$39.62\pm 0.24$	$41.09\pm 0.59\%$	$39.17\pm 0.48$

Appendix

Table 6: Detailed results of safety-critical scenario generation using ADV-0 .

Table 14

Table 15

Figure 10: Learning curves of TD3.

Table 17

Figure 11: Learning curves of GRPO.

Model Phase	Val. Env.	RC $\uparrow$	Crash $\downarrow$	Reward $\uparrow$	Cost $\downarrow$
PlanTF (Pretrained)	Replay	0.727 $\pm$ 0.021	0.220 $\pm$ 0.027	41.53 $\pm$ 3.10	0.76 $\pm$ 0.03
	ADV-0	0.581 $\pm$ 0.030	0.420 $\pm$ 0.035	32.11 $\pm$ 2.54	1.24 $\pm$ 0.05
	CAT	0.615 $\pm$ 0.025	0.385 $\pm$ 0.030	35.26 $\pm$ 2.16	1.09 $\pm$ 0.03
	SAGE	0.595 $\pm$ 0.028	0.407 $\pm$ 0.032	33.80 $\pm$ 2.80	1.15 $\pm$ 0.04
	Heuristic	0.620 $\pm$ 0.020	0.352 $\pm$ 0.032	36.53 $\pm$ 1.97	0.94 $\pm$ 0.03
	Avg.	0.628 $\pm$ 0.025	0.357 $\pm$ 0.031	35.85 $\pm$ 2.51	1.04 $\pm$ 0.04

Appendix

Table 13: Performance comparison of learning-based planners before and after adversarial fine-tuning using ADV-0 (GRPO).

Method	Progress	Collision	Off-road	Comfort	Speed Eff.	Total Score
Method	( + )	( - )	( - )	( - )	( - )	Total Score
PlanTF (Pretrained)	14.17 $\pm$ 1.53	-2.85 $\pm$ 0.92	-0.12 $\pm$ 0.05	-0.34 $\pm$ 0.08	-0.54 $\pm$ 0.12	10.32 $\pm$ 1.79
PlanTF (Fine-tuned)	16.45 $\pm$ 0.85	-0.89 $\pm$ 0.45	-0.05 $\pm$ 0.04	-0.24 $\pm$ 0.05	-0.64 $\pm$ 0.13	14.63 $\pm$ 0.97
Improvement	+16.09%	+68.77%	+58.33%	+29.41%	-18.52%	+41.76%
SMART (Pretrained)	13.20 $\pm$ 1.80	-3.61 $\pm$ 1.11	-0.25 $\pm$ 0.08	-1.26 $\pm$ 0.25	-0.60 $\pm$ 0.15	7.48 $\pm$ 2.14
SMART (Fine-tuned)	15.10 $\pm$ 1.10	-1.75 $\pm$ 0.37	-0.17 $\pm$ 0.05	-0.92 $\pm$ 0.19	-0.65 $\pm$ 0.11	11.61 $\pm$ 1.18

Appendix

Table 14: Detailed breakdown of the trajectory-level reward model values . Values represent the average accumulated discounted reward (over a 2.0s horizon with discount factor

\gamma=0.99

) of the best planned trajectory selected by the model across all validation steps. Reward components & weights : Progress (

w_{\text{prog}}=1.0

, longitudinal advance), Collision (

w_{\text{coll}}=20.0

, penalty for overlap with objects), Off-road (

w_{\text{off}}=5.0

, penalty for lane deviation), Comfort (

w_{\text{acc}}=0.1,w_{\text{jerk}}=0.1

, penalties for harsh dynamics), Speed Efficiency (

w_{\text{eff}}=0.2

, penalty for deviating from 10m/s). Note that the negative improvement in Speed Efficiency reflects a safety-efficiency trade-off, where the fine-tuned planner adopts a more conservative velocity profile to satisfy safety constraints.

	Scenario Category	Percentage	Reactive Traffic	Safety Margin		Stability & Comfort		Defensive Driving
	Scenario Category	Percentage	Reactive Traffic	Avg Min-TTC ( $\uparrow$ )	Avg Min-PET ( $\uparrow$ )	Mean Abs Jerk ( $\downarrow$ )	95% Jerk ( $\downarrow$ )	Near-Miss Rate ( $\downarrow$ )	RDP Violation Rate ( $\downarrow$ )
ADV-O (w/ IPL)	Critical TTC	7.40%	$\checkmark$	$0.645\pm 0.122$	$1.450\pm 0.50$	$1.853\pm 0.188$	$5.850\pm 1.252$	$65.28\%\pm 1.45\%$	$15.53\%\pm 4.16\%$
	Critical PET	3.40%		$0.492\pm 0.025$	$1.150\pm 0.20$	$1.623\pm 0.095$	$5.157\pm 0.653$	$74.57\%\pm 5.52\%$	$58.26\%\pm 3.29\%$
	Hard Dynamics	3.20%		$0.885\pm 0.157$	N/A	$1.685\pm 0.250$	$4.555\pm 0.951$	$55.40\%\pm 8.26\%$	$29.54\%\pm 4.80\%$
	Rare Cluster	5.20%		$1.951\pm 0.450$	$0.850\pm 0.25$	$1.451\pm 0.143$	$4.650\pm 0.684$	$58.55\%\pm 2.12\%$	$31.48\%\pm 2.81\%$
	Avg			$\mathbf{0.993\pm 0.189}$	$\mathbf{1.150\pm 0.317}$	$\mathbf{1.653\pm 0.169}$	$\mathbf{5.053\pm 0.885}$	$\mathbf{63.45\%\pm 4.34\%}$	$\mathbf{33.70\%\pm 3.77\%}$

Appendix

Table 15: Full results of quantitative evaluation on the unbiased long-tailed set mined from real-world data. The benchmark consists of four long-tail scenario categories filtered by strict physical thresholds: Critical TTC (

\min\text{TTC}<0.4s

), Critical PET (

\text{PET}<1.0s

), Hard Dynamics (Longitudinal Acc

<-4.0m/s^{2}

|\text{Jerk}|>4.0m/s^{3}

), and Rare Cluster (topologically sparse trajectory clusters). Reactive Traffic denotes whether background vehicles utilize IDM/MOBIL policies to interact with the agent (

\checkmark

) or strictly follow logged trajectories (

\times

). Metrics assess Safety Margin (higher values indicate earlier risk detection), Stability & Comfort (lower Jerk indicates smoother control), and Defensive Driving performance, quantified by Near-Miss Rate (hazardous proximity without collision) and RDP Violation (percentage of time requiring deceleration

>6m/s^{2}

to avoid collision).

Figure 14: Qualitative comparison of improved safe driving ability after being trained with ADV-0 . We showcase three typical safety-critical scenarios generated by the adversary: Left-turn, Sudden-brake, and Cut-in. In each column, the bottom row (Agent w/o ADV-0 ) shows the baseline agent failing to anticipate the aggressive behavior of the background traffic, resulting in collisions. In contrast, the top row (Agent w/ ADV-0 ) demonstrates that the agent trained with our framework learns robust defensive behaviors, such as yielding at intersections or decelerating in time, thereby successfully avoiding accidents.

Figure 15: Additional qualitative examples of reward-reduced adversarial scenarios from ADV-0 . In the first case (left), the adversary interrupts the straight-going ego, forcing it to deviate from the lane centerline to avoid a crash; this maneuver halts the ego’s progress, causing the cumulative reward to stagnate. In the second case (right), a collision occurs near the end of the episode, triggering a sharp drop in the cumulative reward due to the safety penalty.

Table 16: Hyperparameters for different RL algorithms used in the experiments.

Module	Parameter	Value
Backbone (DenseTNT)	Hidden Size	128
	Sub-graph Depth	3
	Global-graph Depth	1
	Trajectory Modes ( $K$ )	32
	NMS Threshold	7.2
IPL Fine-tuning	Learning Rate	5e-6

Appendix

Table 20: Hyperparameters for ADV-0 Framework and IPL Fine-tuning.

Long-Tail Autonomous Driving

See all 11 papers

From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving
Jun 12, 2026Yuewen Mei, Tong Nie, Jie Sun +3Adversarial Training Autonomous Driving
College of Transportation & Key Laboratory of Road and Traffic Engineering of Ministry of Education, Tongji University, Shanghai, 201804, China · Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China
Evaluation as Evolution: Transforming Adversarial Diffusion into Closed-Loop Curricula for Autonomous Vehicles
Apr 8, 2026Yicheng Guo, Jiaqi Liu, Chengkai Xu +2Autonomous Driving Safety Evaluation Adversarial Scenario Generation
College of Transportation, Tongji University, Shanghai 201804, China · Department of Computer Science, University of North Carolina at Chapel Hill, United States
World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning
Jul 12, 2026Tong Nie, Yuewen Mei, Junlin He +3Adversarial Training Self-Play RL
The Hong Kong Polytechnic University · Tongji University · McGill University +1

Jun 12, 2026J/K move · Enter open · S save

From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving

Yuewen Mei, Tong Nie, Jie Sun, Haotian Shi+2

College of Transportation & Key Laboratory of Road and Traffic Engineering of Ministry of Education, Tongji University, Shanghai, 201804, China · Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China

Adversarial Training Autonomous Driving Curriculum RL RL for Autonomous Driving Autonomous Driving Scenario Generation

Open paper AskPDF arXiv

Closed-loop adversarial training improves autonomous driving safety by exposing policies to rare safety-critical scenarios. Standard pipelines first generate adversarial scenarios and then sample them for policy optimization. However, most existing frameworks remain attack-oriented: collision-driven generators often synthesize unsolvable extreme situations, which can degrade learning, while heuristic samplers ignore the evolving capability of the driving policy, causing sample inefficiency and delayed convergence. We propose AlignADV, a learnability-guided closed-loop adversarial training framework that converts adversarial scenarios into resolvable and capability-aligned curricula. First, we reformulate adversarial scenario generation as a preference alignment problem and employ direct preference optimization to guide the generator toward critical yet resolvable scenarios. Second, we introduce behavioral fingerprints to capture the intrinsic characteristics of the evolving policy and construct a multi-modal capability prediction model that estimates policy performance without expensive closed-loop simulations. By combining resolvability-aligned scenarios with capability predictions, AlignADV develops a dynamic curriculum sampling mechanism that prioritizes scenarios targeting the current policy's vulnerabilities. Experiments on the Waymo Open Motion Dataset demonstrate that AlignADV improves convergence efficiency and final performance, reducing training steps by up to 40.6 percent compared with baseline methods while lowering collision rate and improving route completion under both normal and adversarial traffic conditions. These results highlight a shift from attack-oriented scenario generation to learnability-guided policy improvement, offering a principled direction for safer and more efficient autonomous driving training. Project page: https://meiyuewen.github.io/AlignADV/.

Abstract

Long-Tail Autonomous Driving

Explore similar work

From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving

Evaluation as Evolution: Transforming Adversarial Diffusion into Closed-Loop Curricula for Autonomous Vehicles

World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning