Security of Deep Reinforcement Learning for Autonomous Driving: A Survey

Ambra Demontis; Srishti Gupta; Maura Pintor; Luca Demetrio; Kathrin Grosse; Hsiao-Ying Lin; Chengfang Fang; Battista Biggio

cs.LGDec 12, 2022

Security of Deep Reinforcement Learning for Autonomous Driving: A Survey

Authors: Ambra Demontis, Srishti Gupta, Maura Pintor, Luca Demetrio, Kathrin Grosse, Hsiao-Ying Lin, Chengfang Fang, Battista Biggio,

Organizations: University of Cagliari · Sapienza, University of Rome · University of Genoa · IBM Research Zurich ·

Abstract

Reinforcement learning (RL) enables agents to learn optimal behaviors through interaction with their environment and has been increasingly deployed in safety-critical applications, including autonomous driving. Despite its promise, RL is susceptible to attacks designed either to compromise policy learning or to induce erroneous decisions by trained agents. Although the literature on RL security has grown rapidly and several surveys exist, existing categorizations often fall short in guiding the selection of appropriate defenses for specific systems. In this work, we present a comprehensive survey of 86 recent studies on RL security, addressing these limitations by systematically categorizing attacks and defenses according to defined threat models and single- versus multi-agent settings. Furthermore, we examine the relevance and applicability of state-of-the-art attacks and defense mechanisms within the context of autonomous driving, providing insights to inform the design of robust RL systems.

Figures & tables

	Literature	Defenses	RL-specific	SARL/	Defense
	after 2022			MARL	guidance
Behzadan et al. [ 5 ]	-	✓	✓	✓	-
Chen et al. [ 6 ]	-	✓	✓	✓	✓
Shen et al. [ 7 ]	✓	✓	-	-	-
Ibrahum et al. [ 8 ]	✓	✓	-	✓	-
Badjie et al. [ 9 ]	✓	✓	-	-	✓

TABLE I: Limitations of existing work in DRL security.

RL Symbols
$a$	Action
$\mathcal{E}$	Environment
$\mathbf{s}$	State
$r$	Reward
$\mathbf{o}$	Observations
$\mathcal{R}$	Return

TABLE II: Summary of the notation and abbreviations.

Fig. 1: Major Components of Autonomous Driving Systems.

Fig. 2: Conceptual representation of the components of an RL policy that the attacker can manipulate without having access to the RL algorithm.

	Integrity	Availability	Privacy
Test Data	- $\text{reach}({\mathbf{s}}^{T})$		- $\text{steal}({\mathcal{M}})$
Test Data	- $\min({\mathcal{R}})$
Training Data	- $\text{learn}({\pi}^{B})$	- $\text{learn}({\pi}^{T})$
Training Data	- $\text{learn}({\pi}_{\mathbf{s}}^{T})$	- $\text{learn}({\pi}^{U})$

TABLE III: The attacks against RL, categorized according to the attackers’ capability and the security violation they cause.

	AGENT MODEL			THREAT MODEL
References	Single vs.	Policy	Model	Attack’s	Attacker’s	Poisoning	Attacker’s	Attacker’s	Sequential	Attacker’s
	Multi-agent	Update	Based	Time	Action		Knowledge	Goal	Attack	Agent
Wang et al. [ 17 ]	MARL	off-policy		$tr$	$c({r})$	✓	●	$\text{learn}({\pi}^{B})$		$\in\mathcal{E}$
Gleave et al. [ 19 ]	MARL	on-policy		$ts$	$c({\mathcal{E}})$		◐	$\min({\mathcal{R}})$	✓	$\in\mathcal{E}$
Zhao et al. [ 20 ]	SARL	on/off-policy		$ts$	$c({\mathbf{o}})$		●	$\min({\mathcal{R}})$		$\emptyset$
Rakhsha et al. [ 22 ]	SARL	on-policy, offline	✓	$tr$	$c({r,a,\mathbf{s}})$	✓	○	$\text{learn}({\pi}^{T})$	✓	$\emptyset$

TABLE IV: Attacks against reinforcement learning systems. The presence of the ✓indicates that the corresponding attribute is true for the attack. For the attacker’s knowledge, we use ● to represent black-box, ◐ for gray-box, and ○ for white-box. Refer Table II for attacker’s agent in the system and Table III for attacker’s goal.

References	Target	Attacker’s	Attacker’s	Poisoning	Sequential	Multi	CDS
	Component	Goal	Action		Attack	Agent
Wang et al. [ 17 ]	controller	traffic jam/crash	positions and speeds	✓		✓
Ma et al. [ 32 ]	planning	car is rerouted	training rewards	✓
Yang et al. [ 28 ]	end-to-end	car leaves lane	sensor input / FGSM		✓
Boloor et al. [ 29 ]	end-to-end	car leaves lane	paint pattern on street		✓
Sun et al. [ 30 ]	end-to-end	car leaves lane	sensor environment / FGSM		✓	✓

TABLE V: Attacks evaluated on autonomous driving tasks. /✓indicates both the presence and absence of the feature in that paper. CDS stands for cooperative driving scenario.

Fig. 3: Timeline of attacks against reinforcement learning, compared to those proposed against machine-learning classifiers.

	DEFENSE MODEL			THREAT MODEL
References	Single vs.	Policy	Defense	Defense	Attack’s	Attacker’s	Poisoning	Attacker’s	Attacker’s	Sequential	Attacker’s
	Multi-agent	Update	Type	Technique	Time	Action		Knowledge	Goal	Attack	Agent
Everitt et al. [ 93 ]	SARL	on-policy	proactive	reg.	$tr$	$c({r})$	✓	◐	$\text{learn}({\pi}^{U})$		$\emptyset$
Everitt et al. [ 93 ]	SARL	on-policy	reactive	san.	$tr$	$c({r})$	✓	◐	$\text{learn}({\pi}^{U})$		$\emptyset$
Wang et al. [ 54 ]	SARL	on-policy	reactive	san.	$tr$	$c({\mathcal{E}})$	✓	○	$\text{learn}({\pi}^{B})$	✓	$\in\mathcal{E}$
Wang et al. [ 94 ]	SARL	off-policy	reactive	san.	$tr$	$c({r})$	✓	●	$\text{learn}({\pi}^{U})$		$\emptyset$

TABLE VI: Defenses for reinforcement learning systems. The presence of the ✓indicates that the corresponding attribute is true for the defense. For the attacker’s knowledge, we use ● to represent black-box, ◐ for gray-box, and ○ for white-box. All the defenses proposed so far have been tested on model-free RL algorithms, except [ 92 ] .

Robust RL

Robust reinforcement learning develops agents that maintain performance and safety under uncertainty, disturbances, attacks, and distribution shifts.

See all 229 papers

Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning
Jul 9, 2026Zifan Zhang, Minghong Fang, Dianwei Chen +5Federated RL Autonomous Driving
North Carolina State University, USA · University of Louisville, USA · University of Maryland, USA +2
Threat-guided Policy-aware Scene Perturbation for Safe Autonomous Driving with Online Reinforcement Learning
Aug 11, 2026Xincong Hu, Lei Ou, Maosen Li +3Adversarial Scenario Generation Safe RL
Nanjing University · Yinwang Intelligent Technology Co., Ltd
PolicyGuard: Towards Test-time and Step-level Adversary (Backdoor) Defense for Reinforcement Learning Agent
Jun 11, 2026Junfeng Guo Heng HuangBackdoor Attacks Backdoor Defense
Computer Science, University of Maryland College Park, USA.

Jul 9, 2026J/K move · Enter open · S save

Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning

Zifan Zhang, Minghong Fang, Dianwei Chen, Zhuqing Liu+4

North Carolina State University, USA · University of Louisville, USA · University of Maryland, USA+2

Federated RL Autonomous Driving

Open paper AskPDF arXiv

Federated reinforcement learning (FRL) is crucial for enabling collaborative learning across multiple agents without sharing raw data, thereby enhancing privacy and scalability in the decision-making process within dynamic vehicular environments. However, poisoning attacks pose a significant threat to the security and reliability of FRL-based systems, particularly in safety-critical autonomous driving, where this vulnerability remains largely unexplored. These attacks can compromise the global control model by subtly injecting malicious system parameters, leading to potential hazards. To counter these challenges, we present \alg (Secure Aggregation with poisoning-prevention and historical reinforcement) as a defensive framework aimed at enhancing the robustness of FRL systems designed for safety-critical driving scenarios. \alg strategically integrates digital twins for rehearsal-based learning and leverages historical aggregated model parameters along with a selected central gradient to ensure that only benign data is aggregated, effectively mitigating the influence of malicious agents. Theoretical guarantees are provided for the convergence performance of \alg in the presence of poisoning attacks. We also validate the effectiveness of \alg using developed digital twins that model realistic highway environments to evaluate the control of autonomous vehicles under adversarial conditions.

Abstract

Robust RL

Explore similar work

Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning

Threat-guided Policy-aware Scene Perturbation for Safe Autonomous Driving with Online Reinforcement Learning

PolicyGuard: Towards Test-time and Step-level Adversary (Backdoor) Defense for Reinforcement Learning Agent