Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy

Huan Rong; Chao Yin; Anouar Imel; Yijie Xia; Tinghuai Ma

cs.AISep 29, 2026

Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy

Authors: Huan Rong, Chao Yin, Anouar Imel, Yijie Xia, Tinghuai Ma

Organizations: School of Artificial Intelligence, Nanjing University of Information Science & Technology, Jiangsu, Nanjing, 210-044, China · School of Information and Communication Engineering, University of Electronic Science and Technology of China, Sichuan, Chengdu, 611731, China · School of Software, Nanjing University of Information Science & Technology, 210044, Jiangsu, China · School of Computer Engineering, Jiangsu Ocean University, Lianyungang, 222005, Jiangsu, China

Abstract

Constrained Reinforcement Learning has recently gained increasing attention in the field of Safe Autonomous Driving, where the general mechanism is to maximize the expected reward while keeping the overall action risk bounded. In this way, the safety issues arising in AD can be mitigated through constrained actions. However, existing Constrained RL methods still lack dynamics on the imposed constraints. For instance, the action cost adopted by the existing Primal-Dual/soft-constrained methods is often defined as static state-to-cost mapping, and the safe-action projection in hard-constrained methods relies on the static projection with the fixed feasible region boundary estimated from offline demonstrations. The above drawback tightly couples the imposed constraints to the training scenarios, leaving the AD policy hard to handle different interaction scenarios, due to the improper state-level action-cost and the static projection boundary. Consequently, in this paper, we propose Brain-SAD, a brain-inspired safe autonomous driving control framework with dynamic fear-oriented constraints. By perceiving the current vehicle-interaction scene, Brain-SAD generates dynamic fear signal as fear reaction to online decide long-term policy for regular interaction or short-term policy for urgent-collision defense. In such two policy, the above fear-reaction will be constructed as the dynamic fear constraints, respectively reflecting the overall fear cost directly coupled with action-impact, and the dynamic fear boundary of the feasible region derived from different risky neighbors, both of which will in turn serve for the online policy optimization. Experimental results show that Brain-SAD outperforms existing methods, achieving higher success rate in shorter task-completion and collision-recovery time, and exhibits stronger reliability across continuous intersections of fluctuating complexity.

Figures & tables

Fig. 1: Two Constrained RL paradigms for Safe AD: Soft-constraint (Primal-dual) & Hard-constraint.

Fig. 2: Our proposed Brain-SAD safe AD control framework with dynamic fear-oriented constraint ( NOTE : Striatal D1/D2 (StrD1/StrD2) neurons belong to Direct/Indirect pathway in Striatum [ 4 ] , Subthalamic Nucleus (STN) in Hyper-direct pathway [ 5 ] ).

Fig. 3: Current-Scene Imagination for Dual Policy Selection ( NOTE : We use $\boldsymbol{K}$ parameter-independent yet structurally identical multi-layer MLPs as the Ensemble Environment Model; a multi-layer MLP as the default long-term policy ( Actor ) network).

Fig. 4: Online Long-Term Dynamic Fear-Constrained Policy ( NOTE : We reuse the same $\boldsymbol{K}$ multi-layer MLPs mentioned in Fig. 3 as the Ensemble Environment Model in Fig. 4 ; a multi-layer MLP as long-term policy ( Actor ) network).

\left\{\begin{aligned} &\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{Fear}\leftarrow\varphi\cdot Q_{Cost}^{s_{t},A_{t}^{\theta_{t-1}}}+(1-\varphi)\cdot Fear_{s_{t},A_{t}^{\theta_{t-1}}}\\ &Q_{Cost}^{s_{t},A_{t}^{\theta_{t-1}}}=\frac{1}{N}\sum_{i=1}^{N}Q_{critic,i}^{Cost}(s_{t},A_{t}^{\theta_{t-1}})\\ &Fear_{s_{t},A_{t}^{\theta_{t-1}}}\leftarrow\left\{a_{t,i}^{\theta_{t-1}}\in A_{t}^{\theta_{t-1}}\ \middle|\beta\cdot p_{t+1}^{col,t+L}+(1-\beta)\cdot\sigma_{t+1}^{U,t+L}=\frac{1}{K}\sum_{k=1}^{K}\sum_{t+1}^{t+L}W_{\Phi,k}^{MLP}(s_{t},a_{t,i}^{\theta_{t-1}})\right\}\\ &s_{t}=[Z_{t}^{1},\ldots,Z_{t}^{i},\ldots,Z_{t}^{m},vel_{t}^{ego},\varphi_{t}^{ego}],i\in[1,m]\end{aligned}\right.

(6)

Table 5

\left\{\begin{aligned} &q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})=\operatorname*{argmax}_{q}E_{q(A_{t}^{\theta_{t-1}}\mid s_{t})}[Q_{Reward}^{s_{t},A_{t}^{\theta_{t-1}}}]\\ &\text{s.t.}\;E_{q(A_{t}^{\theta_{t-1}}\mid s_{t})}[\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{Fear}]\leq\varepsilon_{F},\;D_{KL}\!\left[q(A_{t}^{\theta_{t-1}}\mid s_{t})\middle\|\pi_{\theta_{t-1}}^{long-term}(A_{t}^{\theta_{t-1}}\mid s_{t})\right]\leq\varepsilon_{KL}\\ &\int q(A_{t}^{\theta_{t-1}}\mid s_{t})\,dA_{t}^{\theta_{t-1}}=1\end{aligned}\right.

(7)

Table 6

\min\,\underbrace{L(q,\lambda,\eta)}_{\smash{\text{Lagrange Objective}}}\rightarrow q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})=\pi_{\theta_{t-1}}^{long-term}(A_{t}^{\theta_{t-1}}\mid s_{t})\cdot\exp\!\left(\frac{Q_{\text{Reward}}^{s_{t},A_{t}^{\theta_{t-1}}}-\lambda\cdot\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{\text{Fear}}}{\eta}\right)\cdot\frac{1}{Z(s_{t})}

(9)

Table 7

\mathit{Loss}_{\pi_{\theta}}^{\text{long-term}}(\theta)=-E_{s_{t}\sim\mathit{Buff}}\Bigg[\underbrace{\sum_{i=1}^{N}w_{t,i}^{\theta_{t-1}}\cdot\log\pi_{\theta}^{\text{long-term}}\!(a_{t,i}^{\theta_{t}}\mid s_{t})}_{{\text{online posterior supervision from }\,q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})}}+\beta\cdot KL_{s_{t}\sim\mathit{Buff}}\!\left[\pi_{\theta}^{\text{long-term}}\!(A_{t}^{\theta_{t}}\mid s_{t})\middle\|\pi_{\theta_{t-1}}^{\text{long-term}}\!(A_{t}^{\theta_{t-1}}\mid s_{t})\right]\Bigg]

(11)

Table 8

\left\{\begin{aligned} &A_{safe}(s_{t})=G_{proj}^{set}\otimes(a_{t}^{risk})^{T}=\{A_{proj},e_{brake}\}\otimes\begin{pmatrix}a_{t}^{steer}\\ a_{t}^{vertical}\end{pmatrix}\\[1.54996pt] &G_{proj}^{set}=\{A_{proj},e_{brake}\},\;A_{proj}=\begin{pmatrix}1&0\\ 0&1\end{pmatrix},\;e_{brake}=\begin{pmatrix}0&1\end{pmatrix}\\[1.54996pt] &a_{t}^{risk}=\bigl(a_{t}^{steer}\in\mathrm{R},a_{t}^{vertical}\in\mathrm{R}\bigr)\leftarrow\pi_{\varphi}^{short-term}(s_{t})\end{aligned}\right.

(13)

Table 9

\left\{\!\begin{aligned} &\operatorname*{argmin}_{a_{t}^{safe}\in A_{safe}(s_{t})}\left\|a_{t}^{safe}-a_{t}^{risk}\right\|,\;\text{s.t.}\;\overbrace{A_{safe}(s_{t})=G_{proj}^{set}\otimes(a_{t}^{risk})^{T}}^{\text{reachable action set projection}}\leq\underbrace{\Psi_{s_{t},a_{t}}^{\text{Fear}}=\left\{\tau_{safe}=(-1,1)^{T},\;b_{t}^{req}<0\right\}}_{\text{Brain-Inspired Dynamic Constraint}}\\ &b_{t}^{req}\in\mathrm{R}=\min\!\left(brake_{req}^{i,t}\right),\;brake_{req}^{i,t}<0\propto Neighbor(s_{t})\end{aligned}\right.

(15)

Table 10

MLE\_Loss_{\Phi_{k}}=-E_{(s_{t},a_{t},s_{t+1},r_{t},y_{t})\sim Buff}\Big[\log N(s_{t+1};\hat{\mu}_{t+1},(\hat{\sigma}_{t+1}^{U})^{2})+\lambda\Big(y_{t}\log\hat{p}_{t+1}^{col}+(1-y_{t})\log(1-\hat{p}_{t+1}^{col})\Big)\Big]

(16)

Table 11

	$\displaystyle L_{\begin{subarray}{c}\Phi=\{\eta_{NE},\gamma_{NE},\\ \eta_{5-HT},\gamma_{5-HT}\}\end{subarray}}^{\text{Selection}}$	$\displaystyle=-E_{\mathit{Buff}}\Bigl[\underbrace{\lambda_{safe}\\|g_{PFC}^{\Phi,t}-(1-y_{t})\\|_{L_{2}}^{2}+\lambda_{\text{Reward}}\\|g_{PFC}^{\Phi,t}-(0.5\pm\tanh(r_{t}))_{policy\_flag_{t}}\\|_{L_{2}}^{2}}_{{\text{Policy Selection}}}$		(17)
		$\displaystyle}{\displaystyle+\underbrace{\lambda_{NE}\\|\mathit{Signal}_{NE}^{\Phi,t}-\min(1,1.5\!\cdot\!Fear_{s_{t},a_{t}})\\|_{L_{2}}^{2}+\lambda_{5-HT}\\|\mathit{Signal}_{5-HT}^{\Phi,t}-\max(0,1-1.5\!\cdot\!Fear_{s_{t},a_{t}})\\|_{L_{2}}^{2}}_{{\text{Neurotransmitter Alignment}}}\Bigr]$		(17)

Table 12

\left\{\begin{aligned} &Loss_{R,Critic}^{long-term}=-E_{Buff}\Big[\big(Q_{Reward,Critic}^{long-term}(s_{t},a_{t})-\big(r_{t}+\gamma\cdot E_{a\sim\pi_{\theta_{old}}^{long-term}}[Q_{Reward,Critic}^{long-term,Target}(s_{t+1},a_{t+1})]\big)\big)^{2}\Big]\\ &Loss_{C,Critic}^{long-term}=-E_{Buff}\Big[\big(Q_{Cost,Critic}^{long-term}(s_{t},a_{t})-\big(c_{t}+\gamma\cdot E_{a\sim\pi_{\theta_{old}}^{long-term}}[Q_{Cost,Critic}^{long-term,Target}(s_{t+1},a_{t+1})]\big)\big)^{2}\Big]\end{aligned}\right.

(18)

Table 13

\min Loss_{\text{Brain-SAD}}^{\text{Offline}}=\lambda_{1}\cdot MLE\_Loss_{\Phi_{k}}+\lambda_{2}\cdot L_{\Phi}^{\text{Selection}}+\lambda_{3}\cdot Loss_{R/C,\text{Critic}}^{\text{Long-term}}+\lambda_{4}\cdot Loss_{\text{Critic\&Actor}}^{\text{Short-term}}

(20)

Table 14

Fig. 5: No traffic light intersection simulation with different complexity (NOTE: red as ego-vehicle; Scene A : Turn left at intersection only with multiple ahead interaction; Scene B : Straight across at intersection with multiple ahead and single aside interaction; Scene C : Straight across at intersection with multiple ahead and multiple aside interaction).

Combo	Module Combination	TCT $\downarrow$ (s)	SR $\uparrow$ (%)	Reward $\uparrow$	SLC $\uparrow$ (%)	Comfortable $\uparrow$ (%)
1	Brain-SAD-FULL	28.50 $\pm$ 0.10	98.67 $\pm$ 0.67	105.77 $\pm$ 2.57	92.05 $\pm$ 0.04	92.70 $\pm$ 0.28
2	only State Deviation in Scene Imagination	29.20 $\pm$ 0.80	96.22 $\pm$ 0.38	95.11 $\pm$ 3.01	87.29 $\pm$ 0.67	88.90 $\pm$ 0.96
3	only Collision Rate in Scene Imagination	28.40 $\pm$ 0.60	98.44 $\pm$ 0.77	94.43 $\pm$ 3.63	89.10 $\pm$ 0.28	90.13 $\pm$ 0.03
4	w/o Scene Imagination	29.40 $\pm$ 1.50	95.33 $\pm$ 1.15	88.43 $\pm$ 3.57	83.27 $\pm$ 2.18	86.70 $\pm$ 0.55
5	Brain-SAD only Long Term Policy	30.00 $\pm$ 2.20	97.78 $\pm$ 0.38	99.32 $\pm$ 0.63	84.55 $\pm$ 2.43	89.63 $\pm$ 0.08
6	w/o Dynamic Fear Cost	31.60 $\pm$ 1.30	96.67 $\pm$ 1.33	88.39 $\pm$ 0.17	81.35 $\pm$ 2.09	85.40 $\pm$ 0.34

TABLE I: The Ablation Study of Brain-SAD on Scene A ( NOTE: all the performances are averaged by three rounds).

Combo	Module Combination	TCT $\downarrow$ (s)	SR $\uparrow$ (%)	Reward $\uparrow$	SLC $\uparrow$ (%)	Comfortable $\uparrow$ (%)
1	Brain-SAD-FULL	22.30 $\pm$ 0.10	97.56 $\pm$ 0.38	94.35 $\pm$ 0.05	88.91 $\pm$ 0.17	89.97 $\pm$ 1.82
2	only State Deviation in Scene Imagination	27.10 $\pm$ 0.10	97.33 $\pm$ 0.67	89.91 $\pm$ 0.24	86.40 $\pm$ 0.49	87.00 $\pm$ 0.40
3	only Collision Rate in Scene Imagination	28.20 $\pm$ 0.00	96.67 $\pm$ 1.15	86.33 $\pm$ 0.21	82.80 $\pm$ 0.31	86.30 $\pm$ 1.70
4	w/o Scene Imagination	28.60 $\pm$ 0.60	96.00 $\pm$ 0.67	80.56 $\pm$ 0.78	80.60 $\pm$ 1.09	83.00 $\pm$ 0.40
5	Brain-SAD only Long Term Policy	29.80 $\pm$ 0.10	97.33 $\pm$ 1.15	87.78 $\pm$ 0.28	86.50 $\pm$ 1.80	85.60 $\pm$ 0.50
6	w/o Dynamic Fear Cost	29.80 $\pm$ 0.60	93.56 $\pm$ 0.77	82.20 $\pm$ 1.70	76.20 $\pm$ 1.83	80.80 $\pm$ 0.69

TABLE II: The Ablation Study of Brain-SAD on Scene B ( NOTE: all the performances are averaged by three rounds).

NO	Category	Methods	TCT $\downarrow$ (s)	RECT $\downarrow$ (s)	SR $\uparrow$ (%)	Reward $\uparrow$	SLC $\uparrow$ (%)	Comfortable $\uparrow$ (%)	Median-TTC $\uparrow$ (s)
1	Policy-Oriented Constrained	CADRE	36.30 $\pm$ 0.60	0.7181 $\pm$ 0.0032	93.56 $\pm$ 0.83	85.45 $\pm$ 0.44	82.35 $\pm$ 0.51	83.34 $\pm$ 0.72	0.7162 $\pm$ 0.0045
2		Iso-Dream++	30.20 $\pm$ 0.20	0.6245 $\pm$ 0.0057	97.29 $\pm$ 0.13	86.23 $\pm$ 0.82	86.12 $\pm$ 0.10	87.52 $\pm$ 0.23	1.1387 $\pm$ 0.0102
3		RLfOLD	33.20 $\pm$ 0.80	0.5661 $\pm$ 0.0310	92.00 $\pm$ 0.94	82.88 $\pm$ 1.80	83.52 $\pm$ 0.37	84.75 $\pm$ 2.11	1.0266 $\pm$ 0.0053
4		MARL-CCE	31.60 $\pm$ 0.40	0.8417 $\pm$ 0.0335	92.67 $\pm$ 1.10	81.05 $\pm$ 0.02	81.72 $\pm$ 2.58	81.16 $\pm$ 0.04	0.7340 $\pm$ 0.0057
5	Environment-Oriented Prior Constrained	LS-Imagination	31.20 $\pm$ 0.60	0.6547 $\pm$ 0.0125	96.03 $\pm$ 0.00	84.57 $\pm$ 0.02	85.47 $\pm$ 1.11	85.77 $\pm$ 0.06	1.0934 $\pm$ 0.0013
6		CarPlanner	33.10 $\pm$ 0.10	0.8333 $\pm$ 0.0471	95.57 $\pm$ 0.57	84.37 $\pm$ 0.14	85.30 $\pm$ 1.79	82.20 $\pm$ 0.01	0.8956 $\pm$ 0.0060

TABLE III: The Comparative Study of Brain-SAD on Scene B ( NOTE: the bold font represents the most optimal performance in each column, with the colored as the sub-optimal performance in each category. ALL the performance are averaged by three rounds).

Panel of Figure 19 — Fig. 6: The training traces and action distributions on representative methods selected from Table 3 , simulated in Scene B . ( NOTE : In ( a )–( d ), we present training traces on methods with TOP 2 Reward reported in Table 3 ; Here, the Reward are the average cumulative reward on the best training round (150 episodes, 3 rounds in all); In ( e )–( h ), we present action distributions across the best episode (selected in the above best training round) on methods with TOP 1 Reward reported in Table 3 , X-axis for acceleration positive as speeding, Y-axis for steering angle, Z-axis for temporal steps in the best episode.)

Panel of Table 20 — TABLE V: The performance of Brain-SAD on continuous Stress Test across 6 intersections in Fig. IV-D ( NOTE: the bold font represents the most optimal performance in each column, with the colored as the sub-optimal in each column. ALL the performance are averaged by three rounds).

Panel of Figure 21 — Fig. 10: The driving trace of the ego-vehicle when passing 6 continuous intersections controlled by FNI-RL ( a – f ) and Brain-SAD ( g – l ) in the same best episode used in Fig. 9 ( e )–( h ).

Panel of Figure 22 — Fig. 11: Environmental adaptation comparison among Brain-SAD, SVPG, FNI-RL and EFT-RL. ( NOTE: Simulated in the 6 continuous intersections in Fig. 8, we compare: ( a ) . Perceptional fear-signals between Brain-SAD and FNI-RL; ( b ) . Policy selection on Brain-SAD based on the fined-grained signals on calmness and nervousness ; ( c ) . Policy adaptability by step-wise cumulative reward across continuous intersections trained in the best episode).

Explore similar work

Mar 6, 2026cs.RO

Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints

Reinforcement learning has demonstrated significant potential in the field of autonomous driving. However, it suffers from defects such as training instability and unsafe action outputs when faced with autonomous racing environments characterized by high dynamics and strong nonlinearities. To this end, this paper proposes a trajectory guidance and dynamics constraints Reinforcement Learning (TraD-RL) method for autonomous racing. The key features of this method are as follows: 1) leveraging the prior expert racing line to construct an augmented state representation and facilitate reward shaping, thereby integrating domain knowledge to stabilize early-stage policy learning; 2) embedding explicit vehicle dynamic priors into a safe operating envelope formulated via control barrier functions to enable safety-constrained learning; and 3) adopting a multi-stage curriculum learning strategy that shifts from expert-guided learning to autonomous exploration, allowing the learned policy to surpass expert-level performance. The proposed method is evaluated in a high-fidelity simulation environment modeled after the Tempelhof Airport Street Circuit. Experimental results demonstrate that TraD-RL effectively improves both lap speed and driving stability of the autonomous racing vehicle, achieving a synergistic optimization of racing performance and safety.

Bo Leng, Weiqi Zhang, Zhuoren Li +4

College of Automotive and Energy Engineering, Tongji University, Shanghai, 201804, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, 639798, Singapore

Autonomous Racing Autonomous Driving Reinforcement Learning Control Robust Trajectory Offline Reinforcement Learning Trajectory Generation

Ask PDF

May 13, 2026cs.LG

Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraints

Safe reinforcement learning in nonstationary environments requires safety mechanisms that adapt as environmental conditions change. Standard safe reinforcement learning methods often assume fixed constraints or stable environmental conditions, which can become inadequate under distribution shift. We propose LILAC+, a framework for safe continual reinforcement learning under nonstationarity that combines three adaptive safety mechanisms: context-based safety constraints, adaptation-speed constraints, and budget-to-state safety enforcement. Context-based constraints adjust safety requirements using inferred and predicted environmental context. Adaptation-speed constraints tighten safety requirements when the rate of environmental change exceeds the agent's ability to adapt safely. Budget-to-state enforcement converts cumulative safety requirements into local state-level control constraints that can be enforced at decision time. Together, these mechanisms provide a unified approach for proactive and reactive safety adaptation in continual reinforcement learning. We evaluate the framework in simulated driving environments under stationary, seen nonstationary, and unseen nonstationary conditions. The results show that adaptive safety constraints substantially reduce safety violations under distribution shift while maintaining competitive task performance compared with unconstrained and fixed-constraint baselines. These findings suggest that safe continual reinforcement learning requires adaptive constraint mechanisms that respond not only to current state information but also to predicted environmental context, adaptation demand, and remaining safety budget.

Timofey Tomashevskiy

McMaster Centre for Software Certification Department of Computing and Software McMaster University, Canada

Safety Constraints Offline Reinforcement Learning Replay-Based Continual Learning Non-Stationarity

Ask PDF

May 13, 2026cs.LG

From Cumulative Constraints to Adaptive Runtime Safety Control for Nonstationary Reinforcement Learning

Safety in reinforcement learning is often specified through cumulative cost constraints, but these trajectory-level guarantees do not directly prevent unsafe individual decisions, especially under nonstationarity. In continual and nonstationary settings, the difficulty is amplified because the risk associated with the same action can vary across contexts, while a fixed state-level threshold may be either too conservative or too weak. We propose Constraint Projection Safety Shield (CPSS), a runtime mechanism that converts a cumulative safety budget into adaptive state-level control constraints during execution. CPSS tracks the remaining safety budget, projects it into a time-varying admissible risk threshold, and filters policy actions whose predicted safety cost exceeds the active threshold. The threshold is adjusted online using contextual signals so that enforcement becomes stricter in more demanding or rapidly changing regimes and less restrictive when the available safety budget is sufficient. We analyze the resulting shielded policy and show that the mechanism guarantees per-state threshold satisfaction for executed actions, induces finite-horizon cumulative cost bounds, and yields a performance degradation bound in terms of intervention frequency and per-step reward distortion. We evaluate CPSS in nonstationary highway merging scenarios using highway-env. Across multiple seeds, CPSS substantially reduces proximity-based safety violations and increases separation margins while intervening selectively rather than dominating the learned policy. These results support adaptive budget-to-threshold projection as a practical way to transform cumulative safety specifications into effective local safety control for continual reinforcement learning systems.

Timofey Tomashevskiy

McMaster Centre for Software Certification Department of Computing and Software McMaster University, Canada

Safety Constraints Reinforcement Learning Control Offline Reinforcement Learning Structural Constraints Decision Thresholds Traffic

Ask PDF