cs.AISep 29, 2026

Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy

Authors: Huan Rong, Chao Yin, Anouar Imel, Yijie Xia, Tinghuai Ma

Organizations: School of Artificial Intelligence, Nanjing University of Information Science & Technology, Jiangsu, Nanjing, 210-044, China · School of Information and Communication Engineering, University of Electronic Science and Technology of China, Sichuan, Chengdu, 611731, China · School of Software, Nanjing University of Information Science & Technology, 210044, Jiangsu, China · School of Computer Engineering, Jiangsu Ocean University, Lianyungang, 222005, Jiangsu, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Constrained Reinforcement Learning has recently gained increasing attention in the field of Safe Autonomous Driving, where the general mechanism is to maximize the expected reward while keeping the overall action risk bounded. In this way, the safety issues arising in AD can be mitigated through constrained actions. However, existing Constrained RL methods still lack dynamics on the imposed constraints. For instance, the action cost adopted by the existing Primal-Dual/soft-constrained methods is often defined as static state-to-cost mapping, and the safe-action projection in hard-constrained methods relies on the static projection with the fixed feasible region boundary estimated from offline demonstrations. The above drawback tightly couples the imposed constraints to the training scenarios, leaving the AD policy hard to handle different interaction scenarios, due to the improper state-level action-cost and the static projection boundary. Consequently, in this paper, we propose Brain-SAD, a brain-inspired safe autonomous driving control framework with dynamic fear-oriented constraints. By perceiving the current vehicle-interaction scene, Brain-SAD generates dynamic fear signal as fear reaction to online decide long-term policy for regular interaction or short-term policy for urgent-collision defense. In such two policy, the above fear-reaction will be constructed as the dynamic fear constraints, respectively reflecting the overall fear cost directly coupled with action-impact, and the dynamic fear boundary of the feasible region derived from different risky neighbors, both of which will in turn serve for the online policy optimization. Experimental results show that Brain-SAD outperforms existing methods, achieving higher success rate in shorter task-completion and collision-recovery time, and exhibits stronger reliability across continuous intersections of fluctuating complexity.

Figures & tables

Fig. 1: Two Constrained RL paradigms for Safe AD: Soft-constraint (Primal-dual) & Hard-constraint.
Fig. 2: Our proposed Brain-SAD safe AD control framework with dynamic fear-oriented constraint ( NOTE : Striatal D1/D2 (StrD1/StrD2) neurons belong to Direct/Indirect pathway in Striatum [ 4 ] , Subthalamic Nucleus (STN) in Hyper-direct pathway [ 5 ] ).
Fig. 3: Current-Scene Imagination for Dual Policy Selection ( NOTE : We use K\boldsymbol{K} parameter-independent yet structurally identical multi-layer MLPs as the Ensemble Environment Model; a multi-layer MLP as the default long-term policy ( Actor ) network).
Fig. 4: Online Long-Term Dynamic Fear-Constrained Policy ( NOTE : We reuse the same K\boldsymbol{K} multi-layer MLPs mentioned in Fig. 3 as the Ensemble Environment Model in Fig. 4 ; a multi-layer MLP as long-term policy ( Actor ) network).
 {Ψst,Atθt−1Fear←φ⋅QCostst,Atθt−1+(1−φ)⋅Fearst,Atθt−1QCostst,Atθt−1=1N∑i=1NQcritic,iCost(st,Atθt−1)Fearst,Atθt−1←{at,iθt−1∈Atθt−1 |β⋅pt+1col,t+L+(1−β)⋅σt+1U,t+L=1K∑k=1K∑t+1t+LWΦ,kMLP(st,at,iθt−1)}st=[Zt1,…,Zti,…,Ztm,veltego,φtego],i∈[1,m]\left\{\begin{aligned} &\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{Fear}\leftarrow\varphi\cdot Q_{Cost}^{s_{t},A_{t}^{\theta_{t-1}}}+(1-\varphi)\cdot Fear_{s_{t},A_{t}^{\theta_{t-1}}}\\ &Q_{Cost}^{s_{t},A_{t}^{\theta_{t-1}}}=\frac{1}{N}\sum_{i=1}^{N}Q_{critic,i}^{Cost}(s_{t},A_{t}^{\theta_{t-1}})\\ &Fear_{s_{t},A_{t}^{\theta_{t-1}}}\leftarrow\left\{a_{t,i}^{\theta_{t-1}}\in A_{t}^{\theta_{t-1}}\ \middle|\beta\cdot p_{t+1}^{col,t+L}+(1-\beta)\cdot\sigma_{t+1}^{U,t+L}=\frac{1}{K}\sum_{k=1}^{K}\sum_{t+1}^{t+L}W_{\Phi,k}^{MLP}(s_{t},a_{t,i}^{\theta_{t-1}})\right\}\\ &s_{t}=[Z_{t}^{1},\ldots,Z_{t}^{i},\ldots,Z_{t}^{m},vel_{t}^{ego},\varphi_{t}^{ego}],i\in[1,m]\end{aligned}\right. (6)
Table 5
 {q∗(Atθt−1∣st)=argmax⁡qEq(Atθt−1∣st)[QRewardst,Atθt−1]s.t.  Eq(Atθt−1∣st)[Ψst,Atθt−1Fear]≤εF,  DKL ⁣[q(Atθt−1∣st)∥πθt−1long−term(Atθt−1∣st)]≤εKL∫q(Atθt−1∣st) dAtθt−1=1\left\{\begin{aligned} &q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})=\operatorname*{argmax}_{q}E_{q(A_{t}^{\theta_{t-1}}\mid s_{t})}[Q_{Reward}^{s_{t},A_{t}^{\theta_{t-1}}}]\\ &\text{s.t.}\;E_{q(A_{t}^{\theta_{t-1}}\mid s_{t})}[\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{Fear}]\leq\varepsilon_{F},\;D_{KL}\!\left[q(A_{t}^{\theta_{t-1}}\mid s_{t})\middle\|\pi_{\theta_{t-1}}^{long-term}(A_{t}^{\theta_{t-1}}\mid s_{t})\right]\leq\varepsilon_{KL}\\ &\int q(A_{t}^{\theta_{t-1}}\mid s_{t})\,dA_{t}^{\theta_{t-1}}=1\end{aligned}\right. (7)
Table 6
 min⁡ L(q,λ,η)⏟Lagrange Objective→q∗(Atθt−1∣st)=πθt−1long−term(Atθt−1∣st)⋅exp⁡ ⁣(QRewardst,Atθt−1−λ⋅Ψst,Atθt−1Fearη)⋅1Z(st)\min\,\underbrace{L(q,\lambda,\eta)}_{\smash{\text{Lagrange Objective}}}\rightarrow q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})=\pi_{\theta_{t-1}}^{long-term}(A_{t}^{\theta_{t-1}}\mid s_{t})\cdot\exp\!\left(\frac{Q_{\text{Reward}}^{s_{t},A_{t}^{\theta_{t-1}}}-\lambda\cdot\Psi_{s_{t},A_{t}^{\theta_{t-1}}}^{\text{Fear}}}{\eta}\right)\cdot\frac{1}{Z(s_{t})} (9)
Table 7
 Lossπθlong-term(θ)=−Est∼Buff[∑i=1Nwt,iθt−1⋅log⁡πθlong-term ⁣(at,iθt∣st)⏟online posterior supervision from  q∗(Atθt−1∣st)+β⋅KLst∼Buff ⁣[πθlong-term ⁣(Atθt∣st)∥πθt−1long-term ⁣(Atθt−1∣st)]]\mathit{Loss}_{\pi_{\theta}}^{\text{long-term}}(\theta)=-E_{s_{t}\sim\mathit{Buff}}\Bigg[\underbrace{\sum_{i=1}^{N}w_{t,i}^{\theta_{t-1}}\cdot\log\pi_{\theta}^{\text{long-term}}\!(a_{t,i}^{\theta_{t}}\mid s_{t})}_{{\text{online posterior supervision from }\,q^{*}(A_{t}^{\theta_{t-1}}\mid s_{t})}}+\beta\cdot KL_{s_{t}\sim\mathit{Buff}}\!\left[\pi_{\theta}^{\text{long-term}}\!(A_{t}^{\theta_{t}}\mid s_{t})\middle\|\pi_{\theta_{t-1}}^{\text{long-term}}\!(A_{t}^{\theta_{t-1}}\mid s_{t})\right]\Bigg] (11)
Table 8
 {Asafe(st)=Gprojset⊗(atrisk)T={Aproj,ebrake}⊗(atsteeratvertical)Gprojset={Aproj,ebrake},  Aproj=(1001),  ebrake=(01)atrisk=(atsteer∈R,atvertical∈R)←πφshort−term(st)\left\{\begin{aligned} &A_{safe}(s_{t})=G_{proj}^{set}\otimes(a_{t}^{risk})^{T}=\{A_{proj},e_{brake}\}\otimes\begin{pmatrix}a_{t}^{steer}\\ a_{t}^{vertical}\end{pmatrix}\\[1.54996pt] &G_{proj}^{set}=\{A_{proj},e_{brake}\},\;A_{proj}=\begin{pmatrix}1&0\\ 0&1\end{pmatrix},\;e_{brake}=\begin{pmatrix}0&1\end{pmatrix}\\[1.54996pt] &a_{t}^{risk}=\bigl(a_{t}^{steer}\in\mathrm{R},a_{t}^{vertical}\in\mathrm{R}\bigr)\leftarrow\pi_{\varphi}^{short-term}(s_{t})\end{aligned}\right. (13)
Table 9
 { ⁣argmin⁡atsafe∈Asafe(st)∥atsafe−atrisk∥,  s.t.  Asafe(st)=Gprojset⊗(atrisk)T⏞reachable action set projection≤Ψst,atFear={τsafe=(−1,1)T,  btreq<0}⏟Brain-Inspired Dynamic Constraintbtreq∈R=min⁡ ⁣(brakereqi,t),  brakereqi,t<0∝Neighbor(st)\left\{\!\begin{aligned} &\operatorname*{argmin}_{a_{t}^{safe}\in A_{safe}(s_{t})}\left\|a_{t}^{safe}-a_{t}^{risk}\right\|,\;\text{s.t.}\;\overbrace{A_{safe}(s_{t})=G_{proj}^{set}\otimes(a_{t}^{risk})^{T}}^{\text{reachable action set projection}}\leq\underbrace{\Psi_{s_{t},a_{t}}^{\text{Fear}}=\left\{\tau_{safe}=(-1,1)^{T},\;b_{t}^{req}<0\right\}}_{\text{Brain-Inspired Dynamic Constraint}}\\ &b_{t}^{req}\in\mathrm{R}=\min\!\left(brake_{req}^{i,t}\right),\;brake_{req}^{i,t}<0\propto Neighbor(s_{t})\end{aligned}\right. (15)
Table 10
 MLE_LossΦk=−E(st,at,st+1,rt,yt)∼Buff[log⁡N(st+1;μ^t+1,(σ^t+1U)2)+λ(ytlog⁡p^t+1col+(1−yt)log⁡(1−p^t+1col))]MLE\_Loss_{\Phi_{k}}=-E_{(s_{t},a_{t},s_{t+1},r_{t},y_{t})\sim Buff}\Big[\log N(s_{t+1};\hat{\mu}_{t+1},(\hat{\sigma}_{t+1}^{U})^{2})+\lambda\Big(y_{t}\log\hat{p}_{t+1}^{col}+(1-y_{t})\log(1-\hat{p}_{t+1}^{col})\Big)\Big] (16)
Table 11
 LΦ={ηNE,γNE,η5−HT,γ5−HT}Selection\displaystyle L_{\begin{subarray}{c}\Phi=\{\eta_{NE},\gamma_{NE},\\ \eta_{5-HT},\gamma_{5-HT}\}\end{subarray}}^{\text{Selection}}=−EBuff[λsafe∥gPFCΦ,t−(1−yt)∥L22+λReward∥gPFCΦ,t−(0.5±tanh⁡(rt))policy_flagt∥L22⏟Policy Selection\displaystyle=-E_{\mathit{Buff}}\Bigl[\underbrace{\lambda_{safe}\|g_{PFC}^{\Phi,t}-(1-y_{t})\|_{L_{2}}^{2}+\lambda_{\text{Reward}}\|g_{PFC}^{\Phi,t}-(0.5\pm\tanh(r_{t}))_{policy\_flag_{t}}\|_{L_{2}}^{2}}_{{\text{Policy Selection}}} (17)
  \displaystyle}{\displaystyle+\underbrace{\lambda_{NE}\|\mathit{Signal}_{NE}^{\Phi,t}-\min(1,1.5\!\cdot\!Fear_{s_{t},a_{t}})\|_{L_{2}}^{2}+\lambda_{5-HT}\|\mathit{Signal}_{5-HT}^{\Phi,t}-\max(0,1-1.5\!\cdot\!Fear_{s_{t},a_{t}})\|_{L_{2}}^{2}}_{{\text{Neurotransmitter Alignment}}}\Bigr] 
Table 12
 {LossR,Criticlong−term=−EBuff[(QReward,Criticlong−term(st,at)−(rt+γ⋅Ea∼πθoldlong−term[QReward,Criticlong−term,Target(st+1,at+1)]))2]LossC,Criticlong−term=−EBuff[(QCost,Criticlong−term(st,at)−(ct+γ⋅Ea∼πθoldlong−term[QCost,Criticlong−term,Target(st+1,at+1)]))2]\left\{\begin{aligned} &Loss_{R,Critic}^{long-term}=-E_{Buff}\Big[\big(Q_{Reward,Critic}^{long-term}(s_{t},a_{t})-\big(r_{t}+\gamma\cdot E_{a\sim\pi_{\theta_{old}}^{long-term}}[Q_{Reward,Critic}^{long-term,Target}(s_{t+1},a_{t+1})]\big)\big)^{2}\Big]\\ &Loss_{C,Critic}^{long-term}=-E_{Buff}\Big[\big(Q_{Cost,Critic}^{long-term}(s_{t},a_{t})-\big(c_{t}+\gamma\cdot E_{a\sim\pi_{\theta_{old}}^{long-term}}[Q_{Cost,Critic}^{long-term,Target}(s_{t+1},a_{t+1})]\big)\big)^{2}\Big]\end{aligned}\right. (18)
Table 13
 min⁡LossBrain-SADOffline=λ1⋅MLE_LossΦk+λ2⋅LΦSelection+λ3⋅LossR/C,CriticLong-term+λ4⋅LossCritic&ActorShort-term\min Loss_{\text{Brain-SAD}}^{\text{Offline}}=\lambda_{1}\cdot MLE\_Loss_{\Phi_{k}}+\lambda_{2}\cdot L_{\Phi}^{\text{Selection}}+\lambda_{3}\cdot Loss_{R/C,\text{Critic}}^{\text{Long-term}}+\lambda_{4}\cdot Loss_{\text{Critic\&Actor}}^{\text{Short-term}} (20)
Table 14
Fig. 5: No traffic light intersection simulation with different complexity (NOTE: red as ego-vehicle; Scene A : Turn left at intersection only with multiple ahead interaction; Scene B : Straight across at intersection with multiple ahead and single aside interaction; Scene C : Straight across at intersection with multiple ahead and multiple aside interaction).
ComboModule CombinationTCT ↓\downarrow (s)SR ↑\uparrow (%)Reward ↑\uparrowSLC ↑\uparrow (%)Comfortable ↑\uparrow (%)
1Brain-SAD-FULL28.50 ±\pm 0.1098.67 ±\pm 0.67105.77 ±\pm 2.5792.05 ±\pm 0.0492.70 ±\pm 0.28
2only State Deviation in Scene Imagination29.20 ±\pm 0.8096.22 ±\pm 0.3895.11 ±\pm 3.0187.29 ±\pm 0.6788.90 ±\pm 0.96
3only Collision Rate in Scene Imagination28.40 ±\pm 0.6098.44 ±\pm 0.7794.43 ±\pm 3.6389.10 ±\pm 0.2890.13 ±\pm 0.03
4w/o Scene Imagination29.40 ±\pm 1.5095.33 ±\pm 1.1588.43 ±\pm 3.5783.27 ±\pm 2.1886.70 ±\pm 0.55
5Brain-SAD only Long Term Policy30.00 ±\pm 2.2097.78 ±\pm 0.3899.32 ±\pm 0.6384.55 ±\pm 2.4389.63 ±\pm 0.08
6w/o Dynamic Fear Cost31.60 ±\pm 1.3096.67 ±\pm 1.3388.39 ±\pm 0.1781.35 ±\pm 2.0985.40 ±\pm 0.34
TABLE I: The Ablation Study of Brain-SAD on Scene A ( NOTE: all the performances are averaged by three rounds).
ComboModule CombinationTCT ↓\downarrow (s)SR ↑\uparrow (%)Reward ↑\uparrowSLC ↑\uparrow (%)Comfortable ↑\uparrow (%)
1Brain-SAD-FULL22.30 ±\pm 0.1097.56 ±\pm 0.3894.35 ±\pm 0.0588.91 ±\pm 0.1789.97 ±\pm 1.82
2only State Deviation in Scene Imagination27.10 ±\pm 0.1097.33 ±\pm 0.6789.91 ±\pm 0.2486.40 ±\pm 0.4987.00 ±\pm 0.40
3only Collision Rate in Scene Imagination28.20 ±\pm 0.0096.67 ±\pm 1.1586.33 ±\pm 0.2182.80 ±\pm 0.3186.30 ±\pm 1.70
4w/o Scene Imagination28.60 ±\pm 0.6096.00 ±\pm 0.6780.56 ±\pm 0.7880.60 ±\pm 1.0983.00 ±\pm 0.40
5Brain-SAD only Long Term Policy29.80 ±\pm 0.1097.33 ±\pm 1.1587.78 ±\pm 0.2886.50 ±\pm 1.8085.60 ±\pm 0.50
6w/o Dynamic Fear Cost29.80 ±\pm 0.6093.56 ±\pm 0.7782.20 ±\pm 1.7076.20 ±\pm 1.8380.80 ±\pm 0.69
TABLE II: The Ablation Study of Brain-SAD on Scene B ( NOTE: all the performances are averaged by three rounds).
NOCategoryMethodsTCT ↓\downarrow (s)RECT ↓\downarrow (s)SR ↑\uparrow (%)Reward ↑\uparrowSLC ↑\uparrow (%)Comfortable ↑\uparrow (%)Median-TTC ↑\uparrow (s)
1Policy-Oriented ConstrainedCADRE36.30 ±\pm 0.600.7181 ±\pm 0.003293.56 ±\pm 0.8385.45 ±\pm 0.4482.35 ±\pm 0.5183.34 ±\pm 0.720.7162 ±\pm 0.0045
2Iso-Dream++30.20 ±\pm 0.200.6245 ±\pm 0.005797.29 ±\pm 0.1386.23 ±\pm 0.8286.12 ±\pm 0.1087.52 ±\pm 0.231.1387 ±\pm 0.0102
3RLfOLD33.20 ±\pm 0.800.5661 ±\pm 0.031092.00 ±\pm 0.9482.88 ±\pm 1.8083.52 ±\pm 0.3784.75 ±\pm 2.111.0266 ±\pm 0.0053
4MARL-CCE31.60 ±\pm 0.400.8417 ±\pm 0.033592.67 ±\pm 1.1081.05 ±\pm 0.0281.72 ±\pm 2.5881.16 ±\pm 0.040.7340 ±\pm 0.0057
5Environment-Oriented Prior ConstrainedLS-Imagination31.20 ±\pm 0.600.6547 ±\pm 0.012596.03 ±\pm 0.0084.57 ±\pm 0.0285.47 ±\pm 1.1185.77 ±\pm 0.061.0934 ±\pm 0.0013
6CarPlanner33.10 ±\pm 0.100.8333 ±\pm 0.047195.57 ±\pm 0.5784.37 ±\pm 0.1485.30 ±\pm 1.7982.20 ±\pm 0.010.8956 ±\pm 0.0060
TABLE III: The Comparative Study of Brain-SAD on Scene B ( NOTE: the bold font represents the most optimal performance in each column, with the colored as the sub-optimal performance in each category. ALL the performance are averaged by three rounds).
Fig. 6: The training traces and action distributions on representative methods selected from Table 3 , simulated in Scene B . ( NOTE : In ( a )–( d ), we present training traces on methods with TOP 2 Reward reported in Table 3 ; Here, the Reward are the average cumulative reward on the best training round (150 episodes, 3 rounds in all); In ( e )–( h ), we present action distributions across the best episode (selected in the above best training round) on methods with TOP 1 Reward reported in Table 3 , X-axis for acceleration positive as speeding, Y-axis for steering angle, Z-axis for temporal steps in the best episode.)
TABLE V: The performance of Brain-SAD on continuous Stress Test across 6 intersections in Fig. IV-D ( NOTE: the bold font represents the most optimal performance in each column, with the colored as the sub-optimal in each column. ALL the performance are averaged by three rounds).
Fig. 10: The driving trace of the ego-vehicle when passing 6 continuous intersections controlled by FNI-RL ( a – f ) and Brain-SAD ( g – l ) in the same best episode used in Fig. 9 ( e )–( h ).
Fig. 11: Environmental adaptation comparison among Brain-SAD, SVPG, FNI-RL and EFT-RL. ( NOTE: Simulated in the 6 continuous intersections in Fig. 8, we compare: ( a ) . Perceptional fear-signals between Brain-SAD and FNI-RL; ( b ) . Policy selection on Brain-SAD based on the fined-grained signals on calmness and nervousness ; ( c ) . Policy adaptability by step-wise cumulative reward across continuous intersections trained in the best episode).

Explore similar work

CardsList

  1. Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints

    Mar 6, 2026Bo Leng, Weiqi Zhang, Zhuoren Li +4Autonomous RacingAutonomous Driving

    College of Automotive and Energy Engineering, Tongji University, Shanghai, 201804, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, 639798, Singapore

  2. Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraints

    May 13, 2026Timofey TomashevskiySafety ConstraintsOffline Reinforcement Learning

    McMaster Centre for Software Certification Department of Computing and Software McMaster University, Canada

  3. From Cumulative Constraints to Adaptive Runtime Safety Control for Nonstationary Reinforcement Learning

    May 13, 2026Timofey TomashevskiySafety ConstraintsReinforcement Learning Control

    McMaster Centre for Software Certification Department of Computing and Software McMaster University, Canada