SAFTER: Safety-Aware Filtering Through Explicit Recoverability for Robust Reinforcement Learning under Distribution Shift

Benedictus C. G. Cinun; Bin Hu

eess.SYOct 21, 2026

SAFTER: Safety-Aware Filtering Through Explicit Recoverability for Robust Reinforcement Learning under Distribution Shift

Authors: Benedictus C. G. Cinun, Bin Hu

Organizations: Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA · Department of Engineering Technology, University of Houston, Houston, TX 77004, USA

Abstract

Learning safe policies that generalize to unseen environments is difficult under bounded actuation, where an unsafe decision may become unrecoverable. Conventional control barrier function (CBF) filters enforce only instantaneous safety and ignore future recoverability. We propose SAFTER (Safety-Aware Filtering Through Explicit Recoverability), which embeds a differentiable, closed-form backup-CBF filter into Soft Actor-Critic training. SAFTER unifies current safety, finite-horizon backup safety, terminal recoverability, and input admissibility into a single differentiable certificate. Its CBF condition yields an exact minimum-intervention projection without iterative online optimization. In quadruped navigation, SAFTER reaches 90% training success with 30.6% fewer transitions than Vanilla SAC and 34.1-35.1% fewer than current-state CBF-filtered SAC baselines. Across unseen simulated environments, SAFTER improves success by up to 42.4 and 61.4 percentage points over the state-of-the-art methods CBF-RL and SEA-Nav, respectively, and achieves 73.3% and 76.7% success on two real-world navigation maps. Ablations identify accurate differentiation through the safety filter as a key contributor to robustness, in which replacing the projection Jacobian with the identity approximation reduces success rate in a High-Shift environment from 61.4% to 6.2%. These results show that recoverability-aware filtering serves as both a runtime safety mechanism and a learning mechanism for sample-efficient, robust policies.

Figures & tables

Figure 1: Overview of SAFTER. A task policy proposes a nominal command; a prescribed backup rollout evaluates finite-horizon recoverability; current, future, terminal, and input-admissibility conditions are combined into one differentiable certificate; and an analytical minimum-intervention projection determines the safety correction before execution. During training, the critic evaluates the executed action and the actor differentiates through the same safety map.

Figure 2: Training success rates of Vanilla SAC, CBF–QP/CBF–CF, bCBF–LSE–QP, and SAFTER under the same training environments.

Panel of Figure 3 — Figure 3: Real-world map with two different obstacle configurations, and trajectory sample and LiDAR readings taken from the Unitree Go2 on-board state estimator and LiDAR.

	Filter disabled			Filter enabled
Method	SR $\uparrow$	TR $\downarrow$	FR $\downarrow$	SR $\uparrow$	TR $\downarrow$	FR $\downarrow$
SEA–Nav 1	$89.7{\pm}2.4$	$0.0{\pm}0.0$	$10.3{\pm}2.4$	$91.2{\pm}2.9$	$0.0{\pm}0.0$	$8.8{\pm}2.9$
CBF–RL 2	$84.3{\pm}8.2$	$5.9{\pm}5.2$	$9.8{\pm}7.1$	$84.8{\pm}7.4$	$5.9{\pm}5.2$	$9.3{\pm}6.6$
CBF–CF 3	$76.6{\pm}13.8$	$2.7{\pm}6.3$	$20.7{\pm}15.8$	$90.1{\pm}8.2$	$1.9{\pm}4.5$	$8.0{\pm}8.5$
SAFTER 4	$\mathbf{95.1{\pm}7.1}$	$0.0{\pm}0.0$	$\mathbf{4.9{\pm}7.1}$	$\mathbf{99.7{\pm}1.2}$	$0.0{\pm}0.0$	$\mathbf{0.3{\pm}1.2}$

Table 1: Performance on the training-map distribution with each method’s runtime safety filter disabled or enabled. SR, TR, and FR denote success, timeout, and failure rates, respectively. Values are mean

\pm

standard deviation over evaluation seeds and maps.

	Batch = 1			Batch = 100
Method	Total (ms)	Synthesis (ms)	Speedup ( $\times$ )	Total (ms)	Synthesis (ms)	Speedup ( $\times$ )
bCBF–QP	$30.546{\pm}0.324$	$28.369{\pm}0.194$	$1.00$	$2504.789{\pm}22.463$	$2487.383{\pm}16.949$	$1.00$
bCBF–LSE–QP	$5.732{\pm}0.456$	$3.342{\pm}0.405$	$5.33$	$7.084{\pm}0.569$	$4.588{\pm}0.481$	$353.58$
SAFTER	$\mathbf{2.495{\pm}0.312}$	$\mathbf{0.102{\pm}0.011}$	$\mathbf{12.24}$	$\mathbf{2.529{\pm}0.283}$	$\mathbf{0.106{\pm}0.014}$	$\mathbf{990.43}$

Table 2: Safety-filter runtime comparison. Total latency includes certificate construction and synthesis, and speedup is measured relative to the direct bCBF–QP. Values are mean

\pm

standard deviation over 300 timed repetitions.

Figure 4: Representative trajectories in the simulated Medium-Shift environment. Purple and yellow regions denote the initial and goal regions, squares mark initial positions, stars successful terminations, crosses collision terminations, and faint circles indicate the robot footprint.

Ablation Setting		Low-Shift			Medium-Shift			High-Shift
Projection Jacobian	Correction Reward	SR $\uparrow$	TR $\downarrow$	FR $\downarrow$	SR $\uparrow$	TR $\downarrow$	FR $\downarrow$	SR $\uparrow$	TR $\downarrow$	FR $\downarrow$
Exact	On	$\mathbf{98.8{\pm}2.2}$	$0.0{\pm}0.0$	$1.2{\pm}2.2$	$88.6{\pm}7.3$	$0.0{\pm}0.0$	$11.4{\pm}7.3$	$61.4{\pm}10.2$	$0.0{\pm}0.0$	$38.6{\pm}10.2$
Identity	On	$89.8{\pm}6.0$	$0.0{\pm}0.0$	$10.2{\pm}6.0$	$41.6{\pm}9.3$	$0.0{\pm}0.0$	$58.4{\pm}9.3$	$6.2{\pm}4.4$	$0.0{\pm}0.0$	$93.8{\pm}4.4$
Exact	Off	$95.6{\pm}4.9$	$4.2{\pm}4.9$	$\mathbf{0.2{\pm}1.0}$	$\mathbf{94.4{\pm}4.4}$	$3.8{\pm}3.9$	$\mathbf{1.8{\pm}3.5}$	$\mathbf{76.2{\pm}9.2}$	$3.2{\pm}3.8$	$\mathbf{20.6{\pm}8.3}$
Identity	Off	$76.8{\pm}7.1$	$0.0{\pm}0.0$	$23.2{\pm}7.1$	$9.0{\pm}6.1$	$0.0{\pm}0.0$	$91.0{\pm}6.1$	$2.0{\pm}3.5$	$0.0{\pm}0.0$	$98.0{\pm}3.5$

Table 4: Ablation of the projection Jacobian and correction reward on unseen environments. Exact and Identity use the same forward safety-filter map and differ only in the backward differentiation through the analytical projection. Values are mean

\pm

standard deviation across 25 evaluation-seed rates.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Five hand-crafted training maps with fixed obstacle configurations and predefined initial and goal regions. Green and yellow regions indicate the initial and goal regions, respectively, while obstacles are shown in red.

Panel of Figure 6 — Figure 6: Static unseen environments used for distribution-shift evaluation. Green and yellow regions indicate the admissible initial and goal regions, respectively, while obstacles are shown in red. The environments introduce increasingly different obstacle configurations relative to the hand-crafted training maps.

	Low-Shift		Medium-Shift		High-Shift
Method	PL (m) $\downarrow$	NT (s) $\downarrow$	PL (m) $\downarrow$	NT (s) $\downarrow$	PL (m) $\downarrow$	NT (s) $\downarrow$
SEA–Nav 1	$11.57{\pm}0.17$	$13.70{\pm}0.17$	$10.95{\pm}0.30$	$14.21{\pm}0.27$	$--$	$--$
CBF–RL 2	$14.86{\pm}0.78$	$32.02{\pm}4.02$	$14.93{\pm}0.76$	$34.93{\pm}4.3$	$15.72{\pm}1.19$	$38.99{\pm}8.45$
Vanilla SAC 3	$12.76{\pm}0.41$	$26.75{\pm}1.54$	$12.38{\pm}0.68$	$26.31{\pm}2.13$	$12.20{\pm}0.70$	$25.97{\pm}1.77$
CBF–QP 4	$12.05{\pm}0.38$	$25.08{\pm}1.34$	$11.50{\pm}0.45$	$23.77{\pm}1.55$	$11.82{\pm}1.11$	$25.42{\pm}4.02$
CBF–CF 4	$12.26{\pm}0.36$	$26.85{\pm}1.06$	$11.72{\pm}0.42$	$24.84{\pm}1.22$	$11.72{\pm}0.35$	$25.93{\pm}2.06$

Appendix

Table 5: Navigation efficiency on the unseen static environments with the runtime safety filters enabled. Path Length (PL) and Navigation Time (NT) are computed over successful episodes and reported as mean

\pm

standard deviation. Dashes indicate that no successful episodes were available for computing PL and NT.

Control and sensing
Parameter	Symbol	Value
LiDAR update rate	–	$10\,\mathrm{Hz}$
Low-level control rate	–	$50\,\mathrm{Hz}$
LiDAR field of view	–	$360^{\circ}$
Number of LiDAR rays	$N_{h}$	$120$
LiDAR range limits	$\rho_{\min},\rho_{\max}$	$0.05,\ 5.0\,\mathrm{m}$

Appendix

Table 6: Safety-filter and local-navigation parameters used by the verified implementation.

Component	Definition	Weight
Goal progress	$r_{t}^{\mathrm{prog}}=d_{t-1}-d_{t}$	$+4.00$
Goal reached	$r_{t}^{\mathrm{goal}}=\mathbb{I}[d_{t}\leq d_{\mathrm{goal}}]$	$+30.00$
Time	$r_{t}^{\mathrm{time}}=1$	$-0.01$
Timeout	$r_{t}^{\mathrm{timeout}}=\mathbb{I}[\mathcal{T}_{t}]$	$-5.00$
Forward alignment	$r_{t}^{\mathrm{align}}=\dfrac{\operatorname{ReLU}(v_{x,t}^{\mathrm{meas}})}{v_{x,\max}}\cos\beta_{t}$	$+0.20$
Reverse motion	$r_{t}^{\mathrm{rev}}=\left(\dfrac{\operatorname{ReLU}(-v_{x,t}^{\mathrm{meas}})}{v_{x,\max}}\right)^{2}$	$-0.50$

Appendix

Table 7: High-level reward components and signed weights. The correction weight is

-0.05

by default and zero for Vanilla SAC and correction-reward-off ablations.

Safe RL

Safe reinforcement learning studies how agents can learn and act while respecting safety constraints and limiting harmful outcomes.

See all 217 papers

FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems
Oct 8, 2026Songyuan Zhang, Baljeet Singh, Sarthak Ranjeet Kaingade +2Safe RL Constrained RL
Massachusetts Institute of Technology · Amazon
Safe Reinforcement Learning with Receding-Horizon Safety Critic and Differentiable Action Shields
Oct 14, 2026Siwei Ju, Jan Peters, Oleg ArenzSafe RL Safety Filtering
Department of Computer Science, Technical University of Darmstadt, Germany. · Hessian.AI. · German Research Center for AI (DFKI), Research Department: Systems AI for Robot Learning. +1
Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning
May 26, 2026Dhruv S. Kushwaha, Zoleikha A. BironKoopman Operator Learning Control Barrier Functions

Oct 8, 2026J/K move · Enter open · S save

FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems

Songyuan Zhang, Baljeet Singh, Sarthak Ranjeet Kaingade, Chuchu Fan+1

Massachusetts Institute of Technology · Amazon

Safe RL Constrained RL Robot Safety

Open paper AskPDF arXiv

Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates. Safety filters separate them at action execution, but classical designs require an analytic safety function and dynamics model, and standard minimal-intervention filters are myopic to long-horizon task return because they minimize only instantaneous action deviation. Hard projections are also undefined when no safe action exists. We present FAITH, a feasibility-aware, model-free framework that approximates the optimal state-action safety value and amortizes minimal-intervention filtering with a feedforward network. The task policy optimizes the task return through the filtered dynamics, which recovers the feasible constrained problem without a competing safety term in the task-policy update. When no action satisfies the learned safety condition, the same filter approaches the action with minimum predicted peak harm. On a double integrator example and a Safety Gym environment, FAITH achieves the highest return among methods with no feasible-start violations and matches the lowest harm from infeasible starts. On a 29-DoF humanoid, it reaches a 99.95% safety rate while retaining 97% of the unfiltered return in Walking-Avoid, and obtains the highest measured safety rate in Push-Avoid by learning to sacrifice balancing and fall away from the protected region. The same policies are also demonstrated on a real-world Unitree G1 humanoid.

Abstract

Safe RL

Explore similar work

FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems

Safe Reinforcement Learning with Receding-Horizon Safety Critic and Differentiable Action Shields

Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning