How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints

Kensuke Nakamura; Arun L. Bishop; Steven Man; Aaron M. Johnson; Zachary Manchester; Andrea Bajcsy

cs.RONov 23, 2025

How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints

Authors: Kensuke Nakamura, Arun L. Bishop, Steven Man, Aaron M. Johnson, Zachary Manchester, Andrea Bajcsy

Organizations: Carnegie Mellon University

Abstract

Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling safe visuomotor control under hard-to-model constraints. However, existing methods implement "least-restrictive" filtering that discretely switch between nominal and safety policies, potentially undermining the task performance that makes modern visuomotor policies valuable. While reachability value functions can, in principle, be adapted to be control barrier functions (CBFs) for smooth optimization-based filtering, we theoretically and empirically show that current latent-space learning methods produce fundamentally incompatible value functions. We identify two sources of incompatibility: First, in HJ reachability, failures are encoded via a "margin function" in latent space, whose sign indicates whether or not a latent is in the constraint set. However, representing the margin function as a classifier yields saturated value functions that exhibit discontinuous jumps. We prove that the value function's Lipschitz constant scales linearly with the margin function's Lipschitz constant, revealing that smooth CBFs require smooth margins. Second, reinforcement learning (RL) approximations trained solely on safety policy data yield inaccurate value estimates for nominal policy actions, precisely where CBF filtering needs them. We propose the LatentCBF, which addresses both challenges through gradient penalties that lead to smooth margin functions without additional labeling, and a value-training procedure that mixes data from both nominal and safety policy distributions. Experiments on simulated benchmarks and hardware with a vision-based manipulation policy demonstrate that LatentCBF enables smooth safety filtering while doubling the task-completion rate over prior switching methods.

Figures & tables

Figure 1: Safety Filtering a Visuomotor Manipulation Policy. Both the nominal policy and the safety filters take as input the RGB images shown on the bottom. (left) Unfiltered nominal diffusion policy spills the bag’s contents. (center) Least-restrictive latent safety filter prevents spilling, but also stops the diffusion policy from lifting the bag. (right) Our LatentCBF guides the diffusion policy to a safe grasp and it completes the pickup task.

Figure 2: CBFs as a Function of the Margin Function. Even with a perfect model, a classifier-based $\ell(s)$ yields a CBF with poor signal during action filtering (left). A smooth margin function provides a rich signal for the CBF to evaluate alternative actions (right).

Safety Filter	NoGP		GP		None
Safety Filter	Avg. ${\|\Delta a\|}$	Safety Rate	Avg. ${\|\Delta a\|}$	Safety Rate	Safety Rate
LR	$1.5\pm 1.1$	$97\%$	$2.0\pm 1.3$	$100\%$	$41\%$
CBF	$1.3\pm 1.0$	$97\%$	$1.1\pm 0.9$	$100\%$	$41\%$

Table 1: Simulation: On the Quality of Latent Safety Filters. CBFs trained with gradient-penalties yield smoother interventions, decreasing the average override magnitude relative to LR by

45\%

compared to only

13\%

for the CBF without gradient penalties.

Figure 3: Hardware: End-effector Trajectories in the X-Z Plane. LatentCBF maintains control authority and safety compared to LR and unfiltered baselines. Without fitting the critic as in Sec. 5 , LatentCBF-NoMix permits erratic actions and spills in 20 $\%$ of trials.

Safety Filter	$\ell{{}_{\mu}}(z)$	Init Cond 1 (%)			Init Cond 2 (%)			Init Cond 3 (%)			Aggregate (%)
Safety Filter	$\ell{{}_{\mu}}(z)$	Fail	Stall	Success	Fail	Stall	Success	Fail	Stall	Success	Fail	Stall	Success
None	–	100	0	0	0	0	100	85	0	15	62	0	38
LR	GP	0	90	10	0	0	100	0	95	5	0	62	38
CBF	No GP	0	100	0	0	0	100	0	85	15	0	62	38
CBF	GP	0	0	100	0	0	100	0	60	40	0	20	80

Table 2: Hardware: Safely Guiding a Diffusion Policy. Results are averaged over three initial conditions with 20 trials each. All safety filters prevent spilling, but LR and LatentCBF-NoGP do so at the cost of task performance, whereas LatentCBF more consistently steers the policy toward both safe and successful executions.

	Filtering Speed (ms) as a Function of # Samples
# Samples	10	30	50	3,800	7,600
Model-based	40.70 $\pm$ 0.76	114.6 $\pm$ 2.6	out-of-mem	out-of-mem	out-of-mem
Model-free	0.33 $\pm$ 0.12	0.32 $\pm$ 0.08	1.85 $\pm$ 0.32	3.96 $\pm$ 0.19	9.60 $\pm$ 0.57

Table 3: Hardware: LatentCBF Filtering Speed. Model-based filtering is infeasible for over

50

actions (out of memory). Model-free can evaluate thousands of actions in

10

ms.

Hyperparameter	Value
RGB Image Dimension	[128, 128, 3]
IR Image Dimension	[128, 128, 1]
Action Dimension	1
Stochastic Latent	Gaussian
Latent Dim (Deterministic)	512
Latent Dim (Stochastic)	32

Table 4: Dubins’ car RSSM Hyperparameters

Hyperparameter	Values
State Normalization	Yes
Action Normalization	Yes
Action Chunk	16
Image Chunk	2
Image Size	256
Batch size	100

Table 5: Hyperparameters for Diffusion Policy

Hyperparameter	Values
Image size	256
DINOv3 patch size	$(16\times 16,384)$
Optimizer	AdamW
Predictor lr	5e-5
Decoder lr	3e-4
Action Encoder lr	5e-4

Table 6: Hyperparameters for DINO-WM

Hyperparameter	Value
Actor Architecture	[512, 512, 512]
Critic Architecture	[512, 512, 512]
Normalization	LayerNorm
Activation	ReLU
Discount Factor $\gamma$	0.995
Learning Rate (Critic)	3e-4

Table 7: RL hyperparameters.

$\ell{{}_{\mu}}(z)$	TP (%)	TN (%)	FP (%)	FN (%)	$\max_{t}\Delta_{t}\ell(z_{t})$
NoGP	84	13	0.63	2.5	1.2 $\pm$ 0.76
GP	86	13	1.0	0.60	0.17 $\pm$ 0.065

Table 8: Simulation: Accuracy & Smoothness of Latent

\ell{{}_{\mu}}(z)

. In the Dubins’ car simulation, we compare the latent failure set encoded via

\mathcal{F}_{z}=\{z:\ell{{}_{\mu}}(z)<0\}

to the ground-truth failure set,

\mathcal{F}\subset\mathcal{S}

, as well as the maximum

\ell{{}_{\mu}}(z)

gradient along trajectories generated by

\pi^{\text{nom}}

with no filtering, averaged across 100 initial conditions.

Safety Filter	NoGP		GP		None
Safety Filter	Avg. ${\|\Delta a\|}$	Safety Rate	Avg. ${\|\Delta a\|}$	Safety Rate	Safety Rate
LR	$1.5\pm 1.1$	$97\%$	$2.0\pm 1.3$	$100\%$	$41\%$
CBF ( $\alpha=0.7$ )	$1.3\pm 1.1$	$96\%$	$1.3\pm 1.1$	$100\%$
CBF ( $\alpha=0.95$ )	$1.3\pm 1.0$	$97\%$	$1.1\pm 0.9$	$100\%$

Table 9: Simulation: On the Quality of Latent Safety Filters.

Figure 5: Sensitivity Study: Margin magnitude.

Figure 6: Sensitivity Study: Classification Accuracy.

2	$[\mathbf{x},\mathbf{y},\mathbf{z}]$
#	Interp. Dims	Interp. From	Interp. To	Static
1	$[\mathbf{x},\mathbf{y},\mathbf{z},\mathbf{\omega_{x}},\mathbf{\omega_{y}},\mathbf{\omega_{z}}]$	$\pi^{\text{nom}}$	$\pi^{\text{\tiny{\faIcon[*]{shield-alt}}}}$	$\pi^{\text{nom}}$
3	$[\mathbf{\omega_{x}},\mathbf{\omega_{y}},\mathbf{\omega_{z}}]$
4	$[\mathbf{x}]$
5	$[\mathbf{y}]$
6	$[\mathbf{z}]$

Table 10: Scheme for generating action samples for the manipulator. Each linear interpolation takes

N=400

samples for a total of

7600

samples. Each line is interpolated by first deciding which set of actions to interpolate from and to, as well as which dimensions participate in the interpolation; while the rest are held static. Bounds

\bm{\upmu+\upsigma}

and

\bm{\upmu-\upsigma}

represent actions

+1

and

-1

standard deviation away from the mean actions from the dataset.

Explore similar work

Oct 16, 2025

CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions

Reinforcement learning (RL), while powerful and expressive, can often prioritize performance at the expense of safety. Yet safety violations can lead to catastrophic outcomes in real-world deployments. Control Barrier Functions (CBFs) offer a principled method to enforce dynamic safety -- traditionally deployed online via safety filters. While the result is safe behavior, the fact that the RL policy does not have knowledge of the CBF can lead to conservative behaviors. This paper proposes CBF-RL, a framework for generating safe behaviors with RL by enforcing CBFs in training. CBF-RL has two key attributes: (1) minimally modifying a nominal RL policy to encode safety constraints via a CBF term, (2) and safety filtering of the policy rollouts in training. Theoretically, we prove that continuous-time safety filters can be deployed via closed-form expressions on discrete-time roll-outs. Practically, we demonstrate that CBF-RL internalizes the safety constraints in the learned policy -- both enforcing safer actions and biasing towards safer rewards -- enabling safe deployment without the need for an online safety filter. We validate our framework through ablation studies on navigation tasks and on the Unitree G1 humanoid robot, where CBF-RL enables safer exploration, faster convergence, and robust performance under uncertainty, enabling the humanoid robot to avoid obstacles and climb stairs safely in real-world settings without a runtime safety filter.

Lizhi Yang, Blake Werner, Massimiliano de Sa +1

Caltech MCE

RL for Robotics Control Barrier Functions Constrained RL Safety Filtering Safe RL

Ask PDF

May 26, 2026

Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning

Safe reinforcement learning (RL) for robotic systems requires policies that improve task performance while satisfying state and input constraints during both training and deployment. Control barrier functions (CBFs) provide a principled mechanism for enforcing forward invariance through minimally invasive safety filters, but their use in model-free RL is limited by the need for accurate dynamics and hand-designed barrier certificates. We propose Robust Koopman-CBF SAC, a safety-filtered actor--critic framework that learns a finite-dimensional Koopman predictor from data, constructs affine CBF constraints in the lifted space, and enforces them through a quadratic-program safety layer. To account for finite-dimensional Koopman approximation error, the CBF condition is tightened using a projected residual margin estimated from held-out rollout data. The critic is trained on the executed safe action, while the actor is regularized toward the Koopman-CBF feasible set, reducing dependence on the filter over training. Across safe-control benchmarks, the method achieves zero constraint violations on CartPole stabilization and tracking while matching or exceeding unconstrained SAC returns. On high-dimensional Safety Gymnasium locomotion tasks, the method reduces violations in some settings but also exposes important limitations of first-order velocity barriers and linear EDMD models, motivating high-order and multi-step Koopman-CBF extensions. These results suggest that robust Koopman-CBF filters are a promising bridge between model-free RL and certifiable safety, while clarifying the structural conditions under which such filters remain effective.

Dhruv S. Kushwaha, Zoleikha A. Biron

Koopman Operator Learning Control Barrier Functions Safety Filtering Safe RL Robust RL

Ask PDF

Oct 6, 2026

LBA-CBF: Rapidly Adaptive Safety Filters via Parallel Dynamics Inference

Control barrier functions (CBFs) certify commands through an assumed dynamics model, so an abrupt, unmeasured regime change can undermine the certificate exactly when safety matters most. We present Look-Back Adaptive Control Barrier Functions (LBA-CBF), which rank a finite bank of candidate dynamics by recent prediction error over a short look-back window and enforce the high-order CBF condition against every model within a tolerance of the best, spanning best-fit adaptation to full-bank robust filtering. The dynamics may depend nonlinearly on the unknown parameters, and no switching model or continuously parameterized estimator is required. We prove that any feasible filtered input satisfies the true CBF condition whenever a safety-representative candidate is retained. In quadrotor simulation with abrupt wind reversals and an unknown payload, LBA-CBF is safe and reaches the goal from all random initial conditions, matching an oracle, while adaptive and robust baselines achieve 0-88% success. Banks of up to 250,000 models run inside the control loop, and Crazyflie 2.1 and F1TENTH experiments demonstrate adaptation to wind, payload release, and varying tire-road friction. Code, videos, and project details are available at: https://lla-control.github.io

Maitham F. AL-Sunni, Timeea-Andreea Radu, Hassan Almubarak +4

Department of Electrical & Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA · School of Computer Science & Engineering, Constructor University, Bremen, Germany · Control & Instrumentation Engineering Department, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia +1

Safety-Critical Control Control Barrier Functions Robust Control

Ask PDF