Safety-critical Control Under Partial Observability: Reach-Avoid POMDP meets Belief Space Control

Matti Vahs; Joris Verhagen; Jana Tumova

cs.ROMar 11, 2026

Safety-critical Control Under Partial Observability: Reach-Avoid POMDP meets Belief Space Control

Authors: Matti Vahs, Joris Verhagen, Jana Tumova

Organizations: Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden · Digital Futures

Abstract

Partially Observable Markov Decision Processes (POMDPs) provide a principled framework for robot decision-making under uncertainty. Solving reach-avoid POMDPs, however, requires coordinating three distinct behaviors: goal reaching, safety, and active information gathering to reduce uncertainty. Existing online POMDP solvers attempt to address all three within a single belief tree search, but this unified approach struggles with the conflicting time scales inherent to these objectives. We propose a layered, certificate-based control architecture that operates directly in belief space, decoupling goal reaching, information gathering, and safety into modular components. We introduce Belief Control Lyapunov Functions (BCLFs) that formalize information gathering as a Lyapunov convergence problem in belief space, and show how they can be learned via reinforcement learning. For safety, we develop Belief Control Barrier Functions (BCBFs) that leverage conformal prediction to provide probabilistic safety guarantees over finite horizons. The resulting control synthesis reduces to lightweight quadratic programs solvable in real time, even for non-Gaussian belief representations with dimension $>10^4$ . Experiments in simulation and on a space-robotics platform demonstrate real-time performance and improved safety and task success compared to state-of-the-art constrained POMDP solvers.

Figures & tables

Fig. 1: Illustration of our hardware experiments on a space-robotics platform. The robot starts from an unknown initial position that is located somewhere in the initial belief $\bm{b}_{0}$ and has to navigate to the goal region $\mathcal{S}_{g}$ without entering the avoid region $\mathcal{S}_{a}$ . The robot can localize itself in a map by detecting impacts with the walls. Two runs with different initial conditions are shown.

Fig. 2: Illustration of our proposed control architecture on the example of a free floating robot platform that uses impact detections as measurements. The robot uses measurements $\bm{z}$ to update a particle filter belief $\bm{b}$ at time $t$ . Based on the current belief, the reference controller obtains an input based on the mean state $\bm{\mu}$ , the belief CLF serves as information gathering controller and the belief CBF minimally corrects all unsafe control inputs.

Fig. 3: Illustration of an example with a one dimensional state space. In this environment, the state can be accurately localized in a sensing region

\{x\in\mathbb{R}\mid|x|\leq 0.1\}

around the origin. The true state evolves according to a continuous-time SDE. The bottom plot shows the proposed uncertainty quantification as well as the belief empirical standard deviation over time.

Fig. 4: Example visualization of a belief CLF. The blue surface depicts the Lyapunov function with the current belief shown in grey. The maximum information gathering control input is the steepest direction on the BCLF, the reference control input points in a direction where the BCLF value increases and the optimized information gathering control input moves in the direction of the reference while enforcing convergence to the set of localized beliefs.

Fig. 5: Visualization of the network architecture used to learn a belief control lyapunov function. The encoder and MLP are denoted by

e_{\psi}

and

g_{\theta}

with learnable parameters

\psi

and

\theta

, respectively.

\rho^{(i)}=-\xi^{(i)},\quad\forall i=1,\dots,N

Algorithm 1 Horizon BCBF Safety Filter

	CPOMCPOW		CPFT-DPW		$\bm{u}^{\mathrm{ref}}$	$\bm{u}^{\mathrm{ref}}$ +BCBF	$\bm{u}^{\mathrm{ref}}$ +BCLF	Ours
Constrained Lightdark
Reach $\uparrow$	0.72 ( $\delta t{=}1$ )	0.52 ( $\delta t{=}0.1$ )	0.9 ( $\delta t{=}1$ )	0.66 ( $\delta t{=}0.1$ )	0.51	0.51	1.0	0.99
Avoid $\uparrow$	0.69 ( $\delta t{=}1$ )	0.86 ( $\delta t{=}0.1$ )	0.35 ( $\delta t{=}1$ )	0.79 ( $\delta t{=}0.1$ )	1.0	1.0	0.03	1.0
SR $\uparrow$	0.4 ( $\delta t{=}1$ )	0.47 ( $\delta t{=}0.1$ )	0.35 ( $\delta t{=}1$ )	0.56 ( $\delta t{=}0.1$ )	0.51	0.51	0.03	0.99
Constrained Antenna
Reach $\uparrow$	0.0 ( $\delta t{=}0.1$ )		0.0 ( $\delta t{=}0.1$ )		0.53	0.05	0.97	0.95

TABLE I: Quantitative Evaluation across environments. For the MCTS baselines,

\delta t

refers to the planning time step.

Fig. 6: Illustration of the three reach-avoid POMDPs considered in simulation. Goal regions $\mathcal{S}_{g}$ are shown in green, avoid regions $\mathcal{S}_{a}$ are shown in red and the particle filter belief is shown in blue.

Fig. 7: Illustration of all baselines in the Constrained Antenna environment in which the antenna location is indicated by a red cross, the goal region is shown in green and the avoid set is shown in red. In this run, only our proposed control architecture is able to reach the goal without entering the set of unsafe states.

Fig. 8: Illustration of the proposed control architecture (top) and the switching controller (bottom) in the Constrained Bumper environment. Our method first gathers information by bumping into the left wall and then continues towards the goal as uncertainty can be reduced by bumping into the top wall. The switching controller moves south after the first bump resulting in overall longer path length.

Panel of Figure 11 — Fig. 9: Box plots showing median, IQR, and min/max values for the path length for both BCLFs and the switching controller (SC).

SC	–	0.95	0.97	$7.54\pm 1.31$
	Setting	Reach $\uparrow$	Avoid $\uparrow$	PL [m] $\downarrow$
$c$	$0.9$	0.80	0.97	$7.38\pm 1.33$
	$0.95$	0.77	0.97	$7.32\pm 1.39$
	$0.99$	0.89	0.87	$4.43\pm 1.48$
	$0.9999$	0.74	0.91	$3.65\pm 0.97$
$\eta$	$2$	0.96	0.97	$7.19\pm 1.66$

TABLE II: Results for the switching controller (SC), asymptotic BCLF coefficient

c

, and finite-time BCLF coefficient

\eta

Fig. 10: Illustration of the conflict resolution in the Constrained Bumper Environment. The particle filter belief being stuck in a local minima is shown by the blue particles. The ground truth trajectory of the state under conflict resolution is shown in blue.

Fig. 11: Application of the proposed Control Architecture on a different Reach-avoid problem where the robot has to fulfill a circular tracking objective. Different initial conditions (ICs) are shown in blue and red.

Fig. 12: Illustration of the learned BCLF for the example two particle system. The colorgradient illustrates higher values of the BCLF, the ground truth goal set is shown in red while the sub-zero levelset of the BCLF is visualized in orange.

Condition	TPR	FPR
$W_{b}()\leq 0$ for $\in\mathcal{S}_{b}$	0.417	$1e^{-4}$
$W_{b}()>0$ for $\notin\mathcal{S}_{b}$	0.999	0.583
$W_{b}(_{k+1})\leq cW_{b}(_{k})$ for ${}_{k}\notin\mathcal{S}_{b}$	0.982	–
$\Delta W_{b}(_{k})\leq-\min(W_{b}(_{k}),\eta)$ for ${}_{k}\notin\mathcal{S}_{b}$	0.95	–
Reach $\mathcal{S}_{b}$	1.0	–
Reach $\mathcal{S}_{b}$ within upper bound $T<=\mathbb{E}\{K(_{0})\}$	0.983	–

TABLE III: Performance metrics for function

W_{b}()

across sampled points

Fig. 13: Hardware validation for a reach-avoid scenario with a narrow passage. To navigate to the goal region $\mathcal{S}_{g}$ , the vertical uncertainty in the position ( $t=0$ s) needs to be reduced which is done with a bump with the top of the workspace ( $t=12$ s). The belief narrows sufficiently and passage with probabilistic safety guarantees can be facilitated ( $t=25.6$ s). In order to also provide the reach behavior with the desired probabilistic guarantee, a second bump ( $t=41.4$ s) narrows the belief in the horizontal direction.

Explore similar work

Apr 4, 2026

Risk-Constrained Belief-Space Optimization for Safe Control under Latent Uncertainty

Many safety-critical control systems operate under latent uncertainty that sensors cannot resolve at decision time. Such uncertainty, arising from unknown physical properties, disturbances, or unobserved geometry, affects dynamics, task feasibility, and safety margins. Standard methods optimize expected performance and offer limited protection against rare but severe outcomes, while robust formulations treat uncertainty conservatively without exploiting its probabilistic structure. We consider systems with measured state and an unknown, time-invariant parameter represented by a belief distribution. We propose a risk-sensitive belief-space Model Predictive Path Integral (MPPI) controller that plans under this belief, regularizes performance using Conditional Value-at-Risk (CVaR), and imposes a CVaR constraint on a trajectory safety margin over the horizon. For the exact risk-constrained formulation underlying this controller, we establish three properties: (1) the CVaR constraint implies a probabilistic safety guarantee, (2) the controller recovers the risk-neutral optimum as the objective risk weight tends to zero, and (3) a union-bound argument extends the per-horizon guarantee to cumulative safety over repeated solves. In contact-rich MuJoCo simulations of vision-guided dexterous stowing, where a manipulator inserts a grasped object into an occupied slot with pose uncertainty exceeding prescribed lateral clearance requirements, our method achieves 82% success with zero contact violations at high risk aversion, compared with 55% and 50% for a risk-neutral configuration and a chance-constrained baseline, both of which incur nonzero exterior contact forces. Project page: https://clintonenwerem.com/belief-cvar-mppi/.

Clinton Enwerem, John S. Baras, Calin Belta

Institute for Systems Research, University of Maryland, College Park, MD, USA

Belief-Space Planning Safety-Critical Control MPPI Control Conditional Value-at-Risk

Ask PDF

May 14, 2026

Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability

Many safety-critical control problems are modeled as risk-sensitive partially observable Markov decision processes, where the controller must make decisions from incomplete observations while balancing task performance against safety risk. Although belief-space planning provides a principled solution, maintaining and planning over beliefs can be computationally costly and sensitive to model specification in practical domains. We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability. The method constructs a compact finite-history proxy state and learns an action-conditioned predictor of near-term safety violation. This predicted candidate-action risk is used in two complementary ways: as a risk penalty during value learning, and as a decision-time gate that interpolates between optimistic and conservative ensemble value estimates. As a result, low-risk actions are evaluated closer to reward-seeking estimates, while high-risk actions are evaluated more conservatively. We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation. Across adult and adolescent glucose-control cohorts, the method improves overall glycemic tradeoffs and substantially reduces runtime relative to a belief-space planning baseline. On Safety-Gym navigation benchmarks, it achieves a more favorable reward-cost balance than unconstrained RL and several standard safe-RL baselines. These results suggest that action-conditioned near-term risk can provide an effective local signal for approximate risk-sensitive POMDP control when full belief-space planning is impractical.

Yushen Liu, Yin-Jen Chen, Ziyi Chen +4

University of Virginia · Google · University of Maryland, College Park +3

Safety-Critical Control Partially Observable RL Risk-Sensitive RL Feedback Control Safe RL

Ask PDF

Jul 16, 2026

Risk-Aware Belief Control Barrier Functions over Random Finite Sets

Ensuring robot safety in unknown, dynamic environments is a fundamental requirement. It involves inferring the states of an unknown and time-varying number of moving objects from noisy, incomplete measurements. We address safe control under the induced multi-object state uncertainty with a risk-aware belief control barrier function (BCBF) framework. The uncertainty is captured by a random finite set (RFS) belief, estimated by a sequential Monte Carlo probability hypothesis density (SMC-PHD) filter that represents it with a set of particles. Building directly on these particles, we construct a nonsmooth BCBF, establish forward invariance of the safe set under continuous prediction, and derive an explicit condition under which discrete updates preserve safety. Simulation and real-world underwater experiments demonstrate the effectiveness and efficiency of the proposed approach.

Shaohang Han, Gang Chen, Yixi Cai +5

Department of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden · Department of Cognitive Robotics, Delft University of Technology, Delft, The Netherlands · Division of Aerospace, Moveability and Naval Architecture, KTH Royal Institute of Technology, Stockholm, Sweden

Control Barrier Functions Safe Robot Navigation Robot Safety Dynamic Obstacle Avoidance

Ask PDF