Reach-avoid sets, defining the states from which a task can be completed while avoiding unsafe regions, are fundamental tools for deploying robots in environments subject to disturbances and uncertainty, as well as for runtime task-feasibility monitoring, allowing prediction of whether a state is safe and will lead to successful task completion. Classical Hamilton-Jacobi Reachability (HJR) and its recent neural extensions (e.g., Lipschitz Continuous Reachability Learning, DeepReach) treat disturbances adversarially, computing the reach-avoid set under worst-case disturbances from a bounded set. This guarantees safety but can yield overly conservative sets when disturbances are stochastic rather than adversarial, and additionally requires the disturbance set to be known or specified in advance. We propose Probabilistic reach-avoid Value Function (PVF), a self-supervised method, applicable to black-box systems, for computing reach-avoid sets as the states with a certain reach-avoid probability under stochastic, potentially unbounded disturbances. We evaluate our approach against HJR-based baselines on systems of increasing complexity, up to a Hopper locomotion task, and show that our probabilistic sets are less conservative and more accurate than adversarial HJR sets when disturbances are non-adversarial.
Figures & tables
Fig. 1: Overview of the PVF pipeline. Data collection: Trajectories are collected by running a nominal policy π(x) on black-box systems (e.g., legged robots, manipulators, quadrotors) subject to stochastic disturbances w , and stored as transition pairs (x,x′) in a replay buffer. Self-supervision: Piecewise target labels Vtarget(x) are assigned based on constraint violation ( 0 ), task completion ( 1 ), or bootstrapped value evaluation ( Vθ(x′) ). Training: A neural network parameterizing Vθ(x) is optimized via temporal-difference loss to output the continuous reach-avoid success probability.
Fig. 2: Reach-avoid value functions on the scalar toy problem. Comparison between the proposed Probabilistic Value Function ( VPVF , left) and the adversarial Lipschitz Continuous Reachability Learning baseline ( VLCRL , right). Green regions indicate states identified as safe (i.e., able to reach the target set without violating constraints).
Fig. 3: Visual comparison between PVF (left) and LCRL (right) in terms of accuracy, for Drones racing experiment. x and y indicate the initial position of the ego drone in the horizontal plane. The opposing drone starts at x=0.4 m and y=0.0 m
Fig. 4: Comparison between PVF and LCRL in terms of accuracy, for Drones racing experiment. Thresholds for fail detection reported near the medians.
Fig. 5: Snapshot of the Hopper legged system.
Fig. 6: Comparison between PVF and LCRL in terms of accuracy, for Hopper experiment. Thresholds for fail detection reported near the medians.
Drones Racing
Hopper
LCRL
3175 s
361 s
PVF data collection
677 s
7145 s
PVF training
9367 s
23502 s
PVF total
10044 s
30647 s
TABLE I: Mean computation time across the different runs.
Metric
Value (%)
True Positive
74.6
False Positive
23.6
True Negative
0.0
False Negative
0.0
Unclear Cases
1.8
TABLE II: Failures-only Network results for Hopper.
Noise Level ( 4σ )
0.10
0.15
0.20
Shielded
99.1±1.6
97.2±4.2
91.0±8.3
Unshielded
97.1±5.2
90.8±7.7
83.1±10.0
TABLE III: Mean ± standard deviation of success counts (out of 100 trials) across 11 seeds, by noise level.
We study stochastic minimum-cost reach-avoid reinforcement learning, where an agent must satisfy a reach-avoid specification with probability at least p while minimizing expected cumulative costs in stochastic environments. Existing safe and constrained reinforcement learning methods typically fail to jointly enforce probabilistic reach-avoid constraints and optimize cost in the learning setting in stochastic environments. To address this challenge, we introduce reach-avoid probability certificates (RAPCs), which identify states from which stochastic reach-avoid constraints are satisfiable. Building on RAPCs, we develop a contraction-based Bellman formulation that serves as a principled surrogate for integrating reach-avoid considerations into reinforcement learning, enabling cost optimization under probabilistic constraints. We establish almost sure convergence of the proposed algorithms to locally optimal policies with respect to the resulting objective. Experiments in the MuJoCo simulator demonstrate improved cost performance and consistently higher reach-avoid satisfaction rates.
Jingduo Pan, Taoran Wu, Yiling Xue +1
Key Laboratory of System Software (Chinese Academy of Sciences), Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China
Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.
Zeyuan Feng, Ali Fuat Sahin, Santiago Thorup +1
Department of Mechanical Engineering, École Polytechnique Fédérale de Lausanne, Switzerland
Robots must complete their tasks and maintain the achieved outcomes while avoiding safety failures at all times. Strict reach-avoid-stay (sRAS) formalizes this requirement: safely reaching a target and remaining there indefinitely after first entry. We propose an sRAS Q-control barrier function (CBF) safety filter for high-dimensional black-box systems under bounded uncertainty. Our construction combines a stay value encoding safe permanent residence in a target subset with a reach-avoid value encoding safe reachability of this subset while avoiding target states from which safe permanent residence cannot be guaranteed. We prove that these values jointly yield a valid robust discrete-time CBF and lift them to state-action Q-functions for runtime intervention. For exact values and under a measure-zero condition, our filter preserves sRAS feasibility from almost every winnable initial state and keeps the system safely within the target after first entry, against all admissible uncertainty realizations. We adopt reachability-based adversarial reinforcement learning for scalable value approximation using only black-box interactions. Notably, neither synthesis nor deployment of our filter requires known dynamics, affine structure, value derivatives, or hand-designed barriers. We validate our framework in quadruped gap jumping in simulation and hardware, where the robot crosses the gap, lands safely, and remains safe afterward. Simulated F1TENTH races further demonstrate safe overtaking and lead retention.
Donggeon David Oh, Duy P. Nguyen, Gongkai Yuan +3
Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218, USA