Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.
Figures & tables
Figure 1: Method Overview: backward-temporal curriculum with actor-critic learning.
Method
Narrow Passage
State Space of Interest
Entire State Space
FPR % ↓
FNR % ↓
Succ % ↑
Time (h) ↓
Learned Vol %
Verified Vol % ↑
Learned Vol %
Verified Vol % ↑
DeepReachMPC
0.00±0.00
100.00±0.00
2.50±0.90
5
0.00±0.00
0.00±0.00
0.67±0.21
0.00±0.00
RL Baseline
99.97±0.01
0.00±0.00
3.52±0.91
6
99.97±0.02
0.00±0.00
13.19±0.31
0.00±0.00
Ours
100.00±0.00
0.00±0.00
99.71±0.18
1
100.00±0.00
100.00±0.00
4.99±0.02
3.76±0.07
Method
Quadrotor Gate Avoidance
Entire State Space
FPR % ↓
FNR % ↓
Succ % ↑
Time (h) ↓
Learned Vol %
Verified Vol % ↑
Table 1: Quantitative comparison of open-loop safety and closed-loop control performance. Results report mean and standard deviation over 5 random seeds. For Narrow Passage, we further report the verified volume over the entire state space. FNR is N/A when no rollout succeeds. The reduction in training time for our method is primarily attributable to its implementation in JAX.
Method
40D Network Dynamics
80D Network Dynamics
FPR % ↓
FNR % ↓
MSE ↓
FPR % ↓
FNR % ↓
MSE ↓
DeepReachMPC
2.648±0.310
0.001±0.001
0.007±0.001
2.115±0.145
0.000±0.000
0.026±0.001
RL Baseline
23.306±0.397
76.228±0.602
1.717±0.086
25.581±0.941
78.696±0.731
5.886±0.248
Ours
2.319±0.207
0.032±0.011
0.006±0.001
2.099±0.382
0.005±0.010
0.023±0.008
Table 2: Scalability stress-testing on high-dimensional Publisher-Subscriber network dynamics. Since this system admits a ground-truth solution, we directly compute FPR, FNR, and MSE against the ground truth.
Figure 2: Learned value function slices. Quadrotor: x - z plane with vx=2.5 and all other states at hover. Narrow Passage: x - y plane of car 1 at T=8 s, with [θ1,v1,ϕ1,x2,y2,θ2,v2,ϕ2]=[0,6,0,8,0,−π,1,0] . Only our method recovers a valid safe set in both tasks, whereas DeepReachMPC and the RL baseline fail to accurately propagate the value function over long horizons or against complex, non-smooth safety boundaries.
Configuration
FPR % ↓
FNR % ↓
Succ % ↑
Time (mins) ↓
Policy Gradient - 1 Step
99.58±0.09
0.79±0.74
92.20±6.11
96.8
Policy Gradient - 3 Step
93.57±8.71
0.25±0.13
90.15±6.87
140.1
Policy Gradient - 5 Step
99.75±0.09
0.28±0.08
80.81±3.85
210.6
Teacher Student
100.00±0.00
0.00±0.00
99.71±0.18
81.3
Table 3: Ablation of policy optimization objectives on the Narrow Passage benchmark.
Figure 3: Real-world deployment on the Rosmaster R2 RC car, with value function slices visualized at critical states along the trajectory.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Left: (a) 3D Dubins problem setup with policy safe sets and rollouts ( T=1.0 s, varying Δt ), and (b) two-link arm problem setup. Right: Safe set coverage percentages across varying Δt and T .
Discretization Step ( Δt )
FPR % ↓
FNR % ↓
Succ % ↑
Time (h)↓
Δt=0.025s
100.0
0.0
82.01
1.9
Δt=0.050s
100.0
0.0
99.71
1.1
Δt=0.075s
100.0
0.0
93.49
0.9
Δt=0.100s
100.0
0.0
96.31
0.8
Δt=0.125s
100.0
0.0
99.99
0.7
Δt=0.150s
100.0
0.0
99.98
0.6
Appendix
Table 4: Ablation results isolating the impact of the time discretization step size Δt on safety metrics and rollout performance over 20,000 test states.
Curriculum Schedule ( α )
Δ Succ % ↓
Δ Cost Mean ↓
Δ Cost Std ↓
Pure Student ( α=0.0 )
0.2210%±0.0859%
0.0356±0.0015
0.0772±0.0021
Pure Teacher ( α=1.0 )
0.3362%±0.0608%
0.0274±0.0013
0.0643±0.0025
Decaying Curriculum ( α:1.0→0.0 )
0.2267%±0.0607%
0.0299±0.0070
0.0699±0.0061
Constant Mixture ( α=0.5 )
0.2056%±0.0856%
0.0312±0.0018
0.0706±0.0026
Appendix
Table 5: Ablation of the teacher mixing fraction α on the Dubins3D benchmark. Metrics report the absolute difference between our learned policy and the numerical solution over 20,000 test states. Δ Succ denotes the success-rate gap, while Δ Cost Mean and Δ Cost Std denote the differences in mean rollout cost and rollout-cost standard deviation, respectively. Lower values indicate closer agreement with the numerical baseline.
Method
False Positives (FP) ↓
False Negatives (FN) ↓
OptimizedDP
1569
10
Ours
206
35
Appendix
Table 6: Quantitative safety evaluation across 20,000 random rollouts in the 4D collision avoidance environment.
Figure 5: Qualitative comparison of the 4D human-robot collision avoidance game. For both methods, we display the predicted value function (left sub-panel) alongside the reachability rollout cost evaluated from its induced policy (right sub-panel). The heatmap slices are taken on the x-y plane where θrel=π/2 and vego=1.0 . The grid-based OptimizedDP solution exhibits severe numerical artifacts due to the asymmetric control authority between players. In contrast, our neural framework maintains a relatively more consistent safety boundary that closely matches the empirical rollout cost.
Evader Policy
MPPI
Ours
Pursuer Policy
MPPI
36%
0%
Ours
88%
42%
Appendix
Table 7: Quantitative catching rates for the 10D pursuit-evasion game. We evaluate the performance of our method against the MPPI baseline by pairing different combinations of pursuer and evader policies.
i
cix
ciy
ri
1
−0.70
−0.20
0.30
2
−1.20
−1.50
0.35
3
−1.80
−1.00
0.50
4
−2.00
−1.50
0.40
5
−1.50
−2.00
0.25
Appendix
Table 8: Default obstacle parameters for the Dubins car environment.
Hyperparameter
Value
Architecture
Value network width W
512
Value network depth L
8
Policy network width Wπ
512
Policy network depth Lπ
5
Optimization
Appendix
Table 9: Default architecture and training hyperparameters used across all experiments.
Benchmark
Horizon (s)
Windows
Δt
dx
dϕ
du
dd
Dubins Car
4.0
4
0.02
3
4
1
0
Narrow Passage
8.0
16
0.05
10
12
4
0
Quadrotor
1.0
8
0.01
13
13
4
0
Pursuit-Evasion
8.0
8
0.05
10
13
2
2
40D Network
1.0
2
0.01
40
40
39
0
80D Network
1.0
2
0.01
80
80
79
0
Appendix
Table 10: Benchmark-specific configurations. dx : state dimension; dϕ : NN input dimension excluding the time input (after encoding); du / dd : control/disturbance dimensions.