Interacting safely with cyclists using Hamilton-Jacobi reachability and reinforcement learning

Aarati Andrea Noronha; Jean Oh

cs.ROFeb 20, 2026

Interacting safely with cyclists using Hamilton-Jacobi reachability and reinforcement learning

Authors: Aarati Andrea Noronha, Jean Oh

Abstract

In this paper, we present a framework for enabling autonomous vehicles to interact with cyclists in a manner that balances safety and optimality. The approach integrates Hamilton-Jacobi reachability analysis with deep Q-learning to jointly address safety guarantees and time-efficient navigation. A value function is computed as the solution to a time-dependent Hamilton-Jacobi-Bellman inequality, providing a quantitative measure of safety for each system state. This safety metric is incorporated as a structured reward signal within a reinforcement learning framework. The method further models the cyclist's latent response to the vehicle, allowing disturbance inputs to reflect human comfort and behavioral adaptation. The proposed framework is evaluated through simulation and comparison with human driving behavior and an existing state-of-the-art method.

Figures & tables

Panel of Figure 1 — Fig. 1 : The value function has been plotted on the left side.For the sake of visualization, lateral range has been restricted to a value of 1.5m. On the right side is the zero sub-level set of the viscosity solution to the Hamilton-Jacobi Bellman partial differential equation. In each of the annotated boxes, $X$ represents the longitudinal range, $Y$ represents the longitudinal range rate and $Level$ denotes the negated value function. The aqua green portion represents the backward reachable set (a) Value function computed using our algorithm (b) Value function computed using the algorithm deployed by Fisac et al. 2019

	Safety Factor	Unsafe States (%)	Time (sec)	Collisions	Goal Reached
Human driver	-2.31	39.0	253	0	1
Fisac ’19	-2.18	36.8	199	0	1
Our Framework	-2.01	31.6	233	0	1
Bold text is indicative of the best value corresponding to a particular performance metric. The test data-set consists of 60 trajectories.

TABLE I : Quantitative evaluation of trajectories on the Safety Pilot Michigan Database.

Fig. 2 : A visualization of a cyclist event from the test set. A red line denotes the trajectory computed using the level set deployed by Fisac et al. A dark blue line denotes the ground truth i.e. the behavior of human drivers. A cyan blue line indicates the trajectory traversed by our framework. A green line denotes the trajectory of the cyclist. The position of each of these agents at a particular time instant is given by a dot on its trajectory in its corresponding color. The start and goal positions are denoted by a black star and black circle respectively

Autonomous Driving Planning

Autonomous driving planning selects safe, feasible vehicle actions and trajectories amid traffic, uncertainty, road rules, and changing goals.

See all 384 papers

Hamilton-Jacobi Reachability-Based Safe Reinforcement Learning for Emergency Collision Avoidance
Jun 13, 2026Yuhong Jiang, Shiyue Zhao, Junzhi Zhang +4Collision Avoidance Reachability Analysis
School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China
Context-Conditioned Hamilton-Jacobi Reachability for Adaptive Safety Filtering
Oct 6, 2026Ali Fuat Sahin, Yunus Yazoglu, John Talbot +3Reachability Analysis Autonomous Driving Safety Evaluation
Ecole Polytechnique Fédérale de Lausanne · Stanford University · Toyota Research Institute
Enumeration-Guided Training of Neural Hamilton-Jacobi Safety Certificates
Oct 20, 2026Mihaela-Larisa Clement, Tosca Pierro, Ezio Bartocci +1Safety-Critical Control Reachability Analysis
TU Wien, Vienna, Austria · AIT Austrian Institute of Technology, Vienna, Austria · Università Politecnica delle Marche, Italy

Jun 13, 2026J/K move · Enter open · S save

Hamilton-Jacobi Reachability-Based Safe Reinforcement Learning for Emergency Collision Avoidance

Yuhong Jiang, Shiyue Zhao, Junzhi Zhang, Junfeng Zhang+3

School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China

Collision Avoidance Reachability Analysis Constrained RL RL for Autonomous Driving Safe RL

Open paper AskPDF arXiv

Emergency collision avoidance under extreme driving conditions demands safety-critical control that accounts for both obstacle proximity and vehicle dynamic stability over a future time horizon, yet existing methods often rely on instantaneous or local safety evaluations. This paper proposes a safe reinforcement learning framework guided by a Hamilton-Jacobi (HJ) reachability based motion safety set that provides forward-looking safety supervision for constrained policy optimization. Specifically, a unified signed safety function is formulated by combining geometric collision margins and chassis stability limits, and is then extended through reachability analysis into a finite-horizon motion safety set that characterizes whether safety can be maintained under future vehicle state evolution. To enable practical computation, the motion safety set is approximated from offline extreme driving data, mitigating the computational burden of grid-based HJ solvers. The learned motion safety set is then embedded as a continuous safety cost into a constrained Markov decision process, and a PID-Lagrangian policy optimization scheme is employed to adaptively regulate the Lagrange multiplier for safety constraint enforcement. Simulation and real-vehicle experiments on low-adhesion obstacle-avoidance scenarios demonstrate that the proposed method achieves higher goal-reaching rates, produces smoother avoidance maneuvers, and maintains larger unified safety margins than baseline methods.

Abstract

Autonomous Driving Planning

Explore similar work

Hamilton-Jacobi Reachability-Based Safe Reinforcement Learning for Emergency Collision Avoidance

Context-Conditioned Hamilton-Jacobi Reachability for Adaptive Safety Filtering

Enumeration-Guided Training of Neural Hamilton-Jacobi Safety Certificates