Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS

Sachin Giri; Thomas Zhao; Matthew Huynh; YangQuan Chen

cs.ROSep 4, 2026

Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS

Authors: Sachin Giri, Thomas Zhao, Matthew Huynh, YangQuan Chen

Organizations: Electrical Engineering & Computer Science Graduate Program, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA · Dept. of Mechanical Engineering, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA

Abstract

The autonomous localization of fugitive gas emissions using small Unmanned Aircraft Systems (sUAS) constitutes a fundamentally ill-posed inverse problem. In turbulent atmospheric boundary layers, highly intermittent scalar concentration fields violate the assumptions of classical gradient-based navigation, causing data-driven estimators to suffer from severe noise and spurious local minima. To address these challenges, we introduce an Information-Guided Safe Reinforcement Learning framework evaluated within a custom, GPU-accelerated 3D simulation environment coupling an Eulerian wind solver with a Lagrangian puff dispersion model. We identify a critical vulnerability in deterministic information-seeking planners - a Gramian bias where agents act greedily upon flawed early estimates, starving the estimator of spatial diversity. To systematically break this degeneracy, our architecture integrates a classical empirical observability Gramian (EMGR) planner with a learned Soft Actor-Critic (SAC) exploratory policy. A deterministic meta-supervisor actively monitors estimator reliability via Kullback-Leibler (KL) divergence, dynamically blending deterministic exploitation with learned exploration to steer the sUAS into high-information zones. Trained via a progressive curriculum and safeguarded by a strictly enforced Robust Control Barrier Function (RCBF), our RL framework achieves nearly 80% localization success on complex, mobile sources - drastically outperforming classical baselines (~30%) - while ensuring zero safety violations.

Figures & tables

Fig. 1: 3D simulation environment built with NVIDIA Warp, demonstrating the Lagrangian puff dispersion model (color-coded by concentration) navigating around physical obstacles.

Panel of Figure 2 — Fig. 2: 2D cross-sections of the coupled simulation environment.

Panel of Figure 3 — Fig. 3: Sequential execution pipeline. Sensor data feeds parallel estimators whose fused estimate drives both the classical EMGR planner and the learned SAC policy. The meta-supervisor allocates blending authority $\omega_{k}^{act}$ ; the safety layer enforces collision avoidance before actuation.

Method	Success	Med. Dist. [5th, 95th]	Med. Steps
MLE+EMGR	30.4%	24.3 [4.7, 135.8] m	1173
MLE+EMGR+PF+KL	32.1%	25.0 [4.8, 159.3] m	2000
Stage 1 (small)	98.8%	4.9 [4.7, 5.0] m	256
Stage 2 (full)	79.3%	4.9 [4.7, 43.8] m	310
Stage 3 (mobile)	78.6%	4.8 [4.5, 59.2] m	276

TABLE I: Localization Performance (100 episodes,

d_{conv}=5

Fig. 4: Median distance to source with interquartile bands. RL agents converge below $d_{conv}=5m$ within 300 steps; baselines stagnate due to Gramian bias.

Fig. 5: Cumulative success rate over tracking steps.

Method	Active	Max Corr.	Min $h(\mathbf{r})$	Violations
MLE+EMGR	12.9%	4.87	2.54 m	0
MLE+EMGR+PF+KL	7.2%	4.84	2.54 m	0
Stage 1	88.9%	6.91	3.04 m	0
Stage 2	84.3%	5.77	3.04 m	0
Stage 3	88.4%	5.59	3.04 m	0

TABLE II: RCBF Safety Statistics

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing
Mar 30, 2026Alex Zongo, Filippos Fotiadis, Ufuk Topcu +1Robust RL Collision Avoidance
A. Zongo and P. Wei are with the Department of Mechanical and Aerospace Engineering, George Washington University, Washington, D.C. 20052, USA. · F. Fotiadis and U. Topcu are with the Oden Institute for Computational Engineering & Sciences, University of Texas at Austin, Austin, TX 78712, USA.
Multi-Agent Reinforcement Learning for UAV-Based Chemical Plume Source Localization
Mar 12, 2026Zhirun Li, Derek Hollenbeck, Ruikun Wu +4Multi-Agent Reinforcement Learning UAV Navigation
Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM 87131 USA · Department of Mechanical Engineering, University of California, Merced, CA 95343 USA · Department of Electrical Engineering, Colorado School of Mines, Golden, CO 80401 USA +3
Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation
Jul 2, 2026Shenghui Zhang, YuXuan Gao, Songwei Zhao +3Robot Policy Learning UAV Navigation

Mar 30, 2026J/K move · Enter open · S save

Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing

Alex Zongo, Filippos Fotiadis, Ufuk Topcu, Peng Wei

A. Zongo and P. Wei are with the Department of Mechanical and Aerospace Engineering, George Washington University, Washington, D.C. 20052, USA. · F. Fotiadis and U. Topcu are with the Oden Institute for Computational Engineering & Sciences, University of Texas at Austin, Austin, TX 78712, USA.

Robust RL Collision Avoidance Adversarial Perturbations Adversarial Training Cooperative MARL

Open paper AskPDF arXiv

We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) broadcasts its GPS-derived position; when such position broadcasts are corrupted, the entire observed air traffic state becomes unreliable. We cast this state observation corruption as a zero-sum game between the agents and an adversary: with probability R, the adversary perturbs the observed state to maximally degrade each agent's safety performance. We derive a closed-form expression for this adversarial perturbation, bypassing the iterative inner optimization of adversarial training entirely and enabling linear-time evaluation in the state dimension. We show that this expression approximates the exact minimizer of the value function over the modeled uncertainty set with second-order accuracy. We further bound the safety performance gap between clean and corrupted observations, showing that it degrades at most linearly with the corruption probability under Kullback-Leibler regularization. Finally, we integrate the closed-form adversarial policy into a MARL policy gradient algorithm to obtain a robust counter-policy for the agents. In a high-density sUAS simulation, we observe near-zero collision rates under corruption levels up to 35%, outperforming a baseline policy trained without adversarial perturbations.

Abstract

UAV Navigation

Explore similar work

Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing

Multi-Agent Reinforcement Learning for UAV-Based Chemical Plume Source Localization

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation