Multi-Agent Reinforcement Learning for UAV-Based Chemical Plume Source Localization

Zhirun Li; Derek Hollenbeck; Ruikun Wu; Michelle Sherman; Sihua Shao; Xiang Sun; Mostafa Hassanalian

eess.SYMar 12, 2026

Multi-Agent Reinforcement Learning for UAV-Based Chemical Plume Source Localization

Authors: Zhirun Li, Derek Hollenbeck, Ruikun Wu, Michelle Sherman, Sihua Shao, Xiang Sun, Mostafa Hassanalian

Organizations: Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM 87131 USA · Department of Mechanical Engineering, University of California, Merced, CA 95343 USA · Department of Electrical Engineering, Colorado School of Mines, Golden, CO 80401 USA · Department of Electrical Engineering, New Mexico Tech, Socorro, NM 87801 USA ·

Abstract

Undocumented orphaned wells pose significant health and environmental risks to nearby communities by releasing toxic gases and contaminating water sources, with methane emissions being a primary concern. Traditional survey methods such as magnetometry often fail to detect older wells effectively. In contrast, aerial in-situ sensing using unmanned aerial vehicles (UAVs) offers a promising alternative for methane emission detection and source localization. This study presents a robust and efficient framework based on a multi-agent deep reinforcement learning (MARL) algorithm for the chemical plume source localization (CPSL) problem. The proposed approach leverages virtual anchor nodes to coordinate UAV navigation, enabling collaborative sensing of gas concentrations and wind velocities through onboard and shared measurements. Source identification is achieved by analyzing the historical trajectory of anchor node placements within the plume. Comparative evaluations against the fluxotaxis method demonstrate that the MARL framework achieves superior performance in both localization accuracy and operational efficiency.

Figures & tables

Figure 1: In-situ gas concentration measurement results and two UAV-sensor products [ 3 , 4 ] .

Figure 2: Envisioned Multi-UAV CPSL scenario.

Figure 3: Topology of the considered CPSL Problem.

Figure 4: Search area of the defined CPSL problem.

Figure 5: Concentration plot when a UAV is at $h=2$ m (left) and $h=5$ m (right) at a point 6 m downwind from the emitter location along the plume’s center-line. The time resolution is 0.05 sec. and the filament release rate is $N_{f}=50$ filaments/sec.

Notation	Value
Emitter Source Location $\left({\bf p}_{es}\right)$	(80, 60) m
Mean Direction of Wind $\left({\overline{\bf V}^{w}}\right)$	[1,0] m/s
Colored Noise Constants $\left([a,b,G]\right)$	$[0.005,\;0.02,\;1]$
Concentration Sensor Filter Coeffs. $\left(B,c_{h}\right)$	$(0.1,10^{-4})$
Diffusivity Constants $\left(K_{x},K_{y}\right)$	(1000,1000) $\text{m}^{2}$ /s
Relative Diffusion Spectral Density $\left(\sigma_{v_{m,1}},\sigma_{v_{m,2}}\right)$	$(2,2)$ m/s/ $\sqrt{\text{Hz}}$

Table I: Plume Parameter Values

Figure 6: Plume dynamics due to relative diffusion and constant advection, i.e. the mean flow field has velocity ${\overline{\bf V}}^{w}=[1,0]$ m/s, and the emitter source location is at $(x,y)=(80,60)$ m.

Figure 7: (a) Relative angle information of UAV $i^{\prime}$ observed by UAV $i$ (b) relative formation information observed by UAV $i$ .

Figure 8: (a) Example of conditions where the anchor information is updated and (b) Example of conditions where the anchor information is NOT updated.

Description	Value
Geographical Area ( $X\times Y$ )	$200\;\text{m}\times 200\;\text{m}$
Grid element size ( $\Delta x\times\Delta y$ )	$2\;\text{m}\times 2\;\text{m}$
Number of UAVs ( $N$ )	$3$
Number of Obstacles ( $M$ )	$5$
UAV Object Sensing Radius ( $r_{s}$ )	$20$ m
Collision Safety Distance ( $\epsilon$ )	1 m

Table II: Simulation Parameter Values

Notation	Value
Emitter Location	(80, 60) m
	(60, 120) m
Noise Constants $\left([a,b,G]\right)$	$[0.005,\;0.02,\;1]$
	$[0.005,\;0.02,\;3]$
	$[0.005,\;0.02,\;5]$

Table III: Test Plume Parameter Values

Figure 9: Simulation results for Plume Example 1: (a) Mean Total Reward $R_{u}^{i}$ received per agent, (b) Mean Target Found Rate, (c) Mean Final Centroid Error, (d) Mean Agent-to-Agent and Agent-to-Obstacle Collision Rate.

Figure 10: Reward difference during training.

Figure 11: Plume distribution with different wind conditions

Scenario Setup		Succ. Rate	Mean Location Error (m)
Scenario Setup		Succ. Rate	No Offset		With Offset
Wind Meander	Emitter Location		All	Succ.	All	Succ.
No	[80, 60]	1.00	2.34	2.34	1.55	1.55
No	[60, 120]	1.00	2.17	2.17	1.52	1.52
Small	[80, 60]	1.00	2.16	2.16	1.76	1.76
Small	[60, 120]	0.96	2.61	2.40	1.7	1.48

Table IV: Success rate and location error (i.e, final distance to emitter) under various environment conditions

Figure 12: Example of different UAVs behaviors during a successful episode

Figure 13: Trajectory and location error evolution for MARL and fluxotaxis under varying wind meandering conditions

Figure 14: CDF of final distance to emitter for MARL and fluxotaxis (100 tests per case)

Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning studies how interacting agents learn coordinated or competitive policies through shared experience and rewards.

See all 373 papers

Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS
Sep 4, 2026Sachin Giri, Thomas Zhao, Matthew Huynh +1UAV Navigation Safe RL
Electrical Engineering & Computer Science Graduate Program, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA · Dept. of Mechanical Engineering, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA
Chasing Ghosts: A Simulation-to-Real Olfactory Navigation Stack with Optional Vision Augmentation
Feb 23, 2026Kordel K. France, Ovidiu Daescu, Latifur Khan +1Robot Navigation UAV Navigation
Dept. of Computer Science University of Texas at Dallas Richardson, TX, USA
Global monitoring of methane point sources using deep learning on hyperspectral radiance measurements from EMIT
Apr 11, 2026Vishal V. Batchu, Michelangelo Conserva, Alex Wilson +5Remote Sensing
Google Research · Carnegie Institution for Science · Jet Propulsion Laboratory, California Institute of Technology

Sep 4, 2026J/K move · Enter open · S save

Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS

Sachin Giri, Thomas Zhao, Matthew Huynh, YangQuan Chen

Electrical Engineering & Computer Science Graduate Program, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA · Dept. of Mechanical Engineering, University of California, Merced, 5200 N. Lake Rd, Merced, CA 95343, USA

UAV Navigation Safe RL

Open paper AskPDF arXiv

The autonomous localization of fugitive gas emissions using small Unmanned Aircraft Systems (sUAS) constitutes a fundamentally ill-posed inverse problem. In turbulent atmospheric boundary layers, highly intermittent scalar concentration fields violate the assumptions of classical gradient-based navigation, causing data-driven estimators to suffer from severe noise and spurious local minima. To address these challenges, we introduce an Information-Guided Safe Reinforcement Learning framework evaluated within a custom, GPU-accelerated 3D simulation environment coupling an Eulerian wind solver with a Lagrangian puff dispersion model. We identify a critical vulnerability in deterministic information-seeking planners - a Gramian bias where agents act greedily upon flawed early estimates, starving the estimator of spatial diversity. To systematically break this degeneracy, our architecture integrates a classical empirical observability Gramian (EMGR) planner with a learned Soft Actor-Critic (SAC) exploratory policy. A deterministic meta-supervisor actively monitors estimator reliability via Kullback-Leibler (KL) divergence, dynamically blending deterministic exploitation with learned exploration to steer the sUAS into high-information zones. Trained via a progressive curriculum and safeguarded by a strictly enforced Robust Control Barrier Function (RCBF), our RL framework achieves nearly 80% localization success on complex, mobile sources - drastically outperforming classical baselines (~30%) - while ensuring zero safety violations.

Abstract

Multi-Agent Reinforcement Learning

Explore similar work

Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS

Chasing Ghosts: A Simulation-to-Real Olfactory Navigation Stack with Optional Vision Augmentation

Global monitoring of methane point sources using deep learning on hyperspectral radiance measurements from EMIT