Olfactory pursuit: catching a moving odor source in complex flows

Maurizio Carbone; Lorenzo Piro; Robin A. Heinonen; Luca Biferale; Massimo Cencini; Antonio Celani

cs.ROApr 13, 2026

Olfactory pursuit: catching a moving odor source in complex flows

Authors: Maurizio Carbone, Lorenzo Piro, Robin A. Heinonen, Luca Biferale, Massimo Cencini, Antonio Celani

Organizations: Istituto dei Sistemi Complessi, CNR, Via dei Taurini 19, 00185 Rome, Italy and INFN “Tor Vergata", Via della Ricerca Scientifica 1, 00133 Rome, Italy · Department of Physics & INFN, Tor Vergata University of Rome, Via della Ricerca Scientifica 1, 00133 Rome, Italy · Machine Learning Genoa Center (MaLGa) & Department of Civil, Chemical and Environmental Engineering, University of Genoa, Genoa, Italy · Quantitative Life Sciences, The Abdus Salam International Centre for Theoretical Physics, 34151 Trieste, Italy ·

Abstract

Locating and intercepting a moving target from possibly delayed, intermittent sensory signals is a paradigmatic problem in decision-making under uncertainty, and a fundamental challenge for, e.g., animals seeking prey or mates and autonomous robotic systems. Odor signals are intermittent, strongly mixed by turbulent-like transport, and typically lag behind the true target position, thereby complicating localization. Here, we formulate olfactory pursuit as a partially observable Markov decision process in which an agent maintains a joint belief over the target's position and velocity. Using a discrete run-and-tumble model, we compute quasi-optimal policies by numerically solving the Bellman equation and benchmark them against well-established information-theoretic strategies such as Infotaxis. We show that purely exploratory policies are near-optimal when the target frequently reorients, but fail dramatically when the target exhibits persistent motion. We thus introduce a computationally efficient hybrid policy that combines the information-gain drive of Infotaxis with a "greedy" value function derived from an associated fully observable control problem. Our heuristic achieves near-optimal performance across all persistence times and substantially outperforms purely exploratory approaches. Moreover, our proposal demonstrates strong robustness even in more complex search scenarios, including continuous run-and-tumble prey motion with moderate persistence time, model mismatch, and more accurate plume dynamics representation. Our results identify predictive inference of target motion as the key ingredient for effective olfactory pursuit and provide a general framework for search in information-poor, dynamically evolving environments.

Figures & tables

Figure 1: Sketch of typical olfactory pursuit. The target source (blue circle) performs a run-and-tumble motion and emits a blue chemical plume along its trajectory. The agent (red dashed trajectory) pursues the moving target by inferring its position and velocity from discrete odor detections (indicated by the red triangles). The agent maintains a probability map over the target position $\bm{X}_{s}$ and velocity $\bm{U}_{s}$ , namely the joint belief $b_{t}(\bm{x},\bm{u})$ about its state at time $t$ represented in the thought clouds. The search begins with an initial detection at $t=0$ , after which the agent’s position belief corresponds to the likelihood of a detection centered at the agent’s position; the estimated target velocity is initially random uniform over all directions. As time progresses, the target moves and emits, while the agent updates its belief to account for the target source’s motion and accumulates information about its state through successive detections. Over time, the agent’s belief becomes more peaked around the true target position, and its guess of the target velocity gets more accurate. This allows the agent to confidently pursue and intercept the target.

Figure 2: Results from the discrete run-and-tumble model, with both agent and target moving on a periodic lattice. Panel A shows the average search time

\langle T\rangle

as a function of the target persistence time

\tau_{p}

, averaged over

10^{4}

search episodes starting from a detection: the black curve indicates the quasi-optimal strategy obtained from solving the POMDP; blue curve the infotactic policy (

w=0

); gray curve refers to random search, obtained with the agent performing a persistent random walk with no olfactory cues; circles indicate the hybrid heuristic policy 3 for the optimal weight

w

, yielding the lowest average search time, color-coded as in the right bar. The insets show the complementary cumulative distribution function (CCDF) of the search times for the best hybrid heuristic and Infotaxis, at

\tau_{p}=2

and

25

. Panel B details the effect of the policy weight

w

on the average search time, normalized with the best average search time, for

\tau_{p}=2

and

25

. Panels C , D compare the average search times of the best hybrid heuristic with the quasi-optimal policy ( C ) and Infotaxis ( D ). For this set of simulations, the side of the square lattice is

L=51

with spacing

\Delta x=1

, the agent and target speed is

U=1

, the characteristic length scale of the likelihood of a detection is

\lambda=3

, the emission rate

R=1

, and the agent finds the source when

\|\bm{X}_{\rm ag}-\bm{X}_{s}\|\leq\sqrt{2}

Figure 3: Results from the continuous run-and-tumble model, with the target performing a continuous run-and-tumble on a periodic square and the agent moving on a periodic lattice. Panel A shows the average search time $\langle T\rangle$ as a function of the target persistence time, averaged over $10^{4}$ search episodes starting from a detection: circles indicate the hybrid heuristic policy 3 for the best weight $w$ , color-coded as in the right bar; Infotaxis in blue corresponds to $w=0$ ; the gray line refers to random search, obtained with the agent performing a persistent random walk without olfactory cues. The inset compares the average search times of the best hybrid heuristic with Infotaxis. Panel B shows the effect of the policy weight $w$ on the normalized average search time for two persistence times of the target motion. Panel C shows the normalized average search time as a function of $U\tau_{d}/\lambda$ , i.e. the ratio between the agent speed $U$ and the characteristic speed of the chemical diffusion $\lambda/\tau_{d}$ , at emission rate $R=36/\tau_{d}$ . Results in panels A and B correspond to $U\tau_{d}/\lambda=0.1$ at $R=120$ . For the results reported here, the agent and target speed is $U=1$ , the side of the lattice domain is $L=51$ , the detection likelihood length scale is $\lambda=3$ , and the agent finds the source when $\|\bm{X}_{\rm ag}-\bm{X}_{s}\|\leq\sqrt{2}$ .

Policy Optimization

Policy optimization develops algorithms for improving decision rules in reinforcement learning, language models, and sequential decision systems.

See all 224 papers

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery
May 15, 2026Marco Rando, Robin A. Heinonen, Yujia Qi +1
Universit´e Cˆote d’Azur, Inria, CNRS, Laboratoire J.A. Dieudonn´e, 28 avenue Valrose, 06108, Nice, France · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova Via Montallegro 1, 16145 Genoa, Italy · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova, Via Montallegro 1, 16145 Genoa, Italy +1
Solvable models of learning to pursue a moving target
Oct 22, 2026John J. Vastola, Kanaka RajanLinear Quadratic Regulator Policy Learning
Department of Neurobiology, Harvard Medical School · Department of Psychology and Center for Brain Science, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University
Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning
May 16, 2026Changxu Zhao, Dongxiao Zhao, Xin Bian +1Reinforcement Learning Robot Navigation
State Key Laboratory of Ocean Engineering, School of Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, People’s Republic of China · State Key Laboratory of Fluid Power and Mechatronic Systems, Department of Engineering Mechanics, Zhejiang University, Hangzhou 310027, People’s Republic of China

May 15, 2026J/K move · Enter open · S save

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

Marco Rando, Robin A. Heinonen, Yujia Qi, Agnese Seminara

Universit´e Cˆote d’Azur, Inria, CNRS, Laboratoire J.A. Dieudonn´e, 28 avenue Valrose, 06108, Nice, France · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova Via Montallegro 1, 16145 Genoa, Italy · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova, Via Montallegro 1, 16145 Genoa, Italy+1

Open paper AskPDF arXiv

Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.

Abstract

Policy Optimization

Explore similar work

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

Solvable models of learning to pursue a moving target

Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning