Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery
Authors: Marco Rando, Robin A. Heinonen, Yujia Qi, Agnese Seminara
Organizations: Universit´e Cˆote d’Azur, Inria, CNRS, Laboratoire J.A. Dieudonn´e, 28 avenue Valrose, 06108, Nice, France · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova Via Montallegro 1, 16145 Genoa, Italy · Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova, Via Montallegro 1, 16145 Genoa, Italy · Dept Mechanical Engineering, Engineering II, Santa Barbara, CA 93106-5070, USA
Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.