Human-Inspired Pavlovian and Instrumental Learning for Autonomous Agent Navigation

Jingfeng Shan; Francesco Guidi; Mehrdad Saeidi; Enrico Testi; Elia Favarelli; Andrea Giorgetti; Davide Dardari; Alberto Zanella

cs.MAMar 23, 2026

Human-Inspired Pavlovian and Instrumental Learning for Autonomous Agent Navigation

Authors: Jingfeng Shan, Francesco Guidi, Mehrdad Saeidi, Enrico Testi, Elia Favarelli, Andrea Giorgetti, Davide Dardari, Alberto Zanella,

Organizations: WiLab, University of Bologna, via dell’Università 50, 47521 Cesena, Italy · National Research Council of Italy (CNR), viale del Risorgimento 2, 40136 Bologna, Italy · University of Bologna, Department of Psychology “Renzo Canestrari”, 47521 Cesena, Italy

Abstract

Autonomous agents operating in uncertain environments must balance fast responses with goal-directed planning. Classical MF RL often converges slowly and may induce unsafe exploration, whereas MB methods are computationally expensive and sensitive to model mismatch. This paper presents a human-inspired hybrid RL architecture integrating Pavlovian, Instrumental MF, and Instrumental MB components. Inspired by Pavlovian and Instrumental learning from neuroscience, the framework considers contextual radio cues, here intended as georeferenced environmental features acting as CS, to shape intrinsic value signals and bias decision-making. Learning is further modulated by internal motivational drives through a dedicated motivational signal. A Bayesian arbitration mechanism adaptively blends MF and MB estimates based on predicted reliability. Simulation results show that the hybrid approach accelerates learning, improves operational safety, and reduces navigation in high-uncertainty regions compared to standard RL baselines. Pavlovian conditioning promotes safer exploration and faster convergence, while arbitration enables a smooth transition from exploration to efficient, plan-driven exploitation. Overall, the results highlight the benefits of biologically inspired modularity for robust and adaptive autonomous systems under uncertainty.

Figures & tables

Acronym	Meaning
NS	Neutral Stimulus
CS	Conditioned Stimulus
US	Unconditioned Stimulus
UR	Unconditioned Response
CR	Conditioned Response
PIT	Pavlovian-to-Instrumental Transfer

TABLE I : Neuroscience Acronyms

Fig. 1 : Introduced hybrid learning architecture combining MB and MF processes, inspired by neuroscience [ 22 ] .

Pavlovian learning (neuroscience)	Description	In this paper
Neutral Stimulus (NS)	State with no intrinsic value, that is, before association they are not linked with task outcomes.	$(i)$ GPS-denied regions $(ii)$ Gates These contextual radio features, before association , are not initially associated with task outcomes.
Unconditioned Stimulus (US)	Task-relevant outcome signal.	Target localization metrics, such as the position error bound (PEB), depend on various factors. Among them, we focus on: $(i)$ Agents’ position accuracy, $(ii)$ LOS propagation conditions.
Conditioned Stimulus (CS)	Predictive state acquired through learning.	When GPS-denied regions and gates are considered validated contextual radio-cues predictive of outcomes learned from association: $(i)$ GPS-denied regions anticipate uncertainty on agents’ positions and, thus, decreased PEB $(ii)$ Gates anticipate LOS propagation and, thus, improved PEB
Unconditioned Response (UR)	Immediate reflexive response to reinforcement.	Instantaneous value update driven by reward-based temporal-difference error.
Conditioned Response (CR)	Automatic, stimulus-driven behavioral bias.	$(i)$ GPS-denied region: avoidance tendency induced by Pavlovian values through action-dependent policy modulation. $(ii)$ Gates: approach tendency induced by Pavlovian values through action-dependent policy modulation.

TABLE II : Parallelism between Pavlovian conditioning, and our proposed framework.

Fig. 2 : Simulation environment for autonomous agent network localization.

Parameter	Description	Value
$\alpha_{\text{Q}},\alpha_{\text{Q}_{\text{MB}}},\alpha_{\text{V}}$	Learning rate:
	Initial value	0.55
	Decay rate	0.9985
	Final value	0.09
$\gamma_{\text{Q}},\gamma_{\text{Q}_{\text{MB}}},\gamma_{\text{V}}$	Discount factor	0.98
$\kappa_{0}$	Temperature:

TABLE III : Simulation Parameters

Panel of Figure 6 — Fig. 3 : Agent trajectories with PIT MF at (a) Episode 1, (b) Episode 100, (c) Episode 1200 with (top) and without (bottom) motivational signal, under the same seed. As displayed by the results, without motivational signal, agents are more incline to environmental exploration as per humans [ 52 ] .

Panel of Figure 7 — Fig. 4 : Maximum $Q$ -values heatmaps $\max_{\mathbf{a}}\,Q_{\text{MF}}(\mathbf{s}_{i,t},\mathbf{a})$ for PIT MF of each agent at the end of the final training episode. Each column, from left to right, corresponds to a different agent.

Panel of Figure 8 — Fig. 5 : Pavlovian value heatmaps of Agent 4 at various stages, namely, Episode $10$ , Episode $50$ , Episode $100$ , Episode $1200$ , respectively.

Panel of Figure 9 — Fig. 6 : $P_{\text{MB}}$ as a function of episodes. $P_{\text{MB}}$ averaged over the number of agents (denoted as $\hat{P}_{\text{MB}}$ ) between Instrumental MF - MB (blue solid line) and the proposed PIT MF - MB (Hybrid) (red solid line); and $P_{\text{MB}}$ between agents with Instrumental MF - MB and the proposed PIT MF - MB (Hybrid) algorithm. The shaded areas bound the minimum and maximum values across the agents.

Panel of Figure 10 — Fig. 7 : Agent final trajectories with (a) Instrumental MF - MB , (b) PIT MF - MB (Hybrid) learning approaches.

Panel of Figure 11 — Fig. 8 : Learning rates expressed as steps per episodes and for various combinations of learning types. “w/” and “w/o” refer to the presence, or not, of the motivational signal.

Reinforcement Learning

Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.

See all 1,661 papers

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning
May 31, 2026Bernd Frauenknecht, Devdutt Subhasish, Artur Eisele +2Robotic RL Safe RL
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, 52062 Aachen, Germany
Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration
Mar 4, 2026Danish Rizvi, David BoyleUAV Communication Reinforcement Learning
Imperial College London, UK
Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
May 28, 2026Ahmed Abouelazm, Felix Klingebiel, Philip Schörner +1Reinforcement Learning Risk-Sensitive RL
FZI Research Center for Information Technology, Germany · Karlsruhe Institute of Technology, Germany

May 31, 2026J/K move · Enter open · S save

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

Bernd Frauenknecht, Devdutt Subhasish, Artur Eisele, Friedrich Solowjow+1

Institute for Data Science in Mechanical Engineering, RWTH Aachen University, 52062 Aachen, Germany

Robotic RL Safe RL Model-Based RL Uncertainty Estimation in RL

Open paper AskPDF arXiv

Model-based reinforcement learning (MBRL) infers information about the environment from a learned dynamics model and bears the potential to address open problems such as data efficient and safe learning in robotics. However, inaccuracies of the learned dynamics model are typically exploited by the agent, substantially hampering the capabilities of MBRL methods. We present a framework for dealing with inaccuracies of probabilistic models through targeted handling of uncertainty that effectively mitigates model exploitation. We present recent successes in learning directly on hardware and safe exploration, and discuss future directions for uncertainty-aware MBRL.

Abstract

Reinforcement Learning

Explore similar work

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration

Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving