Body-Reservoir Governance in Repeated Games: Embodied Decision-Making, Dynamic Sentinel Adaptation, and Complexity-Regularized Optimization

Yuki Nakamura

cs.GTFeb 24, 2026

Body-Reservoir Governance in Repeated Games: Embodied Decision-Making, Dynamic Sentinel Adaptation, and Complexity-Regularized Optimization

Authors: Yuki Nakamura

Organizations: The Open University of Japan

Abstract

Standard game theory explains cooperation in repeated games through conditional strategies such as Tit-for-Tat (TfT), but these require continuous computation that imposes physical costs on embodied agents. We propose a three-layer Body-Reservoir Governance (BRG) architecture: (1) a body reservoir (echo state network) whose $d$ -dimensional state performs implicit inference over interaction history, serving as both decision-maker and anomaly detector, (2) a cognitive filter providing costly strategic tools activated on demand, and (3) a metacognitive governance layer with receptivity parameter $α\in [0,1]$ . At full body governance ( $α=1$ ), closed-loop dynamics satisfy a self-consistency equation: cooperation is expressed as the reservoir's fixed point, not computed. Strategy complexity cost is defined as the KL divergence between the reservoir's state distribution and its habituated baseline. Body governance reduces this cost, with action variance decreasing up to $1600\times$ with dimension $d$ . A dynamic sentinel generates a composite discomfort signal from the reservoir's own state, driving adaptive $α(t)$ : near baseline during cooperation, rapidly dropping upon defection to activate cognitive retaliation. Overriding the body incurs thermodynamic cost proportional to internal state distortion. The sentinel achieves the highest payoff across all conditions, outperforming static body governance, TfT, and EMA baselines. A dimension sweep ( $d \in \{5,\ldots,100\}$ ) shows implicit inference scales with bodily richness ( $23\times$ to $1600\times$ variance reduction), attributable to reservoir dynamics. A phase diagram in $(d, τ_{\mathrm{env}})$ space reveals governance regime transitions near $d \approx 20$ . The framework reinterprets cooperation as the minimum-dissipation response of an adapted dynamical system -- emergent from embodied dynamics rather than computed.

Figures & tables

Symbol	Meaning
$a_{i}(t)\in[0,1]$	Action of agent $i$ at round $t$
$\mathbf{x}(t)\in\mathbb{R}^{d}$	Reservoir state vector
$d$	Reservoir dimension (number of neurons)
$W\in\mathbb{R}^{d\times d}$	Recurrent weight matrix
$W_{\mathrm{in}}\in\mathbb{R}^{d\times 2}$	Input weight matrix
$w_{\mathrm{out}}\in\mathbb{R}^{1\times d}$	Readout weight vector

Table 1: Summary of principal notation.

Figure 1: Self-consistent convergence of body output at

\alpha=1

. The body readout

a^{*}(t)

converges to approximately

0.98

within a few rounds from a cooperative initial condition. The rapid convergence confirms the local stability of the cooperative self-consistent fixed point (Proposition 3.6 ).

Figure 2: KL divergence, action variance, and mean payoff as functions of metacognitive receptivity

\alpha

. KL divergence reaches its minimum near

\alpha\approx 0.70

(not at

\alpha=1

), reflecting the two-source effect discussed in Section 6.2 . Action variance decreases monotonically (

\sim 250\times

reduction from

\alpha=0

\alpha=1

). The steep initial drop in both quantities suggests that even moderate body governance captures most of the smoothing benefit.

Figure 3: Perturbation response to a sustained opponent defection block. At

\alpha=1

(body governance), the output changes gradually due to the insulating effect of the self-feedback loop. At

\alpha=0

(TfT), the action drops immediately to

0

(full retaliation). The dynamic sentinel (purple) combines the best of both: it detects the defection through body discomfort and activates cognitive retaliation, then smoothly returns to body governance when cooperation resumes.

Figure 4: Habituation dynamics for different governance modes. (a) State-space KL divergence from the noisy state distribution to the habituated baseline: body governance (

\alpha=1

) and the dynamic sentinel provide consistently lower internal distortion than TfT (

\alpha=0

). (b) Action variance on log scale:

\alpha=1

and the dynamic sentinel achieve

\sim 250\times

variance reduction relative to

\alpha=0

. (c) Mean body readout remains near the self-consistent value (

\approx 0.98

) throughout. The coupled habituation proceeds for

300

epochs with measurements every

15

epochs, averaged over

20

seeds.

Figure 5: Free energy landscape

\mathcal{F}(\alpha,H)

. (a) State-space complexity cost

D_{\mathrm{KL}}(p_{\alpha}\|p_{H})

\alpha

for different habituation depths

H

. (b) Free energy at

\lambda=3

: at

H=0

the large KL cost drives the minimum to

\alpha^{*}=1

; as habituation deepens, the optimum settles at the interior value

\alpha^{*}\approx 0.7

. (c) Optimal receptivity

\alpha^{*}

H

for three metabolic cost levels (

\lambda=1,3,8

): all converge to

\alpha^{*}\approx 0.7

at moderate habituation. Averaged over

20

seeds.

Figure 6: Dynamic sentinel response to a multi-phase opponent schedule. (a) The receptivity

\alpha(t)

drops sharply upon defection detection and recovers slowly during cooperative phases; red shading marks the defection block, orange shading marks the noisy phase. (b) Action trajectories: the dynamic sentinel (purple) retaliates during defection but maintains cooperative smoothing otherwise. (c) Cumulative payoff: the dynamic sentinel achieves the highest total payoff across all conditions. Averaged over

20

seeds.

Figure 7: Sentinel parameter sensitivity. Each panel varies one parameter while holding others at defaults, showing mean payoff (blue, left axis) and mean

\alpha

(orange, right axis). The sentinel is robust across a wide parameter range, with the sharpest sensitivity to

\theta

(noise threshold). Averaged over

20

seeds; error bars show SEM.

Figure 8: Reservoir dimension sweep (

d\in\{5,\ldots,100\}

). (a) KL divergence from habituated baseline at

\alpha=0

(red) and

\alpha=1

(blue): body governance consistently reduces KL. (b) Variance reduction ratio (log scale) grows sharply with

d

, from

23\times

d=5

1600\times

d=75

. (c) Free-energy-minimizing

\alpha^{*}

(at

\lambda=3

) is

1.0

for very small reservoirs but stabilizes at

0.6

–

0.8

for

d\geq 10

. (d) Ridge regularization decomposition:

\|w_{\mathrm{out}}\|^{2}

and action variance under scaled (

\lambda_{d}\propto d/30

) vs. fixed regularization, showing that the majority of variance reduction comes from reservoir dynamics rather than regularization alone. Averaged over

20

seeds; error bars show SEM.

Figure 9: Phase transition analysis. (a) FE-minimizing $\alpha^{*}$ vs. dimension $d$ , identifying the critical dimension $d_{c}$ above which body trust dominates. (b) Sentinel collapse: minimum $\alpha$ and payoff advantage vs. defection block length at $d=30$ . (c) Detection time is approximately constant across block lengths. (d) Joint $(d,\tau_{\mathrm{env}})$ phase diagram: sentinel payoff advantage over TfT. Red–blue gradient indicates cognition-dependent vs. body-trust regimes; dashed line shows the approximate phase boundary. Averaged over $20$ seeds.

Agent	Var. reduction	Perturbation depth	Recovery (steps)	Mean payoff
Raw TfT ( $\gamma=0$ )	$1\times$	$0.00$	$1$	$2.89$
EMA ( $\gamma=0.5$ )	$3\times$	$0.00$	$5$	$2.89$
EMA ( $\gamma=0.9$ )	$20\times$	$0.00$	$29$	$2.89$
EMA ( $\gamma=0.95$ )	$41\times$	$0.01$	$59$	$2.89$
EMA ( $\gamma=0.99$ )	$158\times$	$0.37$	$200$	$2.88$
Reservoir ( $\alpha=1$ )	$\mathbf{461\times}$	$\mathbf{0.89}$	$\mathbf{<1}$	$2.74$

Table 2: Reservoir vs. EMA-filtered TfT baselines. Variance reduction is relative to raw TfT within this experiment (independent seeds from Experiment 2, hence the higher reservoir ratio of

461\times

vs. the

\sim 250\times

reported at

d=30

in Experiment 2). Perturbation depth is the minimum action during a

100

-round defection block; higher values indicate greater absorption. Recovery time is steps to regain

95\%

of pre-perturbation output. Means across

20

seeds.

Repeated Games

Repeated games study how strategies, cooperation, and competition develop when players interact repeatedly and respond to past actions.

See all 21 papers

Apr 30, 2026

A Collective Variational Principle Unifying Bayesian Inference, Game Theory, and Thermodynamics

Collective intelligence emerges across biological, physical, and artificial systems without central coordination, yet a unifying principle governing such behaviour remains elusive. The Free Energy Principle explains how individual agents adapt through variational inference, while game theory formalises strategic interactions. Here we introduce the Game-Theoretic Free Energy Principle, a unified framework showing that multi-agent systems performing local free-energy minimisation implicitly implement a stochastic game. We prove that, under bounded rationality and local information constraints, stationary points of collective free energy correspond to approximate Nash equilibria of an induced game. Conversely, a broad class of cooperative games admits a variational representation in which equilibria arise as Gibbs distributions over coalitions, establishing a bridge between Bayesian inference and strategic interaction. To characterise higher-order effects, we introduce a free-energy formulation of the Harsanyi dividend, isolating irreducible multi-agent synergy. This yields a predictive theory of cooperation, including a falsifiable non-monotonic relationship between sensory precision and agent influence. We validate this prediction across neural, biological, and artificial multi-agent systems. These results identify a common variational principle underlying inference, thermodynamics, and game-theoretic equilibrium.

Djamel Bouchaffra, Faycal Ykhlef, Mustapha Lebbah +1

1DAVID Lab, University of Paris-Saclay, UVSQ, 78035 Versailles, France. · 2TSAM, ASM, Centre for Development of Advanced Technologies, Algiers, Algeria. · 3LIPN, UMR CNRS 7030, Sorbonne Paris Nord University, Villetaneuse, France.

Nash Equilibrium Cooperative Game Theory Game Theory Bayesian Inference Free Energy Principle Multi-Agent Systems+2

Ask PDF

Sep 9, 2026

Tapes Together Strong: The Co-evolution of Computation and Cooperation

How does cooperation evolve in complex agentic systems? Prior work in evolutionary game theory studies why individuals are incentivized to cooperate by isolating social interactions from the physical costs of behavior, while artificial life models traditionally study emergent self-replication without formalizing the dilemma between acquiring resources and preserving the shared energy needed to reproduce. In contrast, we introduce Autopoietic Game Theory, a computational model where social interactions, replication mechanisms, and their associated computational costs are endogenous and simultaneously co-evolving. We study these dynamics using a computational substrate of randomly initialized programs in Z80 machine code, showing empirically, and motivating with a simplified theoretical model, that embedding a social dilemma directly into the physics of computation can favor the emergence of self-replicating, cooperative strategies. When resources are scarce, our analysis shows that defection can become self-limiting even in well-mixed populations: parasitic stealing destroys shared energy, slows execution, and can prevent reliable replication. Empirically, evolved programs suppress stealing across several Z80 environments, while spatial assortment further supports structural complexity and task performance. We further show that the framework can incorporate exogenous pressures, such as math tasks structured as sequential social dilemmas, when rewards are tied to computation budgets. These results suggest that coupling an agent's capacity for computation to its available energy transforms cooperation into a dominant scaffolding for building sustainable, self-organizing systems.

Kunal Jha, Francesco Cicala, Blaise Agüera y Arcas +4

Dept. of Computer Science, University of Washington. · Google Paradigms of Intelligence. · School of Computer Science, McGill University; Mila. +1

Artificial Life Evolutionary Game Theory

Ask PDF

Sep 23, 2026

Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence

Cooperation emergence is a central problem in multi-agent systems because decentralized agents must coordinate while adapting to the changing behavior of others. Evolutionary game theory identifies strategically stable outcomes, but stability under a population adjustment dynamic need not imply that finite-sample learning agents can reach the same outcome through local reward feedback. We study this distinction in a transparent three-agent governance-motivated game involving a government, a platform firm, and users. We derive replicator dynamics for the fixed stage-game incentives, evaluate the cooperative evolutionary basin on a symmetric initial-condition grid, and compare it with learning-basin estimates for three decentralized value-based learners. The learning analysis uses independent Q-learning with

\varepsilon

-greedy action selection, scaled Boltzmann exploration, and SA--EA BQL under the same payoff environment and outcome criterion. The evolutionary basin has volume

V_E=1.00

on the sampled grid. The empirical learning basin is

0.88

for

\varepsilon

-IQL and

0.00

for both scaled Boltzmann and SA--EA BQL. Diagnostic traces show that broader action diversity and nonzero value separation can coexist with failure to sustain the cooperative joint action in this fixed configuration. These results indicate that evolutionary stability and learning accessibility are distinct properties of a coupled game--learning system. The shared-bike setting is a motivating application; the broader contribution is a framework for comparing population-level stability with the finite-sample accessibility of cooperation under specified multi-agent learning dynamics.

Yijie Wang

Liupanshui Normal University Liupanshui, Guizhou, China

Emergent Behavior in Multi-Agent Systems Cooperative MARL Evolutionary Game Theory

Ask PDF

Abstract

Repeated Games

Explore similar work

A Collective Variational Principle Unifying Bayesian Inference, Game Theory, and Thermodynamics

Tapes Together Strong: The Co-evolution of Computation and Cooperation

Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence