Feedback autocratic strategies in stochastic games

Masahiko Ueda

physics.soc-phOct 27, 2026

Feedback autocratic strategies in stochastic games

Authors: Masahiko Ueda

Organizations: Department of Physics, Tokyo Metropolitan University, Tokyo 192-0397, Japan

Abstract

A stochastic game is an optimal control problem for multiple agents in a stochastic environment. Stochastic games contain important practical problems, related to environmental issues or traffic flows. If the number of environmental states is one, such stochastic games are called repeated games. In repeated games, several strategies which unilaterally control payoffs have been proposed, and these strategies work in a mechanism similar to feedback control. In this paper, we generalize these strategies as feedback strategies in stochastic games, and investigate their properties. In particular, we provide a necessary and sufficient condition for such feedback strategies to work as controllers of payoffs under some assumptions. We demonstrate the performance of such feedback strategies in several examples related to social dilemmas. Our findings connect the theory of payoff control in repeated games to the theory of feedback control in stochastic games.

Figures & tables

	$C$	$D$
$C$	$R_{1},R_{1}$	$S_{1},T_{1}$
$D$	$T_{1},S_{1}$	$P_{1},P_{1}$

Table 1: Payoffs in state

\sigma_{1}

Figure 1: A finite automaton in mode

\overline{a}_{j}

Figure 2: A finite automaton in mode

\underline{a}_{j}

	$C$	$D$
$C$	$\bar{R}+\delta_{R},\bar{R}-\delta_{R}$	$\bar{S}+\delta_{S},\bar{T}-\delta_{T}$
$D$	$\bar{T}+\delta_{T},\bar{S}-\delta_{S}$	$\bar{P}+\delta_{P},\bar{P}-\delta_{P}$

Table 3: Payoffs in state

\sigma_{1}

Feedback Control

Feedback control uses ongoing measurements to adjust actions, stabilize dynamical systems, track goals, and respond to disturbances and constraints.

See all 255 papers

On the existence of fair zero-determinant strategies in the periodic prisoner's dilemma game
Mar 20, 2026Ken Nakamura, Masahiko UedaGame Theory Repeated Games
Graduate School of Sciences and Technology for Innovation, Yamaguchi University, Yamaguchi 753-8512, Japan · Department of Physics, Tokyo Metropolitan University, Tokyo 192-0397, Japan
Performance of Zero-determinant Strategies in Repeated Games without Discounting
Oct 2, 2026Masahiko UedaRepeated Games
Department of Physics, Tokyo Metropolitan University, Hachioji, Tokyo 192-0397, Japan
Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?
Feb 27, 2026Kush Grover, Markel Zubia, Debraj Chakraborty +3Robust Markov Decision Processes
Fondazione Bruno Kessler, Italy · Ruhr University Bochum, Germany · Nanyang Technological University, Singapore +3

Mar 20, 2026J/K move · Enter open · S save

On the existence of fair zero-determinant strategies in the periodic prisoner's dilemma game

Ken Nakamura, Masahiko Ueda

Graduate School of Sciences and Technology for Innovation, Yamaguchi University, Yamaguchi 753-8512, Japan · Department of Physics, Tokyo Metropolitan University, Tokyo 192-0397, Japan

Game Theory Repeated Games

Open paper AskPDF arXiv

Stochastic games are a framework for investigating long-term interdependence of multi-agent systems with environmental feedback. When the number of environmental states is one, they are reduced to repeated games. In repeated games, zero-determinant (ZD) strategies attract much attention in evolutionary game theory, since they can unilaterally control payoffs. Especially, fair ZD strategies unilaterally equalize the payoff of the focal player and the average payoff of the opponents, and they were found in several games including the social dilemma games. Although the existence condition of ZD strategies in repeated games was specified, its extension to stochastic games remains largely unclear. Here, we investigate the existence condition of fair ZD strategies in the periodic prisoner's dilemma game, which is one of the simplest stochastic games. The periodic prisoner's dilemma game consists of two environmental states and the two states alternate deterministically. Whereas each stage game is not necessarily the prisoner's dilemma game, the whole game can be regarded as the prisoner's dilemma game on average. We show that fair ZD strategies do not necessarily exist in the periodic prisoner's dilemma game, in contrast to the repeated prisoner's dilemma game. Furthermore, we also prove that the Tit-for-Tat strategy, which imitates the opponent's action, is not necessarily a fair ZD strategy in the periodic prisoner's dilemma game, whereas the Tit-for-Tat strategy is always a fair ZD strategy in the repeated prisoner's dilemma game. Our results highlight difference between ZD strategies in the periodic prisoner's dilemma game and those in the standard repeated prisoner's dilemma game.

Abstract

Feedback Control

Explore similar work

On the existence of fair zero-determinant strategies in the periodic prisoner's dilemma game

Performance of Zero-determinant Strategies in Repeated Games without Discounting

Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?