On the existence of fair zero-determinant strategies in the periodic prisoner's dilemma game

Ken Nakamura; Masahiko Ueda

physics.soc-phMar 20, 2026

On the existence of fair zero-determinant strategies in the periodic prisoner's dilemma game

Authors: Ken Nakamura, Masahiko Ueda

Organizations: Graduate School of Sciences and Technology for Innovation, Yamaguchi University, Yamaguchi 753-8512, Japan · Department of Physics, Tokyo Metropolitan University, Tokyo 192-0397, Japan

Abstract

Stochastic games are a framework for investigating long-term interdependence of multi-agent systems with environmental feedback. When the number of environmental states is one, they are reduced to repeated games. In repeated games, zero-determinant (ZD) strategies attract much attention in evolutionary game theory, since they can unilaterally control payoffs. Especially, fair ZD strategies unilaterally equalize the payoff of the focal player and the average payoff of the opponents, and they were found in several games including the social dilemma games. Although the existence condition of ZD strategies in repeated games was specified, its extension to stochastic games remains largely unclear. Here, we investigate the existence condition of fair ZD strategies in the periodic prisoner's dilemma game, which is one of the simplest stochastic games. The periodic prisoner's dilemma game consists of two environmental states and the two states alternate deterministically. Whereas each stage game is not necessarily the prisoner's dilemma game, the whole game can be regarded as the prisoner's dilemma game on average. We show that fair ZD strategies do not necessarily exist in the periodic prisoner's dilemma game, in contrast to the repeated prisoner's dilemma game. Furthermore, we also prove that the Tit-for-Tat strategy, which imitates the opponent's action, is not necessarily a fair ZD strategy in the periodic prisoner's dilemma game, whereas the Tit-for-Tat strategy is always a fair ZD strategy in the repeated prisoner's dilemma game. Our results highlight difference between ZD strategies in the periodic prisoner's dilemma game and those in the standard repeated prisoner's dilemma game.

Figures & tables

	$C$	$D$
$C$	$R^{(1)},R^{(2)}$	$S^{(1)},T^{(2)}$
$D$	$T^{(1)},S^{(2)}$	$P^{(1)},P^{(2)}$

Table 1: Payoffs in state

\sigma_{1}

Panel of Figure 1 — Figure 1: Relations between $\left\langle s_{1}\right\rangle^{*}$ and $\left\langle s_{2}\right\rangle^{*}$ when player $1$ uses the fair ZD strategy ( 155 ) and ( 156 ) with $c_{1}$ satisfying the equality condition of Eq. ( 154 ) and player $2$ uses $200$ randomly generated memory-one strategies. The payoffs in the left panel are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(1,1,1,1)$ (satisfying $\delta_{R}+\delta_{P}=\delta_{S}+\delta_{T}$ ). The payoffs in the right panel are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(2,1,1,1)$ (not satisfying $\delta_{R}+\delta_{P}=\delta_{S}+\delta_{T}$ ). Each $\left\langle s_{j}\right\rangle^{*}$ is calculated by time average over $10^{6}$ time steps. The fair ZD strategy indeed enforces a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ .

	$C$	$D$
$C$	$2\delta_{R},-2\delta_{R}$	$\bar{S}-\bar{T}+\delta_{S}+\delta_{T},\bar{T}-\bar{S}-\delta_{S}-\delta_{T}$
$D$	$\bar{T}-\bar{S}+\delta_{S}+\delta_{T},\bar{S}-\bar{T}-\delta_{S}-\delta_{T}$	$2\delta_{P},-2\delta_{P}$

Table 3: Relative payoffs in state

\sigma_{1}

Panel of Figure 2 — Figure 2: Relations between $\left\langle s_{1}\right\rangle^{*}$ and $\left\langle s_{2}\right\rangle^{*}$ when player $1$ uses TFT and player $2$ uses $200$ randomly generated memory-one strategies. The payoffs in the left panel are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(1,1,1,1)$ (satisfying $\delta_{R}+\delta_{P}=\delta_{S}+\delta_{T}$ ). The payoffs in the right panel are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(2,1,1,1)$ (not satisfying $\delta_{R}+\delta_{P}=\delta_{S}+\delta_{T}$ ). Each $\left\langle s_{j}\right\rangle^{*}$ is calculated by time average over $10^{6}$ time steps. TFT enforces a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ in the left panel, whereas TFT does not enforce a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ in the right panel.

The condition for the existence of fair ZD strategies	$\left\{\begin{array}[]{c}\bar{S}-\bar{T}+\delta_{S}+\delta_{T}\leq 2\min\left\{\delta_{R},\delta_{P}\right\}\\ \bar{T}-\bar{S}+\delta_{S}+\delta_{T}\geq 2\max\left\{\delta_{R},\delta_{P}\right\}\end{array}\right.$
	or
	$\left\{\begin{array}[]{c}\bar{S}-\bar{T}+\delta_{S}+\delta_{T}\geq 2\min\left\{\delta_{R},\delta_{P}\right\}\\ \bar{T}-\bar{S}+\delta_{S}+\delta_{T}\leq 2\max\left\{\delta_{R},\delta_{P}\right\}\end{array}\right.$
The condition under which TFT is a fair ZD strategy	$\delta_{R}+\delta_{P}=\delta_{S}+\delta_{T}$
The condition for the existence of fair partial ZD strategies	$\delta_{R}=0$ , $\delta_{P}=0$ , $\left\|\delta_{S}+\delta_{T}\right\|\leq\bar{T}-\bar{S}$

Table 5: Summary of this study.

	$C$	$D$
$C$	$R_{1},R_{1}$	$S_{1},T_{1}$
$D$	$T_{1},S_{1}$	$P_{1},P_{1}$

Table 6: Payoffs in state

\sigma_{1}

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: A linear relation between $\left\langle s_{1}\right\rangle^{*}$ and $\left\langle s_{2}\right\rangle^{*}$ when player $1$ uses the fair ZD strategy ( 158 ) and $T_{1}(C|\sigma_{1},\bm{a}^{\prime},\sigma_{2})=0$ and player $2$ uses $200$ randomly generated memory-one strategies. The payoffs are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(3.5,1,1,-1.5)$ . Each $\left\langle s_{j}\right\rangle^{*}$ is calculated by time average over $10^{6}$ time steps. The fair ZD strategy indeed enforces a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ .

Figure 2: A linear relation between $\left\langle s_{1}\right\rangle^{*}$ and $\left\langle s_{2}\right\rangle^{*}$ when player $1$ uses the fair ZD strategy ( 167 ) and ( 168 ) with $c_{1}$ satisfying the equality condition of Eq. ( 166 ) and player $2$ uses $200$ randomly generated memory-one strategies. The payoffs are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(-3,1,1,4)$ . Each $\left\langle s_{j}\right\rangle^{*}$ is calculated by time average over $10^{6}$ time steps. The fair ZD strategy indeed enforces a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ .

Figure 3: A linear relation between $\left\langle s_{1}\right\rangle^{*}$ and $\left\langle s_{2}\right\rangle^{*}$ when player $1$ uses the fair ZD strategy ( 176 ) and ( 177 ) with $c_{1}$ satisfying the equality condition of Eq. ( 175 ) and player $2$ uses $200$ randomly generated memory-one strategies. The payoffs are set to $\left(\bar{R},\bar{S},\bar{T},\bar{P}\right)=(3,0,5,1)$ and $\left(\delta_{R},\delta_{S},\delta_{T},\delta_{P}\right)=(4,1,1,-3)$ . Each $\left\langle s_{j}\right\rangle^{*}$ is calculated by time average over $10^{6}$ time steps. The fair ZD strategy indeed enforces a linear relation $\left\langle s_{1}\right\rangle^{*}=\left\langle s_{2}\right\rangle^{*}$ .

Game Theory

Game theory studies how interacting decision-makers choose strategies, form agreements, and shape outcomes under competition and cooperation.

See all 236 papers

Oct 2, 2026

Performance of Zero-determinant Strategies in Repeated Games without Discounting

Zero-determinant (ZD) strategies are a class of strategies in repeated games, which unilaterally control payoffs. It has been shown that several ZD strategies promote cooperation in social dilemma games. It has been widely believed that the performance of ZD strategies is expressed as ``unilaterally enforce linear relations between payoffs''. However, in order to interpret properties of ZD strategies in repeated games without discounting, previous studies assumed that the limit of the Cesaro average of the probability distribution of the action profile exists. Here, we explain the performance of ZD strategies without this assumption, which leads to a stronger result than previous ones.

Masahiko Ueda

Department of Physics, Tokyo Metropolitan University, Hachioji, Tokyo 192-0397, Japan

Repeated Games

Ask PDF

Oct 27, 2026

Feedback autocratic strategies in stochastic games

A stochastic game is an optimal control problem for multiple agents in a stochastic environment. Stochastic games contain important practical problems, related to environmental issues or traffic flows. If the number of environmental states is one, such stochastic games are called repeated games. In repeated games, several strategies which unilaterally control payoffs have been proposed, and these strategies work in a mechanism similar to feedback control. In this paper, we generalize these strategies as feedback strategies in stochastic games, and investigate their properties. In particular, we provide a necessary and sufficient condition for such feedback strategies to work as controllers of payoffs under some assumptions. We demonstrate the performance of such feedback strategies in several examples related to social dilemmas. Our findings connect the theory of payoff control in repeated games to the theory of feedback control in stochastic games.

Masahiko Ueda

Department of Physics, Tokyo Metropolitan University, Tokyo 192-0397, Japan

Feedback Control

Ask PDF

May 31, 2026

Fairness in two-player zero-sum games with bandit feedback

We study two-player zero-sum games (TPZSGs) with bandit feedback under fairness constraints requiring every action to be played with probability at least

α/m

. Existing instance-dependent results target

\textit{pure}

Nash equilibria, while fairness generically produces

\textit{mixed}

equilibria, a harder learning target. Our key technical tool is a reparametrization: every fair strategy decomposes as

p = (α/m)\mathbf{1} + (1-α)\widetilde{p}

with

\widetilde{p} \in Δ_m

, and substituting into the payoff form yields

p^{\top}Aq = \widetilde{p}^{\top}\widetilde{A} q

for a fair payoff matrix

\widetilde{A} := (1-α)A + α\mathbf{1} c^{\top}

, where

c_j = \tfrac{1}{m}\sum_i A(i,j)

is the column-mean vector. The fair game on

A

is then equivalent to a standard zero-sum game on

\widetilde{A}

, so equilibrium existence, KKT structure, and LP basis stability reduce to classical results applied to

\widetilde{A}

. We derive the fair minimax value, fair Nash equilibrium, fair regret, and a clean dual representation showing the price of fairness is at most

α(1-1/m)

and vanishes whenever the unconstrained equilibrium already has full support. Our main result is an

\widetilde{O}(T^{2/3})

regret bound for an Explore-Then-Commit algorithm,

\texttt{Fair-ETC-TPZSG}

, applicable to general mixed fair equilibria, together with a discussion of why naive action elimination does not readily improve it. When the fair equilibrium has a single dominant action, equivalently when

\widetilde{p}^{\star}

is a vertex of

Δ_m

, the bound sharpens to instance-dependent

\widetilde{O}(1/\widetildeΔ(α)^{2})

, where

\widetildeΔ(α)

is the LP-margin gap.

S Akash, Pratik Gajane

LatentForce.ai Bengaluru, India · Laboratoire d’Informatique Fondamentale d’Orléans University of Orléans, France

Nash Equilibrium Zero-Sum Games Algorithmic Fairness Bandit Learning Adversarial Bandits Nash Equilibrium Computation

Ask PDF

Abstract

Game Theory

Explore similar work

Performance of Zero-determinant Strategies in Repeated Games without Discounting

Feedback autocratic strategies in stochastic games

Fairness in two-player zero-sum games with bandit feedback