Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmas

Alper Demir; Hüseyin Aydın; Kale-ab Abebe Tessera; David Abel; Stefano V. Albrecht

cs.LGFeb 17, 2026

Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmas

Authors: Alper Demir, Hüseyin Aydın, Kale-ab Abebe Tessera, David Abel, Stefano V. Albrecht

Organizations: Izmir University of Economics & University of Edinburgh, Izmir, Turkey · Middle East Technical University & Utrecht University, Ankara, Turkey · University of Edinburgh, Edinburgh, United Kingdom · DeepFlow, London, United Kingdom

Abstract

Sequential Social Dilemmas (SSDs) provide a key framework for studying how cooperation emerges when individual incentives conflict with collective welfare. In Multi-Agent Reinforcement Learning, these problems are often addressed by incorporating intrinsic drives that encourage prosocial or fair behavior. However, most existing methods assume that agents face identical incentives in the dilemma and require continuous access to global information about other agents to assess fairness. In this work, we introduce asymmetric variants of well-known SSD environments and examine how natural differences between agents influence cooperation dynamics. Our findings reveal that existing fairness-based methods struggle to adapt under asymmetric conditions by enforcing raw equality that wrongfully incentivize defection. To address this, we propose three modifications: (i) redefining fairness by accounting for agents' reward ranges, (ii) introducing an agent-based weighting mechanism to better handle inherent asymmetries, and (iii) localizing social feedback to make the methods effective under partial observability without requiring global information sharing. Experimental results show that in asymmetric scenarios, our method fosters faster emergence of cooperative policies compared to existing approaches, without sacrificing scalability or practicality.

Figures & tables

	C	D
C	$\color[rgb]{0,0,1}{R_{i}},\color[rgb]{1,0,0}{R_{j}}$	$\color[rgb]{0,0,1}{S_{i}},\color[rgb]{1,0,0}{T_{j}}$
D	$\color[rgb]{0,0,1}{T_{i}},\color[rgb]{1,0,0}{S_{j}}$	$\color[rgb]{0,0,1}{P_{i}},\color[rgb]{1,0,0}{P_{j}}$

Table 1. The matrix form of a social dilemma game between the row agent

i

and the column agent

j

Panel of Figure 1 — Figure 1. Schelling diagrams for example asymmetric Prisoner’s Dilemma games from the literature, between agent $i$ (in blue) and agent $j$ (in red). For each agent, defective strategies lead to higher rewards.

(a) original C D C

\color[rgb]{0,0,1}{-1},\color[rgb]{1,0,0}{-6}

\color[rgb]{0,0,1}{-3},\color[rgb]{1,0,0}{-5}

\color[rgb]{0,0,1}{0},\color[rgb]{1,0,0}{-8}

\color[rgb]{0,0,1}{-2},\color[rgb]{1,0,0}{-7}

(b) after normalization C D C

\color[rgb]{0,0,1}{0.67},\color[rgb]{1,0,0}{0.67}

\color[rgb]{0,0,1}{0},\color[rgb]{1,0,0}{1}

\color[rgb]{0,0,1}{1},\color[rgb]{1,0,0}{0}

\color[rgb]{0,0,1}{0.33},\color[rgb]{1,0,0}{0.33}

Table 2. Example asymmetric Prisoner’s Dilemma game between the row agent

i

(blue) and the column agent

j

(red) before (a) and after normalization (b).

Agent Type	Reward per Item	Abilities / Effects	Environment(s)
Standard	+1.0	Movement; zap in Harvest only	Harvest , Coins
Low-reward	+0.5	Movement; in Coins , collecting a mismatch coin penalizes the other by $-3.0$	Harvest , Coins
High-reward	+1.5	Movement; in Coins , collecting a mismatch coin penalizes the other by $-1.0$	Harvest , Coins
Wide-zap	+1.0	Larger zapping radius	Harvest
Spawn-biased	+1.0	Collecting a mismatch coin makes only this agent’s coins spawn for 25 steps	Coins

Table 3. Agent types used in the Coins and Harvest . In Coins , “mismatch coin” refers to collecting the other agent’s coin.

Panel of Figure 2 — Figure 2. (a) Coins environment with 2 standard (1 red and 1 blue) agents; (b) Harvest environment with 10 standard agents.

Panel of Figure 3 — Figure 3. Performance of IA, SVO, and their Fair&Local versions in Coins with asymmetry in coin rewards . Fair&LocalIA and Fair&LocalSVO shows faster and more stable convergence to prosocial behavior with higher returns than their counterparts.

Panel of Figure 4 — Figure 4. Performance of IA, SVO, and their Fair&Local versions in Harvest environment with asymmetry in apple rewards . Fair&Local versions lead to higher returns and better sustainability by eliminating the gap in peace between agent types.

Panel of Figure 5 — Figure 5. Performance of IA, SVO, and their Fair&Local versions in Coins environment with asymmetry in coin spawn . The proposed approach shows faster convergence to more cooperative policies.

Panel of Figure 6 — Figure 6. Performance of IA, SVO, and their Fair&Local versions in Harvest with asymmetry in zapping . Fair&Local versions with different weights for different types of agents, lead to more cooperative policies with higher peace and similar returns for each agent type.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7. Schelling diagrams for the asymmetric versions of the environments. Note that there is no asymmetry when defection is prevented in Coins with asymmetry in coin spawn and Harvest with asymmetry in zap radius and thus the cooperating agents are represented as a single line. The results are averaged over 5 experiments where each experiment evaluates the policies for 10 episodes.

Panel of Figure 8 — Figure 8. (a) Harvest environment with 10 standard agents and types of agents (b)-(e) used within this environment.

Panel of Figure 9 — Figure 9. Performance of IA and SVO in symmetric Coins environment. Even the best hyperparameters of SVO cannot perform as well as IA in this environment.

Panel of Figure 10 — Figure 10. Performance of IA and SVO in symmetric Harvest environment. Both show outperforming performance compared to baseline IQL.

	$\alpha$	$\beta$	$\phi_{low-reward}$	$\phi_{standard}$	$\phi_{high-reward}$	$\phi_{wide-zap}$	$\phi_{spawn-biased}$
Coins	0.05	0.1	4.0	13.5	12.0	-	8.0
Harvest	3	0.05	4.0	4.0	6.0	6.0	-

Appendix

Table 4. IA and Fair&LocalIA hyperparameters

	$\theta^{SVO}$	$w$	$\phi_{low-reward}$	$\phi_{standard}$	$\phi_{high-reward}$	$\phi_{wide-zap}$	$\phi_{spawn-biased}$
Coins	$45\degree$	0.004	2.0	4.5	6.0	-	1.5
Harvest	$45\degree$	0.02	1.0	1.0	1.5	1.5	-

Appendix

Table 5. SVO and Fair&LocalSVO hyperparameters

Panel of Figure 11 — Figure 11. The average range of temporally smoothed rewards for Fair&Local versions of IA and SVO in Coins environment with asymmetry in coin rewards and Harvest with asymmetry in apple rewards . The value converge quickly during training.

Panel of Figure 12 — Figure 12. The average range of temporally smoothed rewards for Fair&Local versions of IA and SVO in Coins environment with asymmetry in coin spawn and Harvest environment with asymmetry in zap radius .

Panel of Figure 13 — Figure 13. The average age of agent estimates for the temporally smoothed rewards for Fair&Local versions of IA and SVO in Harvest environment with asymmetry in apple rewards and Harvest environment with asymmetry in zap radius .

Cooperative MARL

Cooperative multi-agent reinforcement learning (MARL) trains agents to coordinate toward shared goals under partial information and decentralized control.

See all 244 papers

$α$ -fair heterogeneous agent reinforcement learning
Jun 11, 2026Yao-hua Franck Xu, Tayeb Lemlouma, Jean-Marie Bonnin +1Algorithmic Fairness Policy Optimization
Orange Innov Lannion, France · IRISA Université de Rennes Lannion, France · IRISA IMT Atlantique Rennes, France
Integrated Altruistic and Fairness Preference Induces Advanced Mutual Cooperation in Sequential Social Dilemmas
Jul 6, 2026Yu Wei, Yukiko Ogura, Yoshiyuki Ohmura +3Social Dilemmas Cooperative MARL
The University of Tokyo
Evolution of fairness in multi-objective reinforcement learning framework
Sep 28, 2026Jingyi Zhang, Xin Ou, Guozhong Zheng +3Multi-Objective Reinforcement Learning Game Theory
School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710061, P. R. China · School of Physical Science and Technology, Inner Mongolia University, Hohhot 010021, China · School of Physics, Ningxia University, Yinchuan 750021, P. R. China

Jun 11, 2026J/K move · Enter open · S save

$α$ -fair heterogeneous agent reinforcement learning

Yao-hua Franck Xu, Tayeb Lemlouma, Jean-Marie Bonnin, Arnaud Braud

Orange Innov Lannion, France · IRISA Université de Rennes Lannion, France · IRISA IMT Atlantique Rennes, France

Algorithmic Fairness Policy Optimization Trust Region Policy Optimization Cooperative MARL Multi-Agent Reinforcement Learning Heterogeneous MARL

Open paper AskPDF arXiv

Cooperation in multi-agent systems is typically optimized through utilitarian objectives that maximize overall efficiency but fail to account for reward distribution, often resulting in inequitable "leader-follower" dynamics. While fairness-based approaches encourage pro-social behaviors where every agent benefits from cooperation, many current algorithms - including those utilizing reward shaping - break the stationarity of Markov Games or lack rigorous theoretical guarantees. This creates a critical gap between fair objective methods and theoretically safe learning frameworks. We propose a novel framework that bridges

α

-fairness with Heterogeneous-Agent Trust Region Learning (HATRL), ensuring monotonic improvement and convergence toward Nash Equilibria. Our approach leverages a fair advantage function that dynamically weights agent utilities based on their expected returns, allowing the global objective to transition from purely utilitarian efficiency to

α

-fairness welfare based on the parameter

α

. We introduce two practical algorithms,

α

-fair HATRPO and

α

-fair HAPPO, and demonstrate through experiments in sequential social dilemmas like CleanUp and CommonHarvest that they perform better than HATRL's algorithms from a utilitarian point of view while achieving socially higher outcomes.

Abstract

Cooperative MARL

Explore similar work

ααα-fair heterogeneous agent reinforcement learning

Integrated Altruistic and Fairness Preference Induces Advanced Mutual Cooperation in Sequential Social Dilemmas

Evolution of fairness in multi-objective reinforcement learning framework

$α$ -fair heterogeneous agent reinforcement learning