Gossip Agents: Reputation through Opinion Dynamics over Evolving Networks

Yutong Wu; Arlei Silva

cs.MAOct 15, 2026

Gossip Agents: Reputation through Opinion Dynamics over Evolving Networks

Authors: Yutong Wu, Arlei Silva

Organizations: Rice University Houston, United States

Abstract

Sustained cooperation among self-interested learning agents is essential in many shared environments. Reputation systems are among the best-studied mechanisms for promoting such cooperation. Existing multi-agent reinforcement learning approaches, however, typically assume that reputation is either globally available to all agents or derived solely from an agent's own observations, leaving largely unexplored how reputational information propagates between agents. We study gossip, the exchange of information about third parties, as a mechanism for reputation diffusion over dynamic communication networks. Agents form private opinions from limited observations and exchange them with their neighbors following DeGroot opinion dynamics, while the communication network changes as agents interact and move. A tunable broadcast radius controls the reach of information, spanning private reputation, local gossip, and globally shared reputation. We study this framework in two $n$ -player social dilemmas with distinct network dynamics: a Prisoner's Dilemma with randomly resampled interaction groups and Harvest, where communication neighborhoods emerge from agents' movements. Our results show that the reach of reputational information shapes cooperation in both environments. Moreover, in Harvest, reputation diffusion increases environmental rewards and leads to more peaceful and sustainable behavior.

Figures & tables

Symbol	Description	Range/Type
$N$	Number of agents	$\mathbb{Z}^{+}$
$\sigma_{ij}^{t}$	Observation signal of agent $i$ towards agent $j$ at time $t$	$\{-1,0,1\}$
$\omega_{ij}^{t}$	Raw opinion of agent $i$ towards agent $j$ at time $t$	$[-1,1]$
$\alpha$	Exponential moving average factor	$[0,1]$
$\mu_{j}^{t}$	Reputation entry of agent $j$ at time $t$	$[-1,1]$
$E_{j}^{t}$	Reputation over a sliding window of agent $j$ at time $t$	$[-1,1]$

Table 1. Description of Variables

Figure 1. The reputation architecture includes an opinion formation phase through agents’ local observations, a temporal aggregation phase that calculates the current step reputation, and a broadcast phase that spreads reputation information.

Figure 2. The Harvest environment with reputation. The full image depicts the global environment, while the white box indicates the local observation available to the observing (blue) agent. The white border is shown only for visualization and is not part of the agent’s observation.

Figure 3. Average number of cooperation steps per agent throughout training for the three reputation methods and No Reputation in the Prisoner’s Dilemma. Reputation methods outperform No Reputation.

Figure 4. Average return per agent throughout training for the three reputation methods and No Reputation in Harvest. The results show that reputation methods outperform No Reputation, with Broadcast All achieving the highest return, followed by Local Broadcast and No Broadcast.

Figure 5. Peacefulness throughout training for the three reputation methods and the No Reputation baseline in Harvest. All reputation methods are more peaceful than the baseline, although Broadcast All achieves slightly lower peacefulness than No Broadcast and Local Broadcast.

Figure 6. Equality throughout training for the three reputation methods and No Reputation in Harvest. Broadcast All exhibits a trade-off, achieving higher sustainability and utility but lower equality compared with the baseline.

Figure 7. Sustainability throughout training for the reputation methods and No Reputation in Harvest. Reputation improves sustainability relative to the baseline, with sustainability increasing with the broadcast radius. No Broadcast also exhibits substantially greater variation across runs.

Figure 8. Punishment accuracy throughout training for the three reputation methods and the No Reputation baseline on Harvest. Punishment accuracy increases with the broadcast radius, indicating that broader opinion sharing helps agents direct punishment toward low-reputation agents.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Description	Value
Num episodes	$2000$
Num steps per episode	$1000$
$\gamma$	$0.99$
Batch size	$128$
Learning rate	$1e{-3}$

Appendix

Table 2. Hyperparameters for PD trainers.

Description	Value
Num episodes	$1e4$
Num steps per episode	$1000$
$\gamma$	$0.99$
Batch size	$100$
Learning rate	$1e{-4}$

Appendix

Table 3. Hyperparameters for Harvest trainers.

Figure 9. Ablation study performed without

Cooperative MARL

Cooperative multi-agent reinforcement learning (MARL) trains agents to coordinate toward shared goals under partial information and decentralized control.

See all 244 papers

The Role of Network Topology and Opponent Information in Shaping Cooperation in Multi-Agent Reinforcement Learning Systems
Aug 29, 2026Seongho Son, Stephen Hailes, Mirco MusolesiCooperative MARL Multi-Agent Systems
Department of Computer Science, University College London · UCL Centre for Artificial Intelligence, University College London · Department of Computer Science and Engineering, University of Bologna
Modelling Opinion Dynamics at Scale with Deep MARL
Jun 5, 2026Lukas Seier, Brandon Kaplowitz, Sebastian Towers +2Opinion Dynamics Multi-Agent Consensus
FLAIR University of Oxford United Kingdom · OWL University of Oxford United Kingdom · OUCE University of Oxford United Kingdom
Self-Referenced Social Preferences: Cooperation without Observing Others Rewards
Oct 6, 2026Mohamed Ayman Mohamed, Harshil Kotamreddy, Marcos Menon JoseCooperative MARL Multi-Agent Reinforcement Learning
Amazon, Vancouver, Canada · Nvidia, Santa Clara, United States · Itaú Unibanco, São Paulo, Brazil

Aug 29, 2026J/K move · Enter open · S save

The Role of Network Topology and Opponent Information in Shaping Cooperation in Multi-Agent Reinforcement Learning Systems

Seongho Son, Stephen Hailes, Mirco Musolesi

Department of Computer Science, University College London · UCL Centre for Artificial Intelligence, University College London · Department of Computer Science and Engineering, University of Bologna

Cooperative MARL Multi-Agent Systems Emergent Behavior in Multi-Agent Systems Cooperative Game Theory

Open paper AskPDF arXiv

Several works have investigated the influence of graph topology on cooperation among artificial agents, while the majority of the literature has focused on modelling agents' adaptation through strategy imitation, which relies solely on the cumulative payoffs of others. This paper investigates scenarios in which each agent learns to play the two-player Iterated Prisoner's Dilemma (IPD) using deep reinforcement learning. Each agent is represented as a node in a graph, where its neighbours constitute the pool of opponents with whom it can interact. During each IPD episode, agents are provided with different types of information about their opponent, consisting of action history and opponent identity. Experimental results across different graph topologies show that the number of neighbours per node and the average path length are the main factors affecting the emergence of cooperation. We also show that, while partner selection fosters mutual cooperation by limiting the diversity of the opponent pool, providing agents with the identity of their opponent hinders the proliferation of cooperative strategies.

Abstract

Cooperative MARL

Explore similar work

The Role of Network Topology and Opponent Information in Shaping Cooperation in Multi-Agent Reinforcement Learning Systems

Modelling Opinion Dynamics at Scale with Deep MARL

Self-Referenced Social Preferences: Cooperation without Observing Others Rewards