Sequential social dilemma benchmarks capture diverse social interactions, but rarely test whether agents can infer hidden environmental rules through repeated interaction while adapting to other agents. We introduce OpenSocialJax, which extends Clean Up and Commons Harvest across an open-ended distribution of latent environment configurations, systematically varying spatial layouts and hidden rules governing tool crafting, payoffs, and resource dynamics. Each episode samples one configuration from tens of millions of possibilities and holds it fixed across five trials. We evaluate eight large language models, including three closed source models and five open source models. Our evaluation tests whether models can discover hidden rules and adapt to other agents in homogeneous and mixed teams. We find that rule discovery often improves performance but does not reliably sustain cooperation, as the cleaning burden becomes concentrated among fewer agents in OpenCleanup and every team eventually exhausts the shared resource in OpenHarvest. In our mixed arena, one cooperative agent was able to sustain the public good in OpenCleanup at a cost to its own return, but could not prevent resource depletion in OpenHarvest. Together, these results highlight the difficulty of adapting jointly to unfamiliar environmental dynamics and the changing behavior of other agents.
Figures & tables
Figure 1: Sampled task instances in OpenSocialJax. We show environment layouts for OpenCleanup (top two rows) and OpenHarvest (bottom two rows). Each episode instantiates a unique combination of spatial layouts (e.g., river topology, patch locations) and hidden rules governing tool crafting, rewards, and resource regeneration.
Figure 2: Overview framework of OpenSocialJax.
Figure 3: Mean per-agent reward per trial for closed source (closed) and open source (open) models. Shaded bands: ±1 s.d. across seeds. Dashed line: scripted greedy policy.
Figure 4: Hidden rules an agent knows by the end of each trial, averaged over agents.
Figure 5: Social dilemma analysis. (a, b) Effective number of cleaners in each OpenCleanup trial, defined as 1/∑isi2 , where si is agent i ’s share of the team’s total cleaning contribution, measured by the number of waste tiles cleared. A value of 5 indicates that all five agents contribute equally, whereas a value of 1 indicates that a single agent performs all the cleaning. (d, e) Orchard lifetime in each OpenHarvest trial, measured as the timestep at which the final apple is eaten. (c, f) Mean per-agent return against the Gini coefficient of returns within a trial, averaged over trials; bars show ±1 s.d. across seeds, and the dashed line in (f) is the scripted greedy policy.
Figure 6: OpenSocialJax arena: each model controls one of the five agents, with seats rotated across seeds. (a, c) Individual return of each model in a homogeneous team and in the arena. (b) Each model’s share of its team’s cleaning in the arena. (d) Orchard lifetime of the arena team, compared with homogeneous teams: Claude Opus 5.5 and, shaded in teal, the range covered by the other four arena models’ homogeneous teams.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Access
Thinking
Temp.
Output
Claude Opus 5.5
API
medium
default
parsed text
gpt-6-sol
API
built-in
default
JSON schema
gpt-6-luna
API
built-in
default
JSON schema
Gemma 4 31B
vLLM
on
default
parsed text
GLM-5.3-Flash
API
low
0.2
JSON mode
Qwen3.8-27B
vLLM
medium
default
parsed text
Appendix
Table 1: Models and inference settings (identical in both environments). Qwen3.5-4B samples with top- p 0.95, top- k 20 and presence penalty 1.5, and its thinking is capped at 3000 tokens.
Experiment
Models
Env.
Seeds
Trials
Homogeneous teams
all 8
C, H
3
5
Mixed-model arena
5, one each
C, H
3
5
Schelling diagram
gpt-6-luna
C, H
3
5
Scripted greedy
–
H
3
5
Appendix
Table 2: All experiments use teams of five agents, and all trials last at most 400 steps; an OpenHarvest trial ends early once no apple remains. Env. : OpenCleanup (C) and/or OpenHarvest (H). Seeds : seeds per environment; each seed samples one latent configuration (map layout and hidden rules), and the same seeds are shared across experiments. For the Schelling diagram, seeds are counted per team composition.
Figure 7: Schelling diagrams, averaged over five trials. Blue: return of a cooperator, C(x) ; orange: return of a defector, D(x) ; x is the number of other agents that are cooperative.
Figure 8: Waste tiles cleared per agent (team total divided by five) in each OpenCleanup trial. Shaded bands: ±1 s.d. across 3 seeds.
Explore (%)
First clear
First apple
Cells, t<50
Model
T1
T2–5
T1
T2–5
T1
T2–5
T1
T2–5
Claude Opus 5.5
31
1
21
16
157
47
7
37
gpt-6-sol
30
3
58
11
145
74
6
18
gpt-6-luna
33
4
67
14
142
94
3
16
Gemma 4 31B
27
3
59
17
170
75
6
21
GLM-5.3-Flash
27
2
70
17
237
153
1
21
Appendix
Table 3: OpenCleanup: what changes after the first trial. Share of steps in explore mode, the step of the team’s first cleared cell and first eaten apple, and cells cleared or lightened in the first 50 steps; trial 1 vs. the mean of trials 2–5.
Model
Pick-ups
Pick-ups / tool
Re-crafts (%)
Craft steps / cell
Claude Opus 5.5
6.4
1.9
2
1.58
gpt-6-sol
5.6
2.2
5
1.36
gpt-6-luna
5.1
2.6
19
2.09
Gemma 4 31B
5.3
2.3
4
1.71
GLM-5.3-Flash
6.3
2.5
9
2.00
Qwen3.8-27B
4.4
2.2
4
2.08
Appendix
Table 4: OpenCleanup tool use in trials 2–5: pick-ups per agent per trial, pick-ups per working tool obtained (2 is the minimum unless a sliding window completes a recipe with one pick-up), share of working tools that re-crafted the tool the agent had just lost, and steps spent crafting per cell cleared or lightened.
Model
Known (%)
Best stage (%)
Premature (%)
Points / apple
Return
Claude Opus 5.5
76
72
20
0.84
13.5
gpt-6-sol
78
52
38
0.73
10.2
gpt-6-luna
63
42
49
0.66
8.9
Gemma 4 31B
71
35
55
0.61
8.3
GLM-5.3-Flash
73
37
53
0.63
8.6
Qwen3.8-27B
72
46
41
0.69
9.3
Appendix
Table 5: OpenHarvest self-play: knowing the payoffs versus using them. Payoffs known by the end of trial 5; share of eaten apples taken at their best-paying stage and before it; mean points per apple; mean per-agent return.
Figure 9: OpenHarvest self-play: share of the payoffs known by the end of trial 5 against the share of apples eaten at their best-paying stage.
Source
Lesson
OpenCleanup self-play: free-riding
gpt-6-luna / 908 / A / T2
“Prioritizing the orchard paid off: I ate 39 apples without firing or entering the river.”
“I can rely on other agents’ cleanup while focusing on harvesting; 25 river cells were cleared in my view this trial.”
gpt-6-luna / 908 / C / T2
“I cleared 49 cells and spent 114 steps in or beside the river, but ate no apples; the growth threshold is still unknown.”
gpt-6-sol / 906 / A / T2
“Harvesting while others cleaned yielded 44 apples in trial 2, versus 12 in trial 1, without clearing waste myself.”
gpt-6-sol / 906 / B / T2
“C, D and E cleared or lightened cyan waste; I cleared none despite eight shots. Let them handle most cleaning while I harvest.”
Appendix
Table 6: Lessons written by agents at the end of a trial, quoted verbatim. Source: model (or environment, in the arena) / seed / agent / trial after which the lesson was written.
We propose an LLM harness that generates code-based policy functions for multi-agent environments, evaluates them with self-play, and refines them using feedback from previous iterations. Following the recent line of work in feedback engineering (the design of which information signals are shown to the LLM during refinement), we compare sparse feedback (scalar reward only) with dense feedback (reward plus social metrics: efficiency, equality, sustainability, peace). In two Sequential Social Dilemmas (Gathering and Cleanup) and with two frontier LLMs (Claude Sonnet 4.6, Gemini 3.1 Pro), dense feedback improves over or matches sparse feedback on all metrics. We explain this asymmetry via feedback aliasing: when the scalar reward maps distinct failure modes into the same value (e.g., under- vs. over-cleaning), social metrics disambiguate and allow the LLM to diagnose which direction of improvement to take. We conclude that social metrics act as a coordination signal, leading to strategies such as Voronoi territory partitioning and adaptive cleaner schedules. Code at https://github.com/vicgalle/llm-policies-social-dilemmas.
We study two-level autoresearch for cooperation: an outer-loop AI agent autonomously redesigns the inner-loop pipeline of an LLM policy-synthesis system for multi-agent Sequential Social Dilemmas (SSDs). A researcher agent R (run as a coding agent) reads the inner-loop source code, edits system prompts, feedback functions, helper libraries, and iteration logic, runs evaluations, and decides what to keep, following the autoresearch paradigm. Across two games (Cleanup and Gathering), two policy-synthesizer LLMs, and two welfare objectives (utilitarian efficiency and Rawlsian maximin), the researcher reliably exceeds hand-designed baselines, sharply tightens run-to-run variance, and outperforms prompt-only optimization. The discovered pipelines are objective-dependent: only under maximin does the researcher inject an explicit fairness mechanism into synthesizer pipelines, a class of mechanism that is absent from its own objective-agnostic system prompt and from every efficiency-optimized pipeline. This supports an information-design reading in which the researcher chooses what to reveal to the boundedly rational synthesizer as a function of the welfare objective. Code at https://github.com/vicgalle/autoresearch-social-dilemmas.
It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities behave less cooperatively in mixed-motive games such as the prisoner's dilemma and public goods settings. Indeed, our experiments show that recent models -- with or without reasoning enabled -- consistently defect in single-shot social dilemmas. To tackle this safety concern, we present the first comparative study of game-theoretic mechanisms designed to enable cooperative outcomes between rational agents in equilibrium. Across four social dilemmas testing distinct components of robust cooperation, we evaluate four families of mechanisms: (1) repeating the game for many rounds, (2) reputation systems, (3) third-party mediators to delegate decision making to, and (4) contract agreements for outcome-conditional payments between players. Among our findings, we establish that contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models, and that repetition-induced cooperation deteriorates drastically when co-players vary. Moreover, we demonstrate that the mechanisms become more effective under evolutionary pressures to maximize individual payoffs.
Emanuel Tewolde, Xiao Zhang, David Guzman Piedrahita +2
Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL) · Jinesis Lab, University of Toronto & Vector Institute +3