cs.AIOct 10, 2026
SaveMARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL
Organizations: School of Engineering, University of Edinburgh, Edinburgh, United Kingdom · School of Informatics, University of Edinburgh, Edinburgh, United Kingdom · School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Abstract
We present MARL-BattleGrounds (MARL-BGs), an easy-to-use JAX-native benchmark for heterogeneous and competitive multi-agent reinforcement learning (MARL), designed to be sample-efficient, computationally efficient, and researcher-centric. MARL benchmarks are typically simple and efficient, like SMAX, where winning reduces to unit micromanagement, or rich and costly, like Dota 2. MARL-BGs fills this missing middle. Its first mode, Team Deathmatch, is a simultaneous turn-based game between two teams of five heterogeneous agents. The five classes have distinct basic, passive, and ultimate abilities designed to synergize with allies and counter enemies. Our 52 hand-designed maps with diverse obstacles include a 12-map curriculum and ten held-out validation and test maps. MARL-BGs provides two canonical evaluation modes. First, eight endgame scenarios form a hard generalization challenge: each asks a team to win an unseen, nearly lost position against a strong reactive controller within 5-10 ticks. Second, a monthly round-robin tournament retrains each submission on equal compute, plays it against every member of the current "Big Nine" field, and publishes the Elo results on the official leaderboard. All entrants are publicly released as downloadable opponents. MARL-BGs records 11,192 metrics per game in 28 topics. We also include three strong scripted opponents, an analytical replay viewer, and off-the-shelf tools for self-play, cross-play, curriculum learning, and zero-shot coordination evaluation. We even provide an API for LLM-based agents and use it to evaluate Qwen3.5-9B against our strongest policy. In our BattleClient, humans can play alongside or against any mix of policies, enabling human-AI evaluation. We benchmark MARL-BGs with six additional deep MARL baselines, which we release. On one RTX 5090 with 1,024 parallel games, the simulator runs 50,000 5v5 game steps per second.
Figures & tables
| Lightweight suites | Games | Coordination suites | |||||||||||||||
| Feature | SMAX ( Rutherford et al., 2024 ) | Gigastep ( Lechner et al., 2023 ) | Lux S3 ( Tao et al., 2024 ) | VMAS ( Bettini et al., 2024 ) | MAgent2 ( Zheng et al., 2018 ) | MPE ( Mordatch and Abbeel, 2018 ; Lowe et al., 2017 ) | SMAC ( Samvelyan et al., 2019 ) | SMACv2 ( Ellis et al., 2023 ) | GRF ( Kurach et al., 2020 ) | HoK Arena ( Wei et al., 2022 ) | Mini HoK ( Liu et al., 2025 ) | Dota 2 ( OpenAI et al., 2019 ) | Overcooked ( Carroll et al., 2019 ) | OvercookedV2 ( Gessler et al., 2025 ) | MP 2.0 ( Agapiou et al., 2022 ) | Hanabi ( Bard et al., 2020 ) | MARL-BGs |
| Game | |||||||||||||||||
| JAX / GPU-native simulator | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| Classes with distinct abilities | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | |||||||||
| Healing, auras, crowd control, ultimates | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | |||||
| Line of sight and obstacle cover | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✓ | |||||||||
Table 1. Features of MARL benchmarks, grouped by family: ✓ yes, partial, ✗ no. Section S2 of the supplementary material gives the rule and sources for every mark. † Committed for the public release.
Figure 1. Training map 34, Infinity, with the official tournament’s mirrored 5v5 on the spawn pads, from bottom to top: Mage, Warrior, Hunter, Rogue, and Priest. Watch real games and try our interactive demos at \weblink https://marl-battlegrounds.commarl-battlegrounds.com. A top-down drawing of a 20 by 10 unit map. Team A's five agents stand on spawn pads along the left edge and Team B's along the right edge. Each team's Red Zone, the strip at its own edge, is outlined and labeled. Labels point to a wall and a pillar.
Figure 2. The five classes, each with its health, strength ( ), and weakness ( ). Solid arrows point from a class to the enemy class it counters; dashed arrows point to an ally it helps. Not every interaction is drawn to reduce clutter. Five boxes, one per class, arranged in a pentagon: Warrior at the top, Rogue at the right, Priest at the bottom right, Mage at the bottom left, and Hunter at the left. Each box gives the class's health, one strength, and one weakness. Solid arrows around the outside form a cycle of counters: the Warrior counters the Rogue (Charge stuns and slows it), the Rogue counters the Priest (it catches the Priest, and Poison halves healing), the Priest counters the Mage (a full heal undoes its Burst), the Mage counters the Hunter (it out-damages the Hunter at medium range), and the Hunter counters the Warrior (it outranges and Traps the Warrior). Dashed arrows inside show help between allies: the Warrior's aura cuts the Mage's damage taken by 15 percent, the Mage's aura raises the Rogue's damage by 15 percent, the Hunter spots and slows enemies for the Rogue, and the Priest heals the Warrior.
Figure 3. The MARL-BGs evaluation pipeline. Each small square is one map or scenario, and each stage has its own. Scripted opponents can play at every stage. A left-to-right flow in four stages: Train, Select, Test, and Analyze. Train shows the 42 training maps, 0 to 41, as small squares, with maps 0 to 11 lighter as the curriculum; agents train with self-play, opponent pools, and scripted opponents. An arrow leads to Select: the 5 held-out validation maps, 42 to 46, used to pick the best checkpoint. The arrow splits into the two test modes: Mode 1, the 8 endgame scenarios on their own arenas, where Team A plays TDM-Alpha or TDM-Beta; and Mode 2, the tournament of mirrored 5v5 games on the 5 held-out test maps, 47 to 51, rated with Elo and 95 percent intervals. Both modes lead to Analyze: metrics, 16 or 11,192 per game, including who did what to whom, and replays that show every tick from any agent's view.
| Scenario | Teammate modeling | Opponent modeling | Basic damage target timing | Ultimate timing | Tactical obstacle exploitation | Tactical positioning | Balancing (def/off)ensive play | Strategic sacrifice | Plan adaptation | Multi-step team-level planning | Skills tested |
| 1 | 6 | ||||||||||
| 2 | 6 | ||||||||||
| 3 | 4 | ||||||||||
| 4 | 4 | ||||||||||
| 5 | 5 | ||||||||||
| 6 | 8 |
Table 2. The team skills each endgame scenario tests.
Figure 4. The 11,192 full metrics by topic. Bar length is the topic’s number of columns; the dark part records who did what to whom. † Only enemies or allies already under a harmful effect, such as a slow or a stun. A compact two-column list of the six metric sections and their 28 topics, each with its number of columns and a small bar. Damage and healing has 5,428 columns; kills, deaths, and respawns 2,108; status effects 1,532; results and actions 764; Ultimates 756; and teamwork and positioning 604. The largest topics are status applications (1,388), kills of enemies with a harmful effect (1,274), and damage to enemies with a harmful effect (1,204). Most of each large bar is dark: those columns record who did what to whom. The light part is the rest, mostly measures of one agent or one team.
Figure 5. Season 0 Elo ratings on test maps 47–51 with 95% intervals (filled: our baselines; open: scripted opponents). Dashed line: average Elo, 1,200; bands: tiers (Section 5.2 ). A horizontal dot plot of Elo ratings from about 950 to 1,480 for nine entrants, from top to bottom: recurrent IPPO 1,441, PQN-VDN 1,358, recurrent MAPPO 1,304, QMIX 1,300, feedforward IPPO 1,240, feedforward MAPPO 1,140, TDM-Beta 1,043, TDM-Gamma 1,000, and TDM-Alpha 974. Each point has a 95 percent interval about 10 to 18 points wide on each side. Shaded bands mark the tiers: Bronze below 1,000, Silver from 1,000, Gold from 1,200, and Platinum from 1,400. A dashed line at 1,200 marks the field's average. All six learned baselines sit above all three scripted opponents.
Figure 6. Six team measures against Elo for the nine Season 0 entrants. Points, tier bands, and the dashed line are as in Figure 5 ; every 95% interval hides behind its point. Each panel gives , the rank correlation with Elo, and its exact . Six small scatter plots in two rows of three, each with Elo from about 940 to 1,480 on the horizontal axis, shaded tier bands, a dashed line at 1,200, and one point per Season 0 entrant: filled points for the six learned baselines and open points for the three scripted opponents. Teammate distance falls as Elo rises (rank correlation -0.90). Warrior aura protection per 100 points of damage taken rises (+0.92). Damage to enemies a teammate Hunter had slowed rises (+0.88). The share of kills with two or more helpers rises (+0.85). The useful share of Basic healing rises (+0.90). The share of ticks with two or more attackers shows no trend (-0.02).
| Feature | ✓yes | partial | ✗no |
|---|---|---|---|
| Game | |||
| JAX / GPU-native simulator | The simulator itself runs batched on accelerators (JAX, PyTorch or CUDA). | Only part of the simulation runs on accelerators. | CPU engine. |
| Classes with distinct abilities | Agent types (classes, unit types, heroes, roles) have different sets of actions or abilities, so they play differently. | Types differ only in stats (health, speed, range, damage), or distinct abilities appear only in some tasks or maps. | All agents have the same actions and stats. |
| Healing, auras, crowd control, ultimates | Agents can use at least two of the four kinds, as abilities. | One kind. | None. Consumable items such as potions do not count. |
| Line of sight and obstacle cover | Obstacles (walls, terrain, bushes) block sight in the standard game: a unit behind an obstacle is hidden. | Obstacles block sight only in some tasks or maps, or they block movement or attacks but not sight. | No obstacle blocks sight, movement or attacks. A vision radius alone does not count, and the arena’s outer boundary is not an obstacle. |
| Body blocking | An agent’s body stops another agent from passing through it, including collision physics that bodies cannot pass through. | Blocking is off in some standard tasks, only some ground units or abilities block, or bodies can be pushed through or jumped over. | Agents pass through each other. |
Table S1. The rule for each mark in Table 1 of the paper, in the paper’s row order.
| Benchmark | What was graded |
|---|---|
| SMAX ( Rutherford et al., 2024 ) | The JaxMARL paper and the official JaxMARL repository, which ships SMAX. |
| Gigastep ( Lechner et al., 2023 ) | The benchmark’s NeurIPS paper and its supplementary material, and the official repository, including its third-party-learning folder, which ships the trained agent pool and the human-play script. |
| Lux S3 ( Tao et al., 2024 ) | The official repository (Lux-Design-S3) and GitHub organization, Kaggle’s environment package, which hosts the competition’s starter bot, and the NeurIPS 2024 competition page. One competitor’s public code was checked only to confirm that no official training code or agents exist. |
| VMAS ( Bettini et al., 2024 ) | The benchmark’s paper and official repository, and BenchMARL, the training library that the VMAS authors released as its companion. |
| MAgent2 ( Zheng et al., 2018 ) | Farama’s MAgent2, the original MAgent (geek-ai/MAgent), the MAgent paper, and its NeurIPS 2017 demo page, checked for competitions. |
| MPE ( Mordatch and Abbeel, 2018 ; Lowe et al., 2017 ) | The MADDPG paper, the original OpenAI release (openai/multiagent-particle-envs), its companion training code (openai/maddpg), and Farama’s MPE2 with its PettingZoo documentation. JaxMARL’s JAX version of MPE was checked only to confirm that it is a separate project. |
Table S2. What was graded for each compared benchmark. Every source link is in table1_evidence.csv .
Figure S1. One moment of a test game in the Oracle view (left) and in Team A’s Agent POV (right), where every enemy Team A cannot see vanishes.
| Mage | Warrior | Hunter | Rogue | Priest | |
| Maximum health | 80 | 200 | 100 | 100 | 100 |
| Body radius | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| Speed (per tick) | 1.0 | 1.0 | 1.0 | 1.3 | 1.0 |
| Sight radius | 6 | 6 | 6.5 | 6 | 6 |
| Basic target | enemy | enemy | enemy | enemy | ally |
| Basic range | 3 | 1.5 | 3.5 | 1.5 | 3 |
Table S3. Class values. Radii, ranges, sight, and speed are in map units; out-of-combat healing is a share of maximum health; “–” means none. An ally target includes the agent itself.
Figure S2. The maps of Team Deathmatch, drawn from the map files (maps 0–27). Each title gives the map’s number, its split, and its number of obstacles. Dark shapes are pillars and walls; blue and red dots are the two teams’ spawn pads. Every map is 20 by 10 map units.
Figure S3. The maps of Team Deathmatch, continued (maps 28–51).
| Group | Tests |
| Engine | 1,269 |
| Configuration and game-state validation | 412 |
| Combat and abilities | 271 |
| Reset and step, transition facts, and action mappings | 201 |
| Movement, collisions, and line of sight | 184 |
| Observations, action masks, and action history | 122 |
Table S4. Python tests by what they check, grouped by test file.
| ID | Challenge | Team A v B | Ticks | Score | Opp. | Sol. |
|---|---|---|---|---|---|---|
| 1 | Turn a doomed fight | HRP v MRP | 5 | 18–17 | TDM-Alpha | 3 |
| 2 | Trust the healer | WP v MR | 5 | 18–19 | TDM-Alpha | 4 |
| 3 | Moving cover | WH v R | 10 | 19–19 | TDM-Beta | 8 |
| 4 | Finish before the stun | WR v MWP | 5 | 18–19 | TDM-Alpha | 4 |
| 5 | Protect the healer | MWP v MRP | 5 | 17–19 | TDM-Beta | 5 |
| 6 | Keep the trap | MHP v MWR | 5 | 17–19 | TDM-Alpha | 5 |
Table S5. The eight endgame scenarios. Team A v B: the living agents at the start (M Mage, W Warrior, H Hunter, R Rogue, P Priest); each team’s other class slots start dead and return in the next respawn wave. Ticks: ticks left. Score: Team A against Team B; 20 points win. Opp.: the reactive controller. Sol.: ticks in the verified winning line.
| Team A | S1 | S2 | S3 | S4 | S5–S8 | Wins | Win % |
|---|---|---|---|---|---|---|---|
| Rec. IPPO | 7–160–89 | 0–0–256 | 0–0–256 | 0–3–253 | 0–0–256 | 7 | 0.34% |
| PQN-VDN | 0–256–0 | 0–0–256 | 0–0–256 | 0–0–256 | 0–0–256 | 0 | 0.00% |
| Rec. MAPPO | 3–200–53 | 0–0–256 | 0–0–256 | 0–2–254 | 0–0–256 | 3 | 0.15% |
| QMIX | 0–0–256 | 0–0–256 | 0–0–256 | 0–0–256 | 0–0–256 | 0 | 0.00% |
| FF IPPO | 8–148–100 | 1–3–252 | 2–1–253 | 0–21–235 | 0–0–256 | 11 | 0.54% |
| FF MAPPO | 1–208–47 | 0–0–256 | 1–3–252 | 0–1–255 | 0–0–256 | 2 | 0.10% |
Table S6. Scenario results with each system as Team A: wins–draws–losses in 256 games per scenario. Every system loses all games of Scenarios 5–8, which share a column. Wins and Win % cover all eight scenarios. Rec.: recurrent; FF: feedforward; Qwen3.5-9B: the LLM team (Section 7.6 of the paper).
| Rule | Essentials |
|---|---|
| 1. Who may enter | Any System that runs as pure JAX and exports to the portable file format: learned, scripted, search-based, hybrid, or built on a language model. The table shows each entry’s type and whether it uses an outside pretrained model. Season 0’s scripted opponents are reference opponents, exempt from Rules 2 and 3. |
| 2. Budget and machine | At most 255 million game steps and 8 hours, validation games included; the run stops at the first limit reached. One RTX 5090 (32 GB), a 16-core AMD Ryzen 9 9950X3D, 48 GB of RAM, and 50 GB of disk, with no network. |
| 3. What a run may use | Released MARL-BGs models are free to use as opponents or teammates; used as starting weights, as a frozen part of the entry, or as a teacher, their training steps count toward the budget. Outside pretrained models must have been public for at least 90 days, made by others, never trained on MARL-BGs, and licensed for release. Nothing else learned from MARL-BGs outside the run is allowed. Maps 0–46 are free to use; the run never touches test maps 47–51 or the scenarios, and nothing is chosen by results on them. |
| 4. Play limits | Playing itself with 512 games at once on one RTX 5090, an entry runs at least 5,000 game steps per second and uses at most 10 GB of GPU memory. |
| 5. Information | Teammates may message each other one tick late; within a tick, each actor decides alone. Memory starts fresh every game, and weights never change during evaluation. Each agent sees its own sight plus its living teammates’ sight, and no System reads hidden game data. |
| 6. Submitting | One submission per team per season, and one seat per team: a team’s new System replaces its member only if it rates higher. Everything, admitted or not, is open-sourced under Apache-2.0 and released into the pool of opponents. |
Table S7. The tournament rules, Version 1, numbered as in the rulebook.
| Tier | Elo | Win chance | Tier | Elo | Win chance |
|---|---|---|---|---|---|
| Grandmaster | 2,000+ | 99.0%+ | Gold | 1,200–1,399 | 50.0–76.0% |
| Master | 1,800–1,999 | 96.9–99.0% | Silver | 1,000–1,199 | 24.0–50.0% |
| Diamond | 1,600–1,799 | 90.9–96.9% | Bronze | 800–999 | 9.1–24.0% |
| Platinum | 1,400–1,599 | 76.0–90.9% | Iron | below 800 | below 9.1% |
Table S8. Leaderboard tiers. Each tier below Grandmaster splits into five divisions, 40 Elo apart, numbered 5 (lowest) to 1 (highest). Win chance: the share of decisive games, those not drawn, that a policy with this Elo wins against a policy of the field’s average Elo, 1,200.
| Term | Amount | Who receives it |
|---|---|---|
| Kill credit | per enemy death | Each agent whose attack landed on the enemy on the tick it died, and any Priest whose useful healing reached one of them on that tick |
| Red Zone kill | more | The same agents, when the enemy dies in its own Red Zone |
| Damage | per health point | The attacker; damage beyond the enemy’s remaining health does not pay |
| Useful healing | per health point | The Priest; healing beyond an ally’s maximum health does not pay |
| Teammate death | per death | Every agent (shared) |
| Red Zone death | more | Every agent (shared), when the teammate dies in its own Red Zone |
Table S9. The combat and support reward. Shared terms pay every Team A agent the same amount; the other terms pay only the agent named.
Figure S4. Training curves of each baseline’s chosen run at every routine validation check (Section 7.1 of the paper): point margin (left) and kills (right) per game. Dotted line: the end of the 40-million-step search.
Figure S5. The share of each routine validation check’s 96 games against the three scripted opponents that each baseline won, one run per baseline. Solid line: half; dotted line: the end of the 40-million-step search.
| Baseline | Value | Reward | Games | T | Learning rate | Checkpoint |
|---|---|---|---|---|---|---|
| Recurrent IPPO | Local critic per agent | Per-agent | 512 | 32 | , critic | 250,675,200 |
| PQN-VDN | VDN sum of Q-values | Team, | 768 | 128 | , decaying | 240,156,672 |
| Recurrent MAPPO | Centralized critic | Per-agent | 512 | 32 | , critic | 219,103,232 |
| QMIX | QMIX mixer | Team | 768 | 32 | 219,095,040 | |
| Feedforward IPPO | Local critic per agent | Per-agent | 512 | 32 | , critic | 219,103,232 |
| Feedforward MAPPO | Centralized critic | Team | 512 | 32 | , critic | 208,568,320 |
Table S10. Settings of each baseline’s Season 0 policy. Games: parallel games per update; T: rollout length in ticks. All six share their weights across the five agents and use the left spawn frame.
Figure S6. Simulator throughput on one RTX 5090: random legal actions on test map 47, compiled 500-tick rollouts, compilation excluded.
| Setting | Value |
|---|---|
| Model | Qwen3.5-9B: the original 16-bit weights at the revision pinned in our recipe. vLLM 0.30.0 converts them to 8-bit floating point (FP8) when it loads them. |
| Server | One server on its own RTX 5090 for each experiment, the two running at the same time, each using at most half its card’s memory; a context of 16,384 tokens; up to 16 requests at once; prefix caching off; seed 0. |
| Requests | One call per agent per tick, when the agent has more than one legal choice. Temperature 0, top-p 1, seed 0, thinking off, at most 64 reply tokens, and a strict JSON schema with two fields, move and combat , whose values must be listed action names. |
| View | Text version actor-text-v1 , in the frame where the agent’s own spawn is on the left. Only the agent’s own permitted view; no earlier turns. |
| Failures | A reply that is cut off, breaks the format, or picks a masked action becomes standing still with no attack, and is counted. |
| Games | Against the Big Nine: two games per member on each test map, spawn ends swapped, a 300-tick limit, seed 20261007. Scenarios: 256 games each, 128 side by side, seed 20261006. Full metrics in both. |
Table S11. Settings of the LLM team.
| Opponent | W–D–L | Margin | Kills | Deaths | Ticks | Legal | Seconds |
|---|---|---|---|---|---|---|---|
| Rec. IPPO | 0–0–10 | 0.1 | 11.2 | 87.9 | 99.98% | 87.5 | |
| PQN-VDN | 0–2–8 | 0.0 | 9.3 | 150.5 | 100.00% | 150.8 | |
| Rec. MAPPO | 0–0–10 | 0.0 | 11.2 | 111.4 | 100.00% | 109.8 | |
| QMIX | 0–2–8 | 0.4 | 9.4 | 148.9 | 100.00% | 140.5 | |
| FF IPPO | 0–2–8 | 0.0 | 9.5 | 151.7 | 99.93% | 143.6 | |
| FF MAPPO | 0–0–10 | 0.0 | 12.0 | 152.7 | 99.96% | 149.8 |
Table S12. Results of the LLM team (Qwen3.5-9B, Team A) against each member of the Big Nine: ten games each, two per test map. Margin, kills, deaths, and ticks are averages per game; Legal is the share of model replies that were legal actions; Seconds is each pair’s wall time divided by its two games, which played side by side.
| Basic uses | Damage | Together | Shared | Spread | Home deaths | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Opponent | LLM | Opp. | LLM | Opp. | LLM | Opp. | LLM | Opp. | LLM | Opp. | LLM | Opp. |
| Rec. IPPO | 32.1 | 215.3 | 157 | 1,510 | 1.4 | 43.1 | 0.0 | 9.5 | 6.9 | 2.2 | 9.6 | 0.0 |
| PQN-VDN | 31.5 | 256.5 | 163 | 1,317 | 1.1 | 43.8 | 0.0 | 7.9 | 7.6 | 2.4 | 7.1 | 0.0 |
| Rec. MAPPO | 40.5 | 259.8 | 149 | 1,706 | 1.3 | 51.6 | 0.0 | 8.9 | 7.5 | 2.5 | 9.3 | 0.0 |
| QMIX | 36.3 | 243.4 | 192 | 1,382 | 1.6 | 45.0 | 0.0 | 7.5 | 6.3 | 3.5 | 7.0 | 0.0 |
| FF IPPO | 39.2 | 267.5 | 141 | 1,395 | 1.6 | 42.7 | 0.0 | 7.9 | 7.7 | 2.5 | 8.5 | 0.0 |
Table S13. Teamwork of the LLM team (LLM) and its opponent (Opp.) in the same games, averaged per game: Basic uses; damage done; ticks in which two or more teammates dealt damage (Together); kills shared by two or more agents (Shared); mean distance between living teammates in units (Spread); and deaths inside the team’s own Red Zone, each worth 2 points to the enemy (Home deaths).
| Measure | Holm | Max- | |||
|---|---|---|---|---|---|
| Formation | Teammate distance | 0.27 | 0.12 | ||
| Time inside a Mage aura | 0.39 | 0.17 | |||
| Time inside a Warrior aura | 0.55 | 0.23 | |||
| Aura payoff | Damage Mage auras added* | 0.18 | 0.087 | ||
| Support | Useful share of Basic healing | 0.27 | 0.12 | ||
| Setups | Damage to Hunter-slowed enemies* | 0.39 | 0.17 |
Table S14. Team measures and their tests across the nine Season 0 entrants. *: per 100 ticks; the others are shares or averages. Holm and Max- : after Holm’s and the Westfall-Young max-statistic corrections (bold: below 0.05).
Figure S7. Point margin change per game when another method’s agent plays one class of recurrent IPPO, against recurrent IPPO alone, with 95% intervals. Within each class, strangers run from the highest-rated (top) down; the gray bar is their mean. Under each class name: the cost of a random agent. Dashed line: recurrent IPPO’s own agent.
Figure S8. Each of the 25 learned strangers: how much farther from its four teammates it stands than the agent it replaced, against its margin change, both measured from their class’s mean. : the mean of the five within-class rank correlations, with the exact of shuffling strangers within every class.
| Recurrent IPPO | PQN-VDN | Rec. MAPPO | QMIX | FF IPPO | FF MAPPO | TDM-Beta | TDM-Gamma | TDM-Alpha | |
|---|---|---|---|---|---|---|---|---|---|
| Rec. IPPO alone | |||||||||
| Rec. MAPPO | |||||||||
| QMIX | |||||||||
| FF IPPO | |||||||||
| FF MAPPO | |||||||||
| PQN-VDN |
Table S15. Point margin per game of recurrent IPPO alone against each Season 0 member, and the change when a stranger plays one class, averaged over its five classes (200 games per opponent and team).
| Measure | Expert Human | Rec. IPPO |
|---|---|---|
| Ultimates used the first tick ready | 25% (16/63) | 51% (34/67) |
| Ticks a ready Charge waited (median) | 3 | 0 |
| Damage of the opening Burst | 76 to 95 | 0 to 22 |
| Bursts with an enemy in reach | 14/15 | 0/16 |
| Mage damage in Burst vs. other ticks | 2.9 | 0.8 |
| Ultimates within 4 ticks of an ally’s (luck: 60%) | 89% (56/63) | 57% (38/67) |
Table S16. What the Expert Human, controlling all five agents, did that recurrent IPPO did not, in their three games on maps 47–49. Counts are summed over the games, and every row points the same way in each game. Ready and in reach mean the game allowed the action on that tick. Last two rows: brackets give the share under a random pick.
More in this topic
RL Benchmarks
Reinforcement learning benchmarks provide tasks and evaluation protocols for comparing agents across control, games, robotics, and other decision-making settings.
See all 85 papers