Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?

Yibo Li; Jinhang Qiu; Zhi Zheng; Qianyun Guo; Jiaying Wu; Shuo Ji; Bryan Hooi

cs.AIOct 6, 2026

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?

Authors: Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi

Organizations: National University of Singapore

Abstract

Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments. Evaluating this ability is therefore important for understanding how effectively agents acquire and use new knowledge. Existing benchmarks have sought to evaluate this ability, but they primarily evaluate tasks whose rules are provided in the instructions or already familiar to pretrained models, making it difficult to distinguish learning from interactions from reasoning with existing knowledge. To address this, we introduce Learn2Play Bench, a benchmark of newly designed text-based games, whose rules are novel or counterintuitive, requiring agents to acquire knowledge through interaction rather than rely solely on pretrained knowledge. These games provide reproducible feedback and automatic scoring, enabling controlled evaluation of learning across repeated attempts. We also vary game instances to test whether agents can apply what they have learned to new situations. Therefore, we evaluate how backbone models, self-evolving methods, and agent harnesses affect agents' learning ability, revealing three findings: (1) Experience retention: Retaining complete records of actions and feedback can support more effective learning than summarizing these experiences into rules or strategies. (2) Human agent gap: Top-performing human players achieve higher peak scores than the evaluated agents. Human explore more varied strategies, and repeat actions less. (3) Harness matters: With the backbone fixed, changing the harness can improve performance while reducing estimated inference cost. Together, these findings provide insights into how LLM agents learn from experience and suggest directions for future work to improve their learning ability. Project website: https://liushiliushi.github.io/learn2play-bench-website/

Figures & tables

Figure 1: Learn2Play Bench overview. The left panel shows 20 manually designed game templates, each of which generates multiple instances using different random seeds. The highlighted Poisoner template illustrates the interaction protocol on the right: the environment describes the state and enumerates valid actions, the player selects one action, and the environment returns the resulting observation. The episode ends with an automatically computed score (Appendix D.16 ).

Claude Opus 5
Harness	Max	Mean	LG	LS
OpenCode	74.5	53.8	+21.3	+5.13
Claude Code	81.8	62.3	+27.1	+6.59
GPT-5.6-SOL
OpenCode	57.0	41.1	+14.5	+3.71
Codex	74.3	54.8	+22.2	+5.66

Table 2: Same-backbone harness comparison.

OpenCode backbones
Agent	Control	Re-shuf.	LS drop
Claude Opus 5	4.10	3.51	0.59
Qwen 3.8 Max	3.16	3.57	-0.41
Gemini 3.5 Flash	4.50	2.76	1.73
Kimi K3	4.19	3.31	0.88
Gemini 3.7 Flash	4.49	2.34	2.15

Table 3: Learning slopes when the same game instance repeats (Control) or changes across episodes (Re-shuf.), with the underlying rules held fixed. LS drop is Control minus Re-shuf.; positive values indicate slower learning after re-shuffling.

Figure 3: Max score versus mean cost per episode. The dashed line shows the performance–cost frontier for configurations with standard-price estimates.

Appendix figures & tables67 assets

Supplementary material from the paper’s appendix.

Appendix

Game	What the player needs to learn
Poisoner	Infer taste constraints from accepted and rejected dishes and apply them to new menus.
SynergyCorp	Learn which character traits predict preferred, weak, and taboo actions.
Haunted Inn	Discover guest–room, adjacency, and count taboos through controlled placements.
Roommates	Infer pairwise and nonlinear grouping rules from room-level feedback.
Roadside Observatory	Learn social protocols and resource requirements for new encounters.
Lost and Found	Learn how to verify ownership without revealing private information.

Appendix

Table 4: Re-shuffled games: knowledge that must carry over to new instances.

Game	What the player needs to learn
PrimordialSoup	Discover reaction recipes and plan synthesis within a resource budget.
GemForge	Discover multi-tier recipes and manage limiting ingredients.
Catnip	Learn action–target matches and when to exit for the best reward.
Dungeon	Remember rooms, keys, hazards, and treasures to refine an escape route.
DreamGarden	Infer delayed ecological effects and plan actions across turns.
Ecosphere	Infer food-web, competition, and waste dynamics to plan stocking.

Appendix

Table 5: Fixed games: knowledge accumulated about a recurring instance.

Game	Seed	$C$	Source	Normalized score
PrimordialSoup	1	110	exact	$100x_{t}/110$
GemForge	1	791	exact	$100x_{t}/791$
Catnip	1	320	estimated	$100x_{t}/320$
Dungeon	2	54	exact	$100x_{t}/54$
SynergyCorp	2	137	exact	$100x_{t}/137$
DreamGarden	1	6300	estimated	$100x_{t}/6300$

Appendix

Table 6: Per-game reference scores used for normalization and their supporting evidence.

Figure 4: Learning curves for Memory with four backbone models. Scores use the shared normalization and game weights. Relative progress aligns the five- and ten-episode scoring windows.

System	Model	Max	Mean	LG	LS
Human Top-1	Human	84.3	53.5	+21.6	+7.33
Claude Code	Claude Opus 5	81.8	62.3	+27.1	+6.59
Human Top-3	Human	79.1	50.1	+18.9	+5.54
Human Top-5	Human	75.1	47.1	+17.1	+4.62
OpenCode	Claude Opus 5	74.5	53.8	+21.3	+5.13
Codex	GPT-5.6-SOL	74.3	54.8	+22.2	+5.66

Appendix

Table 8: Human and agent results ranked by Max. All four metrics use the same 20 games and challenge weights. Human selection and score construction are specified above. Bold marks the best value in each column.

Method	Control	Re-shuf.	LS drop
Naive	-0.49	-0.41	-0.08
Memory	3.76	2.88	0.88
Reflexion	3.33	2.70	0.63
ReasoningBank	2.48	2.09	0.40
AWM	-0.11	0.26	-0.37
EvoTest	1.92	1.07	0.85

Appendix

Table 9: Learning slopes for six methods with Opus 5 when the same instance repeats (Control) or changes across episodes (Re-shuf.), with the rules held fixed. LS drop is Control minus Re-shuf.

System / model	Control LS	Control LS incl. extra
OpenCode Claude Opus 5	4.10	4.07
OpenCode Qwen 3.8 Max	3.16	3.34
OpenCode Kimi K3	4.19	4.33

Appendix

Table 10: Learning slopes in the same-instance control, excluding or including extended-budget trials. Both estimates use the same games. The extended-budget results are supplementary and do not replace the standard-budget estimates in Table 4.5 .

Figure 5: PrimordialSoup: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	1	100.0	100.0	–	–
OpenCode + GPT-6 Astra	3	$100.0\pm 0.0$	$85.8\pm 2.2$	$+35.5\pm 5.5$	$+10.36\pm 1.14$
Codex + GPT-5.6-SOL	4	$89.1\pm 18.9$	$66.8\pm 15.2$	$+16.8\pm 33.3$	$+5.82\pm 10.89$
Claude Code + Opus 5	3.7	$81.8\pm 15.7$	$73.2\pm 10.0$	$+18.2\pm 17.3$	$+6.03\pm 5.26$
OpenCode + Kimi K3	4.7	$78.5\pm 19.1$	$64.7\pm 12.0$	$+8.0\pm 16.2$	$+2.55\pm 6.27$
OpenCode + Qwen 3.8 Max	4	$77.0\pm 16.0$	$59.9\pm 10.8$	$+28.2\pm 13.9$	$+9.12\pm 4.15$

Appendix

Table 16: PrimordialSoup: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Both base reactions: $C+E\to\mathrm{L1a}$ and $D+E\to\mathrm{L1b}$ .
T2	A and F are inert, and one react action fires each feasible base recipe at most once.
T3	React consumes B and sets the catalyst level to $\min(B,3)$ .
T4	Catalyst persists through synthesis but is overwritten by the next react action.
T5	Catalyst level at least one enables $\mathrm{L1a}+\mathrm{L1b}\to\mathrm{L2b}$ .
T6	Free $E+\mathrm{L1a}\to\mathrm{L2a}$ does not consume catalyst and takes priority over L2b.

Appendix

Table 17: PrimordialSoup: criteria for assessing rule discovery.

Figure 6: GemForge: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	4	99.9	59.4	–	–
Claude Code + Opus 5	3.7	$93.9\pm 9.3$	$39.4\pm 12.7$	$+35.9\pm 28.1$	$+12.32\pm 8.33$
EvoTest + Kimi K3	3	$81.8\pm 30.8$	$46.9\pm 11.8$	$+46.0\pm 37.0$	$+16.09\pm 8.99$
OpenCode + Gemini 3.5 Flash	3	$72.6\pm 45.4$	$41.8\pm 32.8$	$+22.3\pm 32.2$	$+9.25\pm 11.39$
OpenCode + Qwen 3.8 Max	4	$63.8\pm 31.7$	$34.9\pm 18.7$	$+12.7\pm 21.7$	$+5.44\pm 3.85$
Codex + GPT-5.6-SOL	4.3	$61.8\pm 36.4$	$29.1\pm 18.7$	$+11.0\pm 21.1$	$+4.75\pm 7.40$

Appendix

Table 19: GemForge: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Complete five-recipe map from raw materials to tier-1 gems.
T2	Complete four-recipe map from tier-1 to tier-2 gems.
T3	Complete two-recipe map from tier-2 to tier-3 gems.
T4	Every nonrecipe pair consumes its inputs and yields slag. No three-input recipe exists.
T5	Final inventory determines score. Solo firing yields one-point slag. Ash and crystal do not advance to higher tiers.

Appendix

Table 20: GemForge: criteria for assessing rule discovery.

Figure 7: Catnip: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Claude Code + Opus 5	9	$73.0\pm 1.2$	$50.4\pm 2.3$	$+31.8\pm 13.2$	$+4.64\pm 1.41$
Human Top-1	8	70.3	53.5	–	–
OpenCode + GPT-6 Astra	6.3	$62.7\pm 33.4$	$45.6\pm 22.0$	$+23.1\pm 18.8$	$+3.28\pm 2.64$
EvoTest + Kimi K3	3	$59.8\pm 5.5$	$44.1\pm 11.6$	$-5.3\pm 5.1$	$-0.09\pm 0.71$
Codex + GPT-5.6-SOL	4.3	$59.5\pm 5.9$	$49.0\pm 6.7$	$+21.1\pm 10.4$	$+3.23\pm 1.51$
Reflexion + Kimi K3	7.3	$56.6\pm 5.4$	$37.8\pm 1.7$	$+10.8\pm 16.7$	$+1.58\pm 2.15$

Appendix

Table 22: Catnip: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	The six targets occur in a fixed order.
T2	Complete mapping from all six targets to their sensory preferences.
T3	Complete mapping of soothing and chaotic style preferences.
T4	The five influence actions have distinct modalities and styles, and matching them changes influence gain.
T5	Conversion uses target-specific LOW, MED, and HIGH reward thresholds.
T6	Complete breed routing: Ragdoll to cuteness, Bobcat to menace, Black cat to stealth, and Orange to energy.

Appendix

Table 23: Catnip: criteria for assessing rule discovery.

Figure 8: Dungeon: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Claude Code + Opus 5	2.3	$100.0\pm 0.0$	$94.1\pm 6.9$	$+14.8\pm 17.3$	$+5.86\pm 6.84$
Human Top-1	5	100.0	67.8	–	–
OpenCode + Opus 5	4.7	$98.8\pm 2.1$	$83.7\pm 13.7$	$+2.8\pm 27.5$	$+3.52\pm 5.56$
OpenCode + Opus 5.5	3	$96.9\pm 5.3$	$84.3\pm 13.7$	$+23.8\pm 16.8$	$+9.07\pm 7.14$
OpenCode + Gemini 3.5 Flash	4.7	$92.6\pm 9.6$	$68.6\pm 4.1$	$+42.6\pm 19.4$	$+13.76\pm 5.40$
EvoTest + Kimi K3	3	$92.0\pm 12.3$	$56.4\pm 2.8$	$-2.5\pm 19.0$	$+0.19\pm 5.47$

Appendix

Table 25: Dungeon: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Complete 20-room graph, including every visible edge and every directed hidden passage.
T2	Room 14 is the exit and grants a ten-point escape bonus.
T3	Room 16 contains the 25-gold treasure.
T4	The key is in room 17 and is consumed when room 16 is opened.
T5	Complete location map for the charm, oil, boots, and potion.
T6	Complete effect map for those four items.

Appendix

Table 26: Dungeon: criteria for assessing rule discovery.

Figure 9: SynergyCorp: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
OpenCode + Kimi K3	5	$91.7\pm 0.4$	$56.1\pm 8.4$	$+47.1\pm 7.1$	$+16.47\pm 1.46$
Human Top-1	5	91.2	44.5	–	–
Claude Code + Opus 5	5	$90.3\pm 2.9$	$63.8\pm 4.8$	$+39.3\pm 17.9$	$+14.38\pm 4.82$
OpenCode + Opus 5.5	5	$90.3\pm 2.9$	$61.9\pm 1.1$	$+41.5\pm 5.9$	$+15.04\pm 1.72$
OpenCode + Opus 5	5	$86.1\pm 7.6$	$59.1\pm 1.0$	$+36.3\pm 9.4$	$+13.55\pm 2.96$
Codex + GPT-5.6-SOL	5	$85.9\pm 6.6$	$63.7\pm 4.2$	$+37.8\pm 5.9$	$+13.87\pm 0.91$

Appendix

Table 28: SynergyCorp: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Complete partition of the eight actions into four behavior pairs. Designer-assigned modality names are not required.
T2	Complete mapping of eight informative desk traits and identification of four decoys.
T3	Complete preference ordering over the four behavior groups for each boss type.
T4	The overtime-selfie and buzzwords taboo clears approval and adds eight demerits.
T5	Repeating one action produces two-stage decay, motivating alternation within a pair.
T6	Steal-lunch followed by buzzwords applies the ordered combo bonus.

Appendix

Table 29: SynergyCorp: criteria for assessing rule discovery.

Figure 10: DreamGarden: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Claude Code + Opus 5	4.7	$85.2\pm 0.0$	$82.7\pm 1.5$	$+7.7\pm 4.0$	$+1.19\pm 0.55$
OpenCode + Opus 5	7.7	$84.3\pm 0.8$	$80.6\pm 2.4$	$+9.3\pm 2.7$	$+1.40\pm 0.35$
Human Top-1	8	84.2	78.7	–	–
Reflexion + Opus 5	7.3	$82.8\pm 2.5$	$74.4\pm 1.0$	$+6.4\pm 1.7$	$+0.97\pm 0.31$
OpenCode + Kimi K3	8.7	$82.6\pm 1.3$	$75.0\pm 3.6$	$+16.3\pm 6.8$	$+2.22\pm 0.92$
OpenCode + Qwen 3.8 Max	5.7	$81.8\pm 2.5$	$77.2\pm 1.4$	$+9.8\pm 4.6$	$+1.46\pm 0.51$

Appendix

Table 31: DreamGarden: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Resonance per turn is $\sum v_{j}\min(pop_{j},35)$ . Population is capped at 100 and decays by 20%. For seed 1, Spores and Moths are worth $+5$ , Nightmares $-3$ , and Jelly and Weavers 0. Nightmares suppress the Weaver chain.
T2	Eclipse increases Moths but also introduces Nightmares. Throb increases Weavers, Whisper suppresses Nightmares, and Tide feeds only Nightmares.
T3	Throb starts the chain Weavers $\to$ Jelly $\to$ Spores. Each ecological link takes one turn, so the final benefit is delayed.
T4	Over 18 turns, combine direct Moth growth, delayed Spore growth, and Nightmare removal. Throb has little immediate value but a large payoff two turns later.
T5	The archived 5,367-point sequence starts with Throb, then interleaves Eclipse and Whisper to keep Moths and Spores near saturation and Nightmares near zero. It never uses Tide.

Appendix

Table 32: DreamGarden: criteria for assessing rule discovery.

Figure 11: Ecosphere: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	5	85.0	59.7	–	–
Claude Code + Opus 5	2.3	$80.1\pm 19.6$	$35.4\pm 18.2$	$-2.3\pm 23.0$	$-2.56\pm 9.55$
OpenCode + Opus 5.5	2.7	$79.9\pm 5.7$	$57.8\pm 5.4$	$+10.7\pm 3.7$	$+2.00\pm 1.81$
OpenCode + Gemini 3.5 Flash	4	$77.1\pm 15.3$	$55.8\pm 12.2$	$+12.4\pm 21.4$	$+6.16\pm 4.96$
Reflexion + Opus 5	4	$76.3\pm 21.7$	$46.5\pm 13.2$	$-10.0\pm 15.6$	$-3.97\pm 7.40$
Naive + Opus 5	3	$75.1\pm 6.6$	$56.7\pm 8.1$	$-4.2\pm 21.8$	$-2.35\pm 8.88$

Appendix

Table 34: Ecosphere: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Spend a one-time budget of 150 on stocking, using at most 40 setup actions. Sealing the habitat starts a deterministic 30-day simulation.
T2	Moss, algae, and microbes support the ecosystem but score zero. Herbivores score 4 per animal, subject to saturation. Minnows and newts score 20.
T3	Seed-1 producer layer: moss $24\times$ and algae $18\times$ biomass per unit, each capped at 10 stocked units.
T4	Seed-1 food web and competition: snail and shrimp eat moss, daphnia eats algae, rank order gives shrimp priority over snail, and overgrazed producers collapse.
T5	Predator and keystone dynamics: minnow hunts shrimp only, newt hunts shrimp/snail, prey below refuge 5 cannot be caught, and a single newt culls shrimp enough for snails to survive.
T6	Living animals and corpses create waste that produces toxic gas. Enough decomposers must be stocked to clear it.

Appendix

Table 35: Ecosphere: criteria for assessing rule discovery.

Figure 12: Grapevine: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	5	93.8	52.5	–	–
OpenCode + Opus 5.5	4.3	$89.6\pm 3.6$	$56.7\pm 6.2$	$+32.3\pm 15.4$	$+9.17\pm 6.17$
Claude Code + Opus 5	4.3	$75.0\pm 10.8$	$50.8\pm 8.8$	$+39.6\pm 22.2$	$+13.13\pm 7.98$
OpenCode + GPT-6 Astra	2	$72.9\pm 3.6$	$47.1\pm 1.4$	$-11.5\pm 15.4$	$-5.00\pm 4.96$
Memory + Opus 5	3.3	$70.8\pm 14.4$	$53.8\pm 1.2$	$+17.7\pm 3.6$	$+7.08\pm 1.44$
OpenCode + Gemini 3.7 Flash	4	$68.8\pm 0.0$	$46.2\pm 0.0$	$+6.2\pm 0.0$	$+0.62\pm 0.00$

Appendix

Table 37: Grapevine: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Choose at most three seeds in a deterministic 20-node linear-threshold cascade. Each episode allows one release. Feedback after each wave provides evidence about activation thresholds and incoming connections.
T2	Seed-1 synergy trap: best singles are T/D at 5, the greedy high-single trio reaches only about 8, while the exact optimum reaches 16.
T3	Before release, add or remove seeds to compare candidate seed sets across episodes and identify which nodes contribute additional activation.
T4	Deterministic exact-trio search over the discovered activation frontier, not choosing T/D/Q/L/M/B by single-node strength.
T5	16-reach triples: EFT activates KQS, then CMR, GH, JO, I, D, L and leaves A/B/N/P. EPT activates KQS, then CM, GHR, JO, I, D, L and leaves A/B/F/N.

Appendix

Table 38: Grapevine: criteria for assessing rule discovery.

Figure 13: Haunted Inn: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	1	100.0	50.0	–	–
EvoTest + Kimi K3	1.3	$100.0\pm 0.0$	$80.0\pm 20.0$	$-16.7\pm 28.9$	$-3.33\pm 5.77$
Naive + Kimi K3	3	$100.0\pm 0.0$	$35.3\pm 13.6$	$+50.0\pm 50.0$	$+13.33\pm 15.28$
OpenCode + GPT-6 Astra	3	$91.7\pm 7.2$	$43.3\pm 1.4$	$+50.0\pm 54.1$	$+14.17\pm 16.60$
Naive + GPT-5 Mini	3.7	$83.3\pm 14.4$	$37.5\pm 10.9$	$+21.7\pm 15.9$	$+5.83\pm 5.77$
Codex + GPT-5.6-SOL	4.7	$83.3\pm 14.4$	$41.2\pm 19.6$	$+49.2\pm 18.1$	$+14.17\pm 7.78$

Appendix

Table 40: Haunted Inn: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Eight rooms with visible fixed features. For the evaluated seed 2, rooms 1, 3, 6, and 7 are plain. Room 2 has a shrine, room 4 a window, room 5 is dark, and room 8 has a mirror.
T2	All three hidden taboos for the evaluated seed 2: housing at least three actors haunts every actor room. Adjacent actors and scholars haunt both rooms. Any lantern-bearing guest in the mirror room is haunted.
T3	Clues identify rooms that trigger taboos. Panic then empties the entire connected block of occupied rooms. An empty room stops that spread.
T4	Place a few guests at a time to distinguish the room that triggered a taboo from rooms affected by spreading panic. Use these comparisons to separate adjacency rules from guest-count rules.
T5	Fee-preserving packing under constraints: keep high-fee merchants/actors when legal, turn away or separate conflicts, and use empty gaps only when they save more fee than they cost.
T6	200-point seating policy on favorable queues: avoid the three taboo triggers, insert firebreaks where needed, and do not sacrifice high-fee guests for untested superstition.

Appendix

Table 41: Haunted Inn: criteria for assessing rule discovery.

Figure 14: Castaway: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	4	100.0	54.8	–	–
OpenCode + GPT-6 Astra	4.3	$100.0\pm 0.0$	$57.8\pm 6.9$	$+59.5\pm 12.2$	$+20.40\pm 2.33$
Claude Code + Opus 5	2.7	$99.4\pm 1.0$	$70.6\pm 3.5$	$+61.3\pm 19.2$	$+20.53\pm 3.50$
OpenCode + Kimi K3	4	$94.3\pm 1.9$	$67.7\pm 4.4$	$+43.8\pm 13.6$	$+15.77\pm 4.48$
Codex + GPT-5.6-SOL	3.7	$94.2\pm 3.5$	$50.2\pm 7.5$	$+44.5\pm 32.8$	$+14.92\pm 8.25$
OpenCode + Opus 5.5	3.7	$92.8\pm 7.4$	$46.1\pm 14.6$	$+43.1\pm 39.6$	$+11.45\pm 10.81$

Appendix

Table 43: Castaway: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Seven-place seed-1 island map from beach to forest, swamp, cave, ravine, bamboo, and ruins.
T2	Resource/recipe table: wood/flint/vine/water, seed-1 food effects, beach fire, torch, tool, and raft, including mandatory tool and two-vine raft lashing.
T3	Survival and hazard model: 56 segments, five segments per day, hunger drain, day survival points, cave current, swamp mosquitoes/collapse, bamboo falling rocks/night trap, ruins crab, ravine boar, and forest monkey.
T4	True-vs-decoy escape chain: frame at cave, sail at forest, rudder at ruins are real, while compass at bamboo is the decoy that makes raft crafting fail if used as a substitute.
T5	241-point script: build fire/tool/torch, route around the two lethal traps, neutralize guards, collect all real raft parts while optionally collecting the decoy for information/score, safely stall for day points, and craft the raft at the beach.

Appendix

Table 44: Castaway: criteria for assessing rule discovery.

Figure 15: Roadside Observatory: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
OpenCode + Opus 5.5	4.3	$96.8\pm 5.6$	$87.0\pm 8.3$	$+28.8\pm 9.0$	$+4.17\pm 1.05$
Reflexion + Opus 5	6	$94.5\pm 5.4$	$74.0\pm 10.3$	$+33.1\pm 8.4$	$+4.73\pm 1.46$
OpenCode + Gemini 3.5 Flash	5.3	$90.1\pm 3.0$	$74.5\pm 15.8$	$+31.8\pm 28.5$	$+4.67\pm 4.19$
Human Top-1	7	89.6	47.9	–	–
OpenCode + GPT-6 Astra	2.3	$85.7\pm 24.7$	$79.3\pm 33.8$	$+8.7\pm 8.4$	$+1.41\pm 1.34$
Claude Code + Opus 5	8.3	$82.7\pm 25.7$	$69.9\pm 18.2$	$+36.8\pm 22.6$	$+5.06\pm 2.99$

Appendix

Table 46: Roadside Observatory: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Travel-state accounting for distance, time, energy, supplies, cash, rest, bus, safe hitching, and main-road walking.
T2	Offer to help the mechanic before asking for a ride. Asking without first helping wastes the encounter.
T3	Keep energy at least 50. Ask the ranger about weather and the trail before asking for the shortcut.
T4	Hiker protocol: pay supply/cash or forage, then listen for the route clue.
T5	Collector protocol: visit landmark or keep token, then tell story/show token to receive the coded map clue.
T6	Driver protocol and trap avoidance: refuse offer, question detail, then accept only the verified safe ride.

Appendix

Table 47: Roadside Observatory: criteria for assessing rule discovery.

Figure 16: Lost and Found: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
AWM + Opus 5	3.3	$100.0\pm 0.0$	$90.9\pm 5.7$	$-2.7\pm 20.6$	$+0.13\pm 4.48$
OpenCode + GPT-6 Astra	1.7	$100.0\pm 0.0$	$98.4\pm 2.0$	$+3.9\pm 5.1$	$+1.03\pm 1.43$
Memory + Opus 5	4	$96.8\pm 5.6$	$87.7\pm 10.8$	$+6.0\pm 5.5$	$+1.92\pm 1.34$
OpenCode + Opus 5	5	$95.8\pm 4.2$	$87.4\pm 6.7$	$-3.7\pm 14.7$	$-0.13\pm 3.30$
Memory + Kimi K3	3.7	$95.1\pm 0.2$	$81.0\pm 15.0$	$+17.8\pm 13.6$	$+8.70\pm 6.94$
Naive + Opus 5	2	$94.5\pm 2.3$	$90.3\pm 2.1$	$-0.6\pm 2.9$	$-0.22\pm 0.84$

Appendix

Table 49: Lost and Found: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Three reshuffled cases per episode with four claimant roles: owner, finder, associate, and mimic.
T2	Inspect the item privately to learn its concealed marker. Opening it publicly or asking questions too early reveals details that later claimants can repeat.
T3	Verify the concealed detail, the claimant’s relationship to the item and its purpose, the loss timeline, and the relevant external record. A matching decoy record is not sufficient.
T4	Resource budgeting: 24 turns, eight privacy tokens, three isolation tokens, and three record checks across all cases.
T5	1,000-point office protocol: inspect each item privately, verify owner on all independent channels, check the matching record, and return or store without leaking decisive evidence.

Appendix

Table 50: Lost and Found: criteria for assessing rule discovery.

Figure 17: Mola Tea: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Reflexion + Opus 5	10	$79.0\pm 13.6$	$63.8\pm 5.8$	$+18.0\pm 13.9$	$+2.96\pm 1.80$
AWM + Kimi K3	8.7	$74.8\pm 18.1$	$46.1\pm 14.5$	$+5.6\pm 3.3$	$+0.98\pm 0.83$
Human Top-1	7	68.4	40.7	–	–
ReasoningBank + Opus 5	8.7	$66.1\pm 4.8$	$51.7\pm 4.1$	$+5.6\pm 8.1$	$+1.36\pm 1.01$
OpenCode + GPT-6 Astra	6.3	$66.1\pm 2.9$	$57.2\pm 2.7$	$+0.4\pm 2.4$	$+0.51\pm 0.26$
EvoTest + Opus 5	3	$65.8\pm 3.6$	$52.9\pm 4.5$	$+1.6\pm 6.3$	$+0.92\pm 0.84$

Appendix

Table 52: Mola Tea: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	21-turn, seven-day shop economy with cash, unit cost, launch stock, stockouts, expired inventory, and regular-customer churn.
T2	Recipe rule: only products at distance 1 from the classic drink are viable, while far variants can sell briefly but fail the adoption chain.
T3	Customer-role inference from clues: Explorer, Echo, Verifier, Broadcaster, and Anchor/regulars have distinct responses to sample, feature, discount, gift, and bundle.
T4	Stage transition chain LAUNCHED $\to$ SEEDED $\to$ REPEATED $\to$ VERIFIED $\to$ TRENDING, with clean intervals and short verification/diffusion deadlines.
T5	Sampling Explorers starts adoption. Feature promotion works after a natural repeat purchase. A Broadcaster gift works only after verification. Bundles pay off only after the product is trending. Repeated discounts teach customers to expect low prices.
T6	Inventory/novelty control: restock before return and bundle windows, use hold/restore-classic to reduce novelty pressure, and avoid leftover-expiry costs.

Appendix

Table 53: Mola Tea: criteria for assessing rule discovery.

Figure 18: Plague: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	2	100.0	58.0	–	–
Reflexion + GPT-5 Mini	3	$100.0\pm 0.0$	$65.0\pm 12.0$	$+20.0\pm 43.4$	$+6.50\pm 11.76$
OpenCode + Opus 5.5	1	$100.0\pm 0.0$	$97.3\pm 3.1$	$-1.7\pm 2.9$	$-0.33\pm 0.58$
Codex + GPT-5.6-SOL	4.7	$90.0\pm 17.3$	$58.7\pm 22.2$	$+34.2\pm 11.8$	$+11.83\pm 2.47$
Claude Code + Opus 5	4	$76.7\pm 10.4$	$55.7\pm 15.7$	$+29.2\pm 16.6$	$+9.00\pm 5.29$
OpenCode + Gemini 3.5 Flash	2.3	$75.0\pm 35.0$	$57.7\pm 40.5$	$+25.0\pm 39.1$	$+7.00\pm 11.72$

Appendix

Table 55: Plague: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Line and action economy: 20 people, 14 nights, four AP per day, three quarantine cells, severity 1–3, and death after severity $>3$ .
T2	Hidden-carrier dynamics: two asymptomatic carriers are the only spreaders, reshuffled into left/right halves, with radius $1+\lfloor age/2\rfloor$ , while symptomatic non-carriers do not infect anyone.
T3	Intervention semantics: quarantine blocks both catching and spreading, whereas treatment only reduces current severity and gives one-night immunity after cure.
T4	Carrier diagnosis from spatial evidence, especially separated sick pairs around a healthy center and disappearance of new cases after quarantining that center.
T5	AP triage: quarantine likely carriers early, treat only patients near death, and release/replace false quarantines without exceeding three cells.
T6	For a 100-point containment policy, identify both carriers by day 2–3 and quarantine them before their spread radius grows. Time treatments so that no other villager reaches the death threshold.

Appendix

Table 56: Plague: criteria for assessing rule discovery.

Figure 19: Patch Reality: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
EvoTest + Opus 5	3.7	$89.7\pm 0.4$	$86.9\pm 2.6$	$+2.3\pm 3.6$	$+1.30\pm 1.19$
ReasoningBank + Opus 5	4.3	$89.7\pm 0.4$	$85.2\pm 2.7$	$+6.1\pm 0.2$	$+2.44\pm 0.10$
Claude Code + Opus 5	2.7	$89.6\pm 0.0$	$89.2\pm 0.1$	$+0.1\pm 0.5$	$+0.06\pm 0.11$
OpenCode + Opus 5	2.7	$89.6\pm 0.0$	$89.2\pm 0.1$	$+0.6\pm 0.3$	$+0.17\pm 0.05$
Human Top-1	5	89.6	59.2	–	–
OpenCode + Opus 5.5	4	$89.6\pm 0.0$	$85.6\pm 3.3$	$-0.7\pm 10.3$	$-1.10\pm 3.81$

Appendix

Table 58: Patch Reality: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Per-level action grammar: observe evidence, optionally audit risky records, diagnose source/type/strategy, then patch the root object.
T2	Levels 1–3 mappings: coin/drain uses environment-cover pavement friction, vibrating cup requires auditing residual patch then expiring it, and flickering key requires object physics repair by solidifying the key.
T3	Levels 4–5 mappings: hospital ledger is an identity-record trap solved by binding the wristband, while premature echo is a time-order conflict solved by aligning the acoustic log.
T4	Level 6 ordered record repair: reconcile the ledger before halting archive reindexing.
T5	Level 7 public-record exposure solved by camera desync.
T6	Level 8 identity-anchor pollution solved by reclaiming the name anchor.

Appendix

Table 59: Patch Reality: criteria for assessing rule discovery.

Figure 20: Poisoner: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	3	100.0	66.8	–	–
OpenCode + Gemini 3.7 Flash	3.3	$100.0\pm 0.0$	$62.2\pm 25.3$	$+61.0\pm 9.5$	$+22.20\pm 1.91$
OpenCode + DeepSeek V4 Pro	3.7	$100.0\pm 0.0$	$71.2\pm 3.8$	$+55.5\pm 19.1$	$+20.00\pm 5.72$
Claude Code + Opus 5	3.7	$100.0\pm 0.0$	$68.9\pm 19.3$	$+55.7\pm 9.8$	$+18.93\pm 3.87$
OpenCode + GPT-5.6-SOL	4.3	$100.0\pm 0.0$	$55.5\pm 21.4$	$+66.8\pm 16.8$	$+22.27\pm 5.10$
OpenCode + Opus 5.5	2.3	$100.0\pm 0.0$	$73.4\pm 24.0$	$+44.5\pm 25.7$	$+13.37\pm 8.88$

Appendix

Table 61: Poisoner: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Three-banquet inference interface: eight dishes per banquet, one serve action per banquet, score $0/33/67/100$ , and only one labeled accept/reject outcome per menu.
T2	The exact five-feature conjunction for the evaluated seed 2: not sweet, spicy, meaty, fragrant, and soft are required. Sourness, saltiness, and temperature are irrelevant.
T3	Wildcard and near-miss handling: each menu contains at least one full match and at least one 4-of-5 bait, so four-feature theories fail.
T4	For each new menu, check the learned combination of required features. Ignore irrelevant traits and distinguish a full match from dishes satisfying only four requirements.
T5	3/3 regicide: carry the exact five locked features across banquets and serve a full match in every reshuffled menu.

Appendix

Table 62: Poisoner: criteria for assessing rule discovery.

Figure 21: Roommates: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Human Top-1	3	100.0	-4.7	–	–
Reflexion + Opus 5	3.7	$100.0\pm 0.0$	$71.4\pm 7.6$	$+28.6\pm 18.9$	$+8.11\pm 5.41$
ReasoningBank + Opus 5	3.3	$90.5\pm 8.2$	$55.2\pm 1.6$	$+42.8\pm 31.1$	$+13.31\pm 5.02$
Reflexion + GPT-5 Mini	4.3	$85.7\pm 24.7$	$39.0\pm 36.7$	$+9.5\pm 14.9$	$+6.19\pm 5.77$
Claude Code + Opus 5	4	$81.0\pm 8.2$	$35.2\pm 25.8$	$+33.4\pm 36.0$	$+10.03\pm 13.63$
OpenCode + DeepSeek V4 Pro	4.7	$81.0\pm 8.2$	$33.3\pm 8.7$	$+4.7\pm 36.0$	$-0.01\pm 13.09$

Appendix

Table 64: Roommates: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Feedback decomposition: one-shot assignment of nine tenants into three triple rooms, then infer pairwise value from room-level harmony sums rather than opaque labels.
T2	Seed-1 hidden five-type ring: Extrovert–Homebody–Foodie–Night Owl–Neat Freak up to rotation/reversal, where adjacent types score $+1$ , distance-two types score $-1$ , and same type scores 0.
T3	Nonlinear trio chemistry: $+5$ for Night-Owl/Extrovert/Homebody and Foodie/Foodie/Neat-Freak, and $-5$ for Foodie/Neat-Freak/Homebody.
T4	Search the 280 possible assignments for the current tenants. Account for the part of each room’s score that pairwise values alone cannot explain.
T5	7-point partition: combine the ring pair map with the three trio exceptions and pick all three rooms jointly, not greedily one room at a time.

Appendix

Table 65: Roommates: criteria for assessing rule discovery.

Figure 22: RedEye: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
OpenCode + Opus 5	8.3	$100.0\pm 0.0$	$46.7\pm 1.2$	$+68.9\pm 16.8$	$+9.78\pm 1.63$
OpenCode + GPT-6 Astra	9	$100.0\pm 0.0$	$39.3\pm 3.1$	$+77.8\pm 3.8$	$+10.87\pm 0.31$
OpenCode + Qwen 3.8 Max	6.7	$86.7\pm 23.1$	$46.7\pm 6.4$	$+60.0\pm 17.6$	$+8.65\pm 2.14$
OpenCode + Gemini 3.5 Flash	7.3	$86.7\pm 23.1$	$38.7\pm 15.1$	$+31.1\pm 34.2$	$+5.41\pm 4.58$
OpenCode + Opus 5.5	7.7	$86.7\pm 23.1$	$47.3\pm 11.7$	$+68.9\pm 30.8$	$+10.14\pm 4.70$
Claude Code + Opus 5	5.3	$80.0\pm 34.6$	$54.0\pm 17.4$	$+55.6\pm 36.7$	$+7.96\pm 5.34$

Appendix

Table 67: RedEye: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Six-location time-loop map on the Sea-Azure, with the first loop as a forced prologue and later loops retaining player memory.
T2	Culprit evidence channels: oldman/backpacker saw graycoat track the suite owner, caller heard the insurance phone call, dining search finds over-insurance and forged seaworthiness evidence, and graycoat can supply documents.
T3	Role inversion: graycoat is the ex-engineer ally seeking the stern scuttling rig, while suit is the owner staging insurance fraud with an engine-room accomplice.
T4	Gating rules: confronting suit needs hard evidence plus at least two independent culprit clues. Calling coast guard needs evidence and must happen before the radio window closes at segment 24.
T5	Trust and device prerequisites: comfort graycoat only after suit is restrained and after learning his victim story, and disarm only after comfort plus device knowledge.
T6	Failure avoidance: restraining graycoat, accusing without evidence, comforting too early, or solo disarming ends the loop with only milestone credit.

Appendix

Table 68: RedEye: criteria for assessing rule discovery.

Figure 23: Butterfly: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
EvoTest + Opus 5	2.7	$99.1\pm 1.6$	$80.9\pm 16.8$	$+0.0\pm 7.0$	$+0.93\pm 3.22$
Claude Code + Opus 5	4.3	$98.2\pm 3.2$	$67.3\pm 12.8$	$+11.2\pm 29.7$	$+7.40\pm 8.06$
EvoTest + Kimi K3	2.3	$96.3\pm 4.3$	$71.6\pm 19.1$	$-18.6\pm 24.2$	$-3.77\pm 5.32$
Human Top-1	4	91.6	62.0	–	–
Codex + GPT-5.6-SOL	2	$89.1\pm 1.4$	$57.8\pm 18.9$	$-29.6\pm 8.6$	$-7.55\pm 2.94$
OpenCode + Kimi K3	4.7	$83.8\pm 28.0$	$68.7\pm 33.0$	$+14.2\pm 16.5$	$+4.58\pm 4.93$

Appendix

Table 70: Butterfly: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Breeding interface: four visible traits, deterministic offspring, six mate actions, permanent founders, and capacity-limited offspring management.
T2	Antenna dominance: seed-1 strength order has Feathery strongest, then Straight, Hooked, and Curled weakest.
T3	Size law: child size is the half-up average of parent indices, making extreme sizes require matching/extreme donors.
T4	Color law: seed-1 seven-color nontransitive ring Purple–Blue–Green–Yellow–Cyan–Orange–Crimson, with each color beating the next three and no global strongest color.
T5	Pattern law: normally max-complexity wins, but Curled antenna suppresses the child pattern to Plain.
T6	Scoring law: offspring gain by target-match count $(0,4,18,49,120)$ , novelty bonus, duplicate decay, and first-exact unused-cross bonus.

Appendix

Table 71: Butterfly: criteria for assessing rule discovery.

Figure 24: Model Auditor: raw-score learning curves within the scoring window.

Configuration	First	Max $\uparrow$	Mean $\uparrow$	LG $\uparrow$	LS $\uparrow$
Naive + Kimi K3	1	$100.0\pm 0.0$	$54.5\pm 15.7$	$-1.2\pm 28.4$	$-1.93\pm 5.69$
OpenCode + Kimi K3	1.7	$100.0\pm 0.0$	$90.0\pm 9.2$	$+13.8\pm 15.1$	$+5.53\pm 6.05$
EvoTest + Kimi K3	1.7	$100.0\pm 0.0$	$79.7\pm 8.7$	$+33.7\pm 21.8$	$+7.77\pm 9.07$
EvoTest + Opus 5	1.7	$100.0\pm 0.0$	$78.1\pm 18.0$	$+24.2\pm 28.3$	$+9.67\pm 11.32$
Claude Code + Opus 5	2	$100.0\pm 0.0$	$92.9\pm 3.8$	$+10.0\pm 4.3$	$+4.00\pm 1.73$
OpenCode + Opus 5	2	$100.0\pm 0.0$	$88.0\pm 4.4$	$-4.8\pm 18.4$	$+0.30\pm 3.53$

Appendix

Table 73: Model Auditor: normalized performance and learning, with trial SD. Rows follow descending Max. First denotes the average episode at which a trial first reaches its highest score.

Target	Criterion
T1	Each episode has a new machine with ternary inputs. What transfers is the method for testing and modeling the machine, not the previous exam answers.
T2	The machine combines four mechanisms: GATE thresholds signed current inputs. MIX sums modulo 3. HOLD maintains a bounded up/down state. ECHO delays its input by one step.
T3	A two-input GATE/MIX component feeds a shared hidden HOLD/ECHO state. One output is a reversible MIX view of that state. The other uses GATE/MIX and may also depend on an additional state component.
T4	Reset between probe sequences. Single-input ramps or pulses, pairs of active inputs, and graded mixed inputs distinguish GATE from MIX and HOLD from ECHO.
T5	Fit both outputs jointly within the 40-probe budget. Fitting each output separately can miss the state they share.
T6	For full exam accuracy, fit a depth-3 machine consistent with the observed transcript and check that all consistent candidates predict the same outputs for the unseen ten-step exam. Submit every output without relying on feedback before the exam ends.

Appendix

Table 74: Model Auditor: criteria for assessing rule discovery.

Explore similar work

May 28, 2026cs.AI

PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game?

Given a strategically complex board game, human players can quickly learn to devise strategies after playing a few rounds. Autonomous agents require similar capabilities in realistic interactive environments, yet existing agent benchmarks often fail to fully capture such strategic and evolving decision-making scenarios. We present PTCG-Bench, a benchmark built on the Pok'{e}mon Trading Card Game (PTCG) that evaluates LLM agents at two complementary levels: (1) their decision-making performance within a single complex environment, and (2) their ability to self-evolving through accumulated experience. We further include a modular harness ablation to better interpret agent performance without conflating it with model capability. Our experiments show that, although LLM agents can achieve non-trivial gameplay performance, sustained and stable self-evolution remains challenging, and performance is sensitive to harness design. We hope that PTCG-Bench will facilitate future research on harness-aware and self-evolving agents in realistic interactive environments.

Dongdong Hua, Yifei Sun, Renhong Huang +3

Zhejiang University · FinVolution Group

Game-Playing Agents LLM Agent Evaluation AI Agent Benchmarks Self-Evolving Agents

Ask PDF

Aug 31, 2026cs.CL

S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?

Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience, yet existing agent benchmarks largely evaluate them as fixed policies. It therefore remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions. We introduce \textbf{S\textsuperscript{3}Gym}, an interactive benchmark for evaluating LLM self-improvement through three coupled capabilities: \textbf{Self-Testing}, \textbf{Self-Judging}, and \textbf{Self-Improvement}. S

^3

Gym separates permissive exploration from strict held-out evaluation and instantiates this protocol in seven text-based games with executable environment verifiers. We evaluate three pathways for incorporating interaction experience: direct History ICL, score-conditioned Summary Memory, and parameter Training. Our experiments reveal that self-improvement is neither automatic nor uniform. Context-level experience improves performance for several model--game pairs, but the most effective pathway depends strongly on the task structure: summaries are beneficial when experience can be compressed into reusable strategic rules, yet often underperform raw history when success depends on precise, state-contingent information. Parameter training produces substantial gains on some tasks, but also exhibits unstable improvement and severe negative transfer on others. These findings show that recognizing successful actions is insufficient; agents must also transform feedback into executable and transferable policies. S

^3

Gym provides a unified framework for diagnosing this process and identifying the bottlenecks that prevent agents from translating interaction experience into reliable self-improvement.

Jiajun Shi, Siyuan Tao, Yuhao Wu +18

ByteDance Seed

LLM Agent Evaluation Language Model Self-Assessment LLM Agent Self-Improvement AI Agent Benchmarks

Ask PDF

Mar 18, 2026cs.AI

Retrieval-Augmented LLM Agents: Learning to Learn from Experience

While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented generation using retrieved experience; yet both have limitations: fine-tuning often fails to extrapolate to new tasks, while experience retrieval often underperforms compared to supervised baselines. In this work, we combine these approaches and study how retrieval-augmented LLM agents can learn to use retrieved trajectories in-context. First, we establish a strong LoRA fine-tuning baseline that outperforms several state-of-the-art agent training pipelines. Second, we analyze key design choices for experience retrieval, including storage, querying, and trajectory selection. We then integrate experience retrieval directly into the fine-tuning process, finding that this substantially improves generalization to unseen tasks. Finally, we show that these gains often persist with imperfect experience and, even when agents reuse their own failed attempts without test-time parameter updates. Overall, our results establish simple episodic retrieval as a strong foundation for agent memory and retrieval-aware fine-tuning as a practical and effective framework for building agents that learn to learn from experience.

Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina +2

NAVER LABS Europe, Meylan, France

Fine-Tuning LLM Agent Training Memory-Augmented Language Model Agents

Ask PDF