Environment Duels: A Microcosm of LLM Co-Evolution

Shibo Jin; Zhiqiu Xu; Yiling Zou; Shangyu Gong; Mayur Naik

cs.CLOct 21, 2026

Environment Duels: A Microcosm of LLM Co-Evolution

Authors: Shibo Jin, Zhiqiu Xu, Yiling Zou, Shangyu Gong, Mayur Naik

Organizations: University of Pennsylvania

Abstract

Through a tangle of real-world interactions, from synthetic training pipelines to data shared on the open web, frontier LLMs absorb what their predecessors and competitors create. How well does each model play its role in this ecosystem, as a creator and as a solver? We introduce Environment Duels, a dynamic agentic benchmark that evaluates both capabilities on interactive environments. Each model writes environments, each with a privileged hint, aiming for tasks that capable solvers fail without the hint and pass with it. Across nine frontier LLMs, solving and authoring prove to be distinct skills: the strongest solvers are not the strongest authors, while smaller models learn more from how others played and write the hardest environments and the most useful hints. Beyond the scores, the pairwise interactions form a microcosm of how models affect one another. Head-to-head outcomes are transitive and so yield a stable ranking; independent authors converge on the same designs; and in a reinforcement-learning study with Qwen3.8-27B, the learner's hint lift on an author's environments tracks how much training on those environments improves it on held-out reasoning and agentic benchmarks, whether the environments are its own or another model's. Each of these patterns has a counterpart in how models shape one another outside the duel. By staging these interactions under controlled conditions, Environment Duels offers a window onto how LLMs teach, challenge and learn from one another. Code is available at https://github.com/LLM-Duels/EnvDuels.

Figures & tables

Figure 1: Overview of Environment Duels. Frontier LLMs create and solve each other’s environments. These interactions form a microcosm of the real-world model ecosystem.

Figure 2: Overall ranking in Environment Duels. Each model’s rank on Solve, Difficulty and Hint lift, with the underlying scores (best in bold). The strongest solvers lead Solve, while three smaller models lead both authoring axes. Difficulty is the no-hint fail rate of all nine solvers on the model’s environments, and Hint lift is in points; hint-lift ties are broken by solve rank.

Figure 3: The best teacher is not the best solver. Left: attempts each author’s hint fixes (orange) and breaks (purple); a mid-ranked solver writes the most useful hints. Right: the breaks come from strong authors’ hints reaching weak solvers, while strong solvers are almost never harmed.

Figure 4: Pass rate before and after the counter-move. Stronger solvers improve less.

Figure 5: What the duel matrix shows beyond the scores. (a) Duels give a stable ranking: each cell is the row model’s pass rate on the column model’s environments minus the reverse (points), orange means the row model wins, and no model loses to one ranked below it. (b) Some authors design alike: each cell is how much more alike two authors’ design plans are than their general similarity predicts ( $\times 100$ ), and Claude-Opus-5 , DeepSeek-V4.1-Flash and GLM-5.3 form the one clear cluster (outlined).

		Gain from training on environments by
	Base	GPT	Qwen (self)	Inkling	DeepSeek
AIME	73.1	$+4.2$	$+2.5$	$+1.3$	$+1.4$
ACEBench-Agent	64.1	$+1.4$	$+1.5$	$+0.6$	$-1.6$
EvalPlus	83.6	$+2.4$	$+1.5$	$+1.8$	$+0.7$
LiveBench	63.1	$+1.2$	$+0.6$	$+0.4$	$-0.1$
Reasoning Gym	20.6	$+0.7$	$+0.6$	$+1.2$	$+0.2$

Table 2: Hint lift in the duel anticipates training gains. Gains of Qwen3.8-27B from reinforcement learning on each author’s environments, in points over the base model. Authors are ordered by Qwen3.8-27B ’s hint lift on their environments in the duel, before any training, and mean gains fall in the same order: the duel anticipates which author teaches the model most. Shading is relative within each row. Columns stand for GPT-5.6-Sol , Qwen3.8-27B , Inkling-Small and DeepSeek-V4.1-Flash .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Environment	Seed	No hint	Hint	Opus	GPT	Manual
qwen3.8-27b/env_007	1697986	0/9	8/9	Invalid	Invalid	Invalid
gemini-3.8-flash/env_002	1452908	0/9	4/9	Invalid	Invalid	Invalid
gemini-3.8-flash/env_002	1697986	0/9	4/9	Invalid	Invalid	Invalid

Appendix

Table 3: Post-duel verification of the flagged final environment versions. Success counts are numbers of models out of nine on the indicated seed and hint condition.

Model	Agent harness	Designer effort	Solver effort
Claude-Opus-5	Claude Code 2.1.271	max	max
GPT-5.6-Sol	Codex CLI 0.155.0	max	max
DeepSeek-V4.1-Flash	DeepSeek Harness 0.1.5-rc.1	max	max
GLM-5.3	Z Code CLI 0.16.5	max	max
Kimi-K3	Kimi Code 0.43.0	max	max
Inkling-Small	OpenCode 1.18.31	max	max

Appendix

Table 4: The nine evaluated models, the agent harness (with version) that runs every call for that model, and configured reasoning effort.

Author	Environment	Author	Environment	Cosine
DeepSeek	Bay 12 Coolant Loop	GLM	Coolant Bay 7	0.281
Opus	Sable Ridge Heat Loop	DeepSeek	Vesta-2	0.255
DeepSeek	Loop 3	GLM	Coolant Bay 7	0.249
DeepSeek	Sluice	Grok	Redwater Cascade	0.249
DeepSeek	The Concord Vault	Qwen	The Five-Seal Vault	0.248

Appendix

Table 5: The five most similar pairs of environments written by different authors, by TF-IDF cosine similarity of their design plans.

Setting	Value
Trained model	Qwen3.8-27B
Authors	4 $\times$ 200 environments
Algorithm	GRPO
Fine-tuning	LoRA
LoRA rank / $\alpha$	32 / 64
Epochs	1

Appendix

Table 6: Reinforcement-learning hyperparameters.

	AIME avg@8		ACEBench-Agent		EvalPlus pass@1
Author	2025	2026	Multi-step	Multi-turn	HumanEval+	MBPP+	Overall
Base model	70.4	75.8	89.4	38.8	93.3	79.4	83.6
GPT-5.6-Sol	74.6	80.1	91.3	39.6	94.5	82.3	86.0
Qwen3.8-27B (self)	71.7	79.6	86.9	44.2	93.3	81.5	85.1
DeepSeek-V4.1-Flash	70.8	78.3	86.3	38.8	92.7	80.7	84.3
Inkling-Small	71.7	77.1	86.9	42.5	94.5	81.5	85.4

Appendix

Table 7: Full downstream results for the base Qwen3.8-27B and for each copy trained on one author’s environments. EvalPlus overall pools the HumanEval+ and MBPP+ problems. In Table 2 , AIME averages the two years, ACEBench-Agent averages its two splits, and Reasoning Gym is avg@3.

Figure 6: Before and after the counter-move. Each author’s no-hint pass rate (left, lower is harder) and hint lift (right), before (open) and after (filled) its counter-move; dotted lines mark the averages over all authors. Authors are ordered by their gain in hint lift.

Phase	Average tokens per call
Plan	2.21M
Generate	3.03M
Check	1.55M
Repair	2.08M
Solve	0.66M

Appendix

Table 8: Average token usage by phase. The averages are calculated over completed calls with reported token counters. M denotes millions of tokens.

Model	Designing tokens	Solving tokens
Gemini-3.8-Flash	504.83M	2.15B
Deepseek-V4.1-Flash	456.50M	793.30M
Claude-Opus-5	221.80M	907.62M
GLM-5.3	160.44M	725.60M
Inkling-Small	135.94M	805.90M
Qwen3.8-27B	131.61M	1.55B

Appendix

Table 9: Token usage by model. The final column reports the estimated total cost across all listed models. M denotes millions and B denotes billions of tokens.

LLM Agent Evaluation

Large language model agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently.

See all 1,217 papers

Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?
Apr 12, 2026Wanyi Chen, Xiao Yang, Xu Yang +7Agentic RL AI Agent Benchmarks
Soochow University · Microsoft Research Asia · Stony Brook University +2
MathDuels: Evaluating LLMs as Problem Posers and Solvers
Apr 23, 2026Zhiqiu Xu, Shibo Jin, Shreya Arya +1Mathematical Reasoning Benchmarks LLM Evaluation
Department of Computer and Information Science, University of Pennsylvania · Rabdos AI · Department of Mathematics, University of Pennsylvania
EnvSimBench: A Benchmark for Evaluating and Improving LLM-Based Environment Simulation
May 8, 2026Yi Liu, TingFeng Hui, Wei Zhang +4AI Agent Evaluation AI Agent Benchmarks
Beijing University of Posts and Telecommunications · The Hong Kong University of Science and Technology · Chongqing University

Apr 12, 2026J/K move · Enter open · S save

Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?

Wanyi Chen, Xiao Yang, Xu Yang, Tianming Sha+6

Soochow University · Microsoft Research Asia · Stony Brook University+2

Agentic RL AI Agent Benchmarks LLM Post-Training LLM Agent Evaluation

Open paper AskPDF arXiv

We introduce Agent2 RL-Bench, a compact diagnostic benchmark for evaluating agentic RL post-training, which tests whether LLM agents can autonomously design, implement, debug, and execute post-training pipelines that improve foundation models. RL post-training increasingly drives model alignment and specialization, yet existing benchmarks are largely static, rewarding supervised fine-tuning or script generation without assessing an agent's ability to close an interactive RL loop. Agent2 RL-Bench provides a unified agent-facing interface: each run starts from an isolated workspace containing a base model, task data, instructions, and a grading API, and agents must iterate within a fixed budget by training models and submitting artifacts for evaluation. The benchmark spans six tasks across three levels, from static rule-based training to judge-based optimization and closed-loop online RL with trajectory collection. Two diagnostic skills, namely runtime recording and post-hoc summarization, enable structured analysis of agent behavior, facilitating smooth and effective iteration of the benchmark's evaluation framework. Across five agent systems and six driver LLMs, agents show intelligent behavior but clear limitations: one RL-oriented run improves ALFWorld from 4.85 to 93.28 via SFT warm-up and GRPO with online rollouts, yet DeepSearchQA remains difficult, most successful routes rely on supervised pipelines, and interactive outcomes show large single-run differences across agent stacks. Overall, Agent2 RL-Bench shows that current agents can sometimes engineer online RL, but stable agent-driven RL post-training remains rare under fixed budgets. It also demonstrates that our benchmark provides a strong and effective evaluation framework for future research in this direction. Code is available at https://github.com/microsoft/RD-Agent/blob/main/rdagent/scenarios/rl/autorl_bench/README.md

Abstract

LLM Agent Evaluation

Explore similar work

Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?

MathDuels: Evaluating LLMs as Problem Posers and Solvers

EnvSimBench: A Benchmark for Evaluating and Improving LLM-Based Environment Simulation