EnvHarness: Awakening Static Worlds for Agent Learning

Chengsong Huang; Zifeng Wang; Rujun Han; Jun Yan; Yanfei Chen; Zoey CuiZhu; Ke Jiang; Peng Xia

cs.AIAug 20, 2026

EnvHarness: Awakening Static Worlds for Agent Learning

Authors: Chengsong Huang, Zifeng Wang, Rujun Han, Jun Yan, Yanfei Chen, Zoey CuiZhu, Ke Jiang, Peng Xia,

Organizations: Washington University in St. Louis · Google Cloud AI Research · University of North Carolina at Chapel Hill · Google Cloud

Abstract

LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.

Figures & tables

Figure 1: Overall performance. Left: Agents learning from EnvHarness environments consistently outperform those learning from the original environments across software engineering and office automation benchmarks, including SWE-bench Verified, OfficeQA, and SpreadsheetBench. Right: On SWE-bench Verified, under an identical environment budget, EnvHarness keeps improving as environments scale, while real and generated environments flatten out.

Figure 2: While an agent harness transforms a frozen LLM into a capable agent via plug-in components (e.g., skills, memory, tools) without altering model weights, EnvHarness applies this same principle to the other side of the interaction. It customizes a frozen environment with plug-in components while leaving original environment unchanged.

	Agent Harness	EnvHarness (Ours)
Base System	Frozen LLM	Static environment
Designed to Solve	Lack of action, memory, or loops	Hardcoded interaction logic
Harness Layer	Capabilities (tools, memory)	Customization (states, rules, observations)
Unified Output	Autonomous agent	Customized environment

Table 1: Analogy between Agent Harness and EnvHarness . Both scale capabilities through external layers rather than changing the core system.

Figure 3: Overview of EnvHarness components wrapping the standard environment interface. The underlying base environment (native state transitions and original task verifier) remains completely frozen. From left to right: the base environment, followed by three EnvHarness components— Stage , Contract , and Chain . Highlighted arrows and headers indicate overridden interface methods, with code blocks showing how each wrapper modifies state initialization, transition dynamics, or observation handling without altering the base environment.

Figure 4: EnvRigger generating EnvHarness components for a target policy based on given task. The execution loop on the left runs the policy against the current environment, which is a frozen base environment wrapped by the active EnvHarness containing accepted components $w_{1},\ldots,w_{k}$ , while the resulting rollout trajectories feed the EnvRigger loop on the right. The EnvRigger operates systematically through four distinct stages: Observe, Diagnose, Write, and Validate, where the last two steps form a write-and-validate loop that generates a candidate component, evaluates it on fresh rollouts, and revises it upon failure.

Skill Source	ALFWorld			WebArena
Skill Source	In-Dist	OOD	Avg.	Reddit	Shopping	Shop Admin	GitLab	Avg.
No Skills	62.6 1.7	60.7 5.2	61.7 3.4	39.6 2.3	35.2 3.3	44.1 2.3	35.8 8.4	38.7 2.3
Original Envs	63.3 2.8	61.4 4.3	62.4 3.4	38.7 9.7	35.2 1.3	44.6 3.0	35.4 4.0	38.5 3.1
GenEnv	63.3 1.2	61.9 2.7	62.6 1.9	–	–	–	–	–
VeriEnv	–	–	–	39.6 4.2	30.2 0.0	49.7 2.4	38.9 5.6	39.6 1.4
EnvHarness Envs	66.2 0.3	70.4 2.3	68.3 1.3	40.6 4.7	37.4 0.3	50.8 1.5	37.7 3.1	41.6 1.8

Table 2: Performance of agents equipped with skills extracted from different environment sources on ALFWorld and WebArena. All numbers are the mean over three independent runs, with standard deviations as gray subscripts . Higher is better for every metric, and the last row reports the improvement of EnvHarness Envs over Original Envs. “–” denotes that the method is benchmark-specific and cannot be applied to the other domain.

Skill Source	SWE-verified		OfficeQA		SpreadsheetBench
Skill Source	SR ( $\uparrow$ )	Average Step ( $\downarrow$ )	EM ( $\uparrow$ )	F1 ( $\uparrow$ )	Pass@1 ( $\uparrow$ )	Mean Score ( $\uparrow$ )
No Skills	47.67 0.93	53.58 2.93	54.23 2.84	55.77 2.98	46.44 0.15	61.32 0.37
Original Envs	49.88 2.59	55.01 1.69	54.40 1.84	55.77 1.59	45.88 1.19	61.47 0.59
SWE-smith	50.12 1.74	54.72 2.03	–	–	–	–
EnvHarness Envs	52.58 2.72	49.61 2.49	56.20 2.34	57.73 2.29	49.15 0.36	62.48 0.27
Improvement	+2.70	+5.40	+1.80	+1.96	+3.27	+1.01

Table 3: Performance of agents equipped with skills extracted from different environment sources on SWE-bench Verified, OfficeQA, and SpreadsheetBench. Standard deviations are gray subscripts . “–” denotes that the method is benchmark-specific and cannot be applied to the other domain.

	ALFWorld			WebShop
Training Set	In-Dist	OOD	Avg.	Score	SR
Original Envs	81.4	89.6	85.5	75.6	66.0
EnvHarness Envs	87.9	88.8	88.4	79.2	67.4

Table 4: Reinforcement learning on ALFWorld and WebShop, comparing policies trained on the original environments and on EnvHarness environments. ALFWorld is scored by success rate on in-distribution and held-out instance types; WebShop reports environment score and success rate.

	SR (%) $\uparrow$	AS $\downarrow$
No Skills	47.67	53.58
Original Envs	49.88	55.01
EnvHarness ( Stage / Contract Only)	52.58	49.61
EnvHarness ( Chain Only)	49.63	41.96
Combined Skills ( Stage / Contract + Chain )	54.30	43.12

Table 5: Performance on long-horizon environments. SR stands for Success Rate, and AS represents Average Steps.

Figure 5: Environment scaling on SWE-bench Verified. All three sources supply the same number of environments and feed the same extraction and retrieval protocol.

Figure 6: Cross-model results on SWE-bench Verified. Each group represents one policy model, ordered from weakest to strongest. Bars show success rates with no skills, with skills from unmodified environments, and with skills from EnvHarness environments. Percentages represent relative gains over original environments.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Paper	Code	Emitted field
Stage	Setups	in_env_actions
Contract	Rules	rules_code
Chain	Link	—

Appendix

Table 6: Component names in the paper and in the release.

Figure 7: Class structure and instance structure of the framework. Top left, seven Bridges adapt heterogeneous native runtimes to the abstract ActionableEnv contract, and the dashed box marks that further Bridges plug in the same way. Top right, EnvHarness is the abstract decorator over the same contract, and the three shipped components derive from it. Bottom, the instance view shows one EnvHarness assembled as an ordered stack of components over a Bridge. Each layer holds the next as inner and, by construction, never accesses the runtime beneath the interface. Any class honoring the contract is a valid component, and the family is open to extension.

Benchmark	Training	Evaluation
ALFWorld	100 tasks from the standard train set	all remaining held-out tasks
WebArena	20 tasks per sub-domain	all remaining tasks
SWE-bench	100 tasks from SWE-bench Lite	407 Verified issues not in Lite
OfficeQA	50 tasks (official split)	172 official test tasks
SpreadsheetBench	100 of the 400 verified tasks	299 held-out tasks (897 instances)

Appendix

Table 7: Training and evaluation splits per benchmark.

Stage	Parameter	Value
Observe	Baseline rollouts per task ( $K$ )	5
Write	Components per candidate	unbounded (designer’s choice)
Validate	Fresh rollouts per candidate ( $K$ )	5
	Revision budget (write–validate rounds)	5
General	Designer backbone	same as policy

Appendix

Table 8: EnvRigger hyperparameters. The same settings are used across all benchmarks.

Skill Source	Gemini 3.1 Flash-Lite		Qwen3.6 27B		Gemini 3.5 Flash		Claude Sonnet 4.6
Skill Source	SR ( $\uparrow$ )	AS ( $\downarrow$ )	SR ( $\uparrow$ )	AS ( $\downarrow$ )	SR ( $\uparrow$ )	AS ( $\downarrow$ )	SR ( $\uparrow$ )	AS ( $\downarrow$ )
No Skills	30.7	36.7	41.0	69.8	47.7	53.6	67.2	29.3
Original Envs	36.8	50.0	48.4	37.1	49.9	55.0	69.2	25.4
EnvHarness Envs	40.0	50.6	52.1	40.8	52.6	49.6	72.4	25.6

Appendix

Table 9: Cross-model results on SWE-bench Verified under the same protocol as Table 3 ; the Gemini 3.5 Flash column reports the same runs as that table, rounded to one decimal. SR is success rate and AS is average steps. Best in bold .

Benchmark	Held-out Type	Orig.	EnvHarness	$\Delta$
ALFWorld	clean	54.8	71.2	+16.4
	cool	38.5	39.3	+0.8
	heat	61.1	52.4	-8.7
	look_lamp	79.0	82.7	+3.7
	simple	83.6	83.6	0.0
	two_obj	46.4	52.9	+6.5

Appendix

Table 10: Leave-one-out generalization on ALFWorld. Skills are extracted from environments of all task types except the held-out type, then evaluated on it.

Benchmark	Method	Design Tok.	Rollout Tok.	Total Tok.
ALFWorld	GenEnv	38K	64.2M	64.2M
ALFWorld	EnvHarness	1.46M	226.6M	228.0M
WebArena	VeriEnv	20K	137.7M	137.8M
WebArena	EnvHarness	1.58M	135.7M	137.3M

Appendix

Table 11: Estimated token consumption. Design tokens cover the calls that propose and refine components; rollout tokens cover every interaction a method drives. These rollouts are not the same kind of work across rows: EnvHarness and VeriEnv execute them against the real environment, whereas GenEnv’s are LLM-simulated.

Metric	Band	Orig.	EnvHarness
Success rate (SR)	$[0.4,0.6]$	6.0	80.0
Avg. steps (AS)	$[25,35]$	18.0	53.0

Appendix

Table 12: Objective metric targeting on ALFWorld. Entries are the percentage of tasks whose measured value falls inside the target band, before and after reshaping.

ALFWorld
Axis	Specified weakness	Generated component	Distilled skill
Stage	Takes objects from closed containers without opening them	Target object starts inside a closed drawer	Pre-Interaction State Verification
Stage	Searches containers in an inefficient order	Three drawers pre-opened to stage an ordering	Semantic Container Prioritization
Stage	Forgets the second object in multi-object tasks	First sub-goal completed in advance	Task-State Verification Loop
WebArena
Contract $f_{A}$	Concludes without scrolling to content below the fold	Retrieval actions blocked until a scroll happens	Incremental Viewport Expansion

Appendix

Table 13: Nine specified weaknesses, the component the designer generated, and the skill distilled from the trajectories collected in the reshaped environment.

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 381 papers

EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
May 18, 2026Minrui Xu, Zilin Wang, Mengyi DENG +12Tool-Augmented Language Model Agents Agentic RL
LARK, HKUST (GZ) · University of Cambridge · UCL +2
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
Jun 16, 2026Chao Chen, Chengzu Li, Zhiwei Li +2Reinforcement Learning LLM-Guided RL
1LARK, HKUST (GZ) · University of Cambridge · 3HKUST
Breaking the Environment Wall: A Unified Framework for Preparing and Evolving Agent-Native Environments
Sep 24, 2026Yukai Wu, Yuanjing Yang, Le Zhou +8LLM Agent Self-Improvement LLM Agent Training
Shanghai Jiao Tong University · Theseus Labs · Tencent Hunyuan

May 18, 2026J/K move · Enter open · S save

EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL

Minrui Xu, Zilin Wang, Mengyi DENG, Zhiwei Li+11

LARK, HKUST (GZ) · University of Cambridge · UCL+2

Tool-Augmented Language Model Agents Agentic RL LLM Agent Trajectory Synthesis LLM Agents LLM Agent Training RL for Tool Use

Open paper AskPDF arXiv

Equipping LLMs with tool-use capabilities via Agentic Reinforcement Learning (Agentic RL) is bottlenecked by two challenges: the lack of scalable, robust execution environments and the scarcity of realistic training data that captures implicit human reasoning. Existing approaches depend on costly real-world APIs, hallucination-prone LLM simulators, or synthetic environments that are often single-turn or depend on pre-collected documents. Moreover, synthetic trajectories are frequently over-specified, resembling instruction sequences rather than natural human intents, reducing their effectiveness for RL training. We introduce EnvFactory, a fully automated framework that addresses both challenges. EnvFactory autonomously explores and verifies stateful, executable tool environments from authentic resources, and synthesizes natural multi-turn trajectories through topology-aware sampling and calibrated refinement, producing grounded queries with implicit intents. Using only 85 verified environments across 7 domains, EnvFactory generates 2,575 SFT and RL trajectories. Despite using significantly fewer environments than prior work, which are often 5 times more, EnvFactory achieves superior training efficiency and downstream performance, improving Qwen3-series models by up to +15% on BFCLv3, +8.6% on MCP-Atlas, and +6% on conversational benchmarks including

τ^2

-Bench and VitaBench. By fully automating both environment construction and trajectory synthesis, EnvFactory provides a scalable, extensible, and robust foundation for Agentic RL.

Abstract

LLM Agent Harnesses

Explore similar work

EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL

From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning

Breaking the Environment Wall: A Unified Framework for Preparing and Evolving Agent-Native Environments