WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Hang He; Linting Wang; Guanming Liu; Bo Mao; Jinxiu Liu; Yiran Suo; Maosen Zhou; Zhiheng Xi

cs.AIOct 13, 2026

WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Authors: Hang He, Linting Wang, Guanming Liu, Bo Mao, Jinxiu Liu, Yiran Suo, Maosen Zhou, Zhiheng Xi,

Abstract

Scaling interactive environments is important for training long-horizon agents through reinforcement learning. These environments often require further testing and refinement after initial generation, and correcting defects in a task may also require updating its work context and verifier. However, existing construction pipelines offer limited support for such iterative refinement while keeping these components aligned. We introduce WorkRover, a framework for training long-horizon cowork agents through recursive environment evolution. Its harness-driven synthesis enables construction agents to jointly build and validate semantically aligned work contexts, executable tasks, and evidence-grounded verifiers, adapting their actions to workspace state and execution feedback rather than following a fixed construction workflow. To support environment evolution, we introduce Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints. To provide finer-grained training signals, we propose Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with signals from deterministic workspace-progress checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Using WorkRover, we post-train Qwen3.5-35B-A3B to obtain WorkRover-35B-A3B and evaluate it on several cowork-agent benchmarks. Its reported scores exceed those of Qwen3.5-35B-A3B by 26.27 percentage points on JobBench, 29.32 percentage points on OfficeQA Pro, and 413.25 Elo points on GDPval v2. These results suggest that WorkRover offers a promising foundation for scalable training of long-horizon cowork agents.

Figures & tables

Panel of Figure 1 — Figure 1: Cross-benchmark performance. Results on five cowork-agent benchmarks.

Figure 2: Cowork agents: challenges and approaches.

Figure 3: Environment construction and recursive evolution in WorkRover . The Harness coordinates Work Context construction, Task synthesis, and Verifier synthesis and validation. FCR repairs compatibility mismatches under frozen safety gates; verified trajectories support policy learning.

Figure 4: Skill-library coverage. Six largest categories; others aggregated. Tile sizes show shares of skills (a, c) or category memberships (b).

Figure 5: Overview of VTCA. Same-task rollouts yield terminal rewards and checked progress. Bounded prerequisite allocations and local trace costs shape turn-balanced GRPO advantages.

Figure 6: 35B-A3B comparison under matched rollout budgets. Backbone denotes Qwen3.5-35B-A3B before our SFT or RL. Gains compare SFT + VTCA with SFT + E-GRPO. GDPval v2 uses Elo; other scores are percentages.

Figure 7: FCR mechanism, stability, and evolution. (a) Pairwise joint necessity (C: Context; T: Task; H: Harness; V: Verifier). (b) Natural-trigger outcomes. (c) Repair reliability across three generations, measured by SRR and sealed-audit pass rate. (d) Task success across five repair iterations.

Table 8

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: From Deep Research files to verified task workspaces. Instructions and visible invariants define requirements; the dashed green arrow supplies recomputable Reference State anchors. Case 1781 uses two IPSS workbooks; shaded cells mark output values to populate.

Figure 9: Recorded example 1875: prompt-engineering course design. This researcher branch writes teaching_design.md , inspects four segments, and adds the BIG-Bench Hard paper citation in three places. The proposed task reuses captured CoT, SoT, and DSPy paper-source archives to produce a method comparison workbook and three lab briefs with baselines and grading rubrics. Claims cite local source sections; the original archives remain unchanged.

Figure 10: Recorded example 1781: policy research on older adults living alone. The proposed task selects hprj2024_gaiyo_kekka1.xlsx and hprj2024_gaiyo_kekka2.xlsx , retaining category definitions, units, and source-sheet references. The revised policy_memo.md is observed through tool readbacks. Source-page references at $m_{39}$ cite IPSS and MHLW.

Figure 11: Recorded quarterly executive handoff. (a) File, capability, and app context. (b) Selected rollout stages. (c) Revenue and export-path evidence supporting consistent updates. (d) Deliverables, archived evaluation, retained follow-ups, and validation scope.

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 586 papers

CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents
Oct 18, 2026Zhongwei Xie, Yiming Du, Jiaxin Bai +10LLM Agent Training RL for Tool Use
The Hong Kong University of Science and Technology · The Chinese University of Hong Kong · Hong Kong Baptist University +1
Cowork-Gym: Synthesizing Interactive Work Environments at Scale
Oct 13, 2026Anni Zou, Cheng Chen, Jiaji Deng +18Tool-Using Agents LLM Agent Training
Alibaba Token Foundry
Scaling Verifiable Environments for Long-horizon Work Agents
Oct 4, 2026Jiazheng Zhang, Long Ma, Yunxian Yang +23LLM Agents LLM Agent Training
LLM Department, Tencent · Fudan University

Oct 18, 2026J/K move · Enter open · S save

CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents

Zhongwei Xie, Yiming Du, Jiaxin Bai, Baixuan Xu+9

The Hong Kong University of Science and Technology · The Chinese University of Hong Kong · Hong Kong Baptist University+1

LLM Agent Training RL for Tool Use

Open paper AskPDF arXiv

AI coworkers must coordinate tools, communicate across turns, and act on changing workspace state while respecting workflow constraints. Constructing training experience for these capabilities requires coherent requests, service records, tool interfaces, and grading checks. We present CoworkEnv, a capability-guided framework for synthesizing executable training environments for AI coworkers. Workflow requirements guide task construction, and programmatic validation with feedback repair checks generated specifications and grading behavior. Teacher executions yield curated demonstrations for supervised fine-tuning (SFT), while student interactions provide rollouts and rewards for reinforcement learning (RL). Using these resources, we train compact PocketWorker models. For Qwen3-VL-8B-Instruct, SFT raises the mean score from 46.6 to 64.1 on Claw-Eval, including a 31.9-point gain on Multi-turn tasks. An SFT-only extension to ClawMark demonstrates applicability across coworker benchmarks. We further compare RL training dynamics across configurations that vary reward processing and credit assignment. Improving training rewards provide preliminary evidence that the constructed environments support policy optimization.

Abstract

LLM Agent Training

Explore similar work

CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents

Cowork-Gym: Synthesizing Interactive Work Environments at Scale

Scaling Verifiable Environments for Long-horizon Work Agents