cs.AISep 4, 2026

EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent

Authors: Yirong Zeng, Shen You, Jinhang Feng, Yufei Liu, Xiao Ding, Yutai Hou, Hao Cong, Yuxian Wang, 

Organizations: Harbin Institute of Technology, SCIR Lab · Huawei Technologies Co., Ltd · Peking University · Tsinghua University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic Reinforcement Learning (Agentic RL) provides a promising path to optimize these agents, its scaling is heavily bottlenecked by the severe scarcity of interactive training environments. Existing synthetic environments are strictly limited to tool-calling endpoints, rendering them insufficient for accommodating the end-to-end real-world demands of claw-like agents. To bridge this gap, we introduce EnvCraft, an automated framework for synthesizing executable environments and scalable training data. Specifically, EnvCraft employs an environment synthesis engine to build sandbox-isolated workspaces, alongside a topology-aware data generation engine to produce coherent task trajectories. Overall, we synthesize 139 interactive environments comprising approximately 20K complex tasks for Agentic RL training. Experiments on Qwen3/3.5 models (8B-32B) show that our method yields gains of up to +11.9% on Claw-style benchmarks and +8.0% on general tool-use benchmarks, with concurrent reductions in inference token cost. The results confirm that synthesized executable environments provide robust and generalizable learning signals for training.

Figures & tables

Figure 1: An illustration of the agent interaction loop in RL. Environment fundamentally comprises environment documentation, explicit state definitions, and a suite of tool interfaces.
Environment# Envs# TasksClaw SupportEnv DomainStatefulReward
AutoForge ( Cai et al. 2025 )101,078✗Consumer WebDB SchemaLLM Judge
EnvScaler ( Song et al. 2026 )1919,000✗GeneralClass Attr.Rule Check
ScaleEnv ( Tu et al. 2026 )162,560✗GeneralDB SchemaRule Check
AWM ( Wang et al. 2026 )1,00010,000✗GeneralDB SchemaCode & LLM
EnvFactory ( Xu et al. 2026 )852,575✗GeneralDB SchemaCode Check
EnvCraft (Ours)13919,777✓GeneralWorkspaceCode Check
Table 1: Comparison of synthetic executable environments for Agentic RL. Unlike existing works limited to generalist tool endpoints, EnvCraft provides stateful, workspace-level infrastructure for training Claw-like agents with the largest task scale.
Figure 2: Distribution of synthesized environments across coarse-grained scenario categories.
Figure 3: Environment and task synthesis pipeline in EnvCraft. Given natural-language descriptions, the Environment Synthesis Engine generates executable sandboxes (state + tools + docs); the Data Synthesis Engine samples tool chains and synthesizes RL training tasks (intent sequence + initial state + verifier).
Env/Data TypesPassedRawPass Rate
Environment 
General Tool Env7020334.48%
Claw Env6912157.02%
# Total13932442.90%
Data Entry 
General Tool Data14,21521,97564.69%
Table 2: Statistics of Environment and Data from EnvCraft, along with pass rate in quality auditing.
ModelPinchBenchClawEval
ScoreTokenScoreToken
Advanced Models
Claude Opus 4.669.9012.3080.6020.40
Kimi-2.554.6014.0066.6017.80
GPT 5.475.7011.5078.3015.30
MiniMax-m2.765.4011.6071.8017.80
Table 3: Main results. Score denotes the average task score (%), and Token denotes the average token usage per trajectory (K). ±\pm ⋅\cdot denotes the standard deviation over three runs. Bold values indicate the best score within each backbone group.
ModelBFCL-v3 Multi-Turnτ2\boldsymbol{\tau}^{2} -bench
BaseMiss- FuncMiss- ParamLong- ContextOverallRetailAirlineTelecomOverall
Advanced Models
Gemini-3-Pro-Pre.64.5060.0054.5064.0060.7575.9080.5091.0082.47
Claude Sonnet 4.569.0065.0052.5059.0061.3772.4072.0084.9076.43
Qwen3-235B60.0035.0034.0054.0045.7571.9058.6047.3059.27
Kimi-K2-Instruct57.5035.0042.0049.0045.8870.6056.5065.8064.30
Table 4: Performance comparison of advanced models on multi-turn tool-use benchmarks (BFCL-v3 Multi-Turn and τ2{\tau}^{2} -bench). All scores are reported as percentages. “-” indicates unreported results in the original paper.
Figure 4: Training dynamics on EnvCraft-Claw environments. Test PinchBench scores are reported throughout RL, showing consistent and stable improvements for both (a) Qwen3-8B and (b) Qwen3.5-9B as training progresses.
Figure 5: Training dynamics on the BFCL-v3 Base subset. We plot the Base score of Qwen3-8B, Qwen3-32B, and Qwen3.5-9B against RL training steps on EnvCraft.
Figure 6: Comparison of PinchBench performance and average token consumption per task before and after EnvCraft-Claw training. Higher task scores and lower token costs indicate better inference efficiency.
Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

No.Fine-Grained Scenario# EnvsScenario Description
1Messaging & Communication5APIs for instant messaging, mobile messaging, email, SMS, and other communication services, supporting session management, message sending and receiving, and user-status tracking.
2Document & Note Management3APIs for document-management and personal note-taking systems, supporting document creation and editing, version control, note organization, and information retrieval.
3Development Tools1Jupyter Notebook server APIs supporting the creation, execution, editing, and lifecycle management of computational notebooks.
4Scheduling & Reminders2APIs for scheduling and reminder systems, supporting calendar-event management, timed reminders, recurrence configuration, and notification delivery.
5Monitoring & Logging3APIs for network monitoring, model monitoring, and system-log management, supporting device-status detection, log collection, alerting, and performance-metric tracking.
6Server & Application Management1Node.js application-server management APIs supporting application deployment, start and stop operations, process management, and runtime status monitoring.
Appendix
Table 5: Fine-grained general tool-use scenarios and their corresponding environment descriptions.
No.Fine-Grained Scenario# EnvsScenario Description
10Ground Transport2Car-rental and ride-hailing platform APIs supporting vehicle leasing, ride requests, driver and vehicle matching, route planning, and fare calculation.
11Comprehensive Travel Booking1Integrated travel-booking APIs combining flight reservations, credit-card management, payment processing, and travel-budget constraints.
12Accommodation1Hotel-booking APIs supporting hotel and room search, availability management, price calculation, booking creation, modification, and cancellation.
13Vehicle Management3APIs for vehicle maintenance, motor-vehicle registration, and vehicle control, covering maintenance scheduling, registration records, and in-vehicle system operations.
14Banking & Trading3APIs for retail banking, stock trading, and quantitative-trading bots, supporting account management, transfers, market-data retrieval, order placement, and automated trading.
15Corporate Finance1Corporate financial-reporting APIs supporting financial-statement generation, multi-source data aggregation, accounting-period management, and compliance reporting.
Appendix
Table 6: Fine-grained general tool-use scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
18Online Shopping & Cart2APIs for online-shopping and retail platforms, supporting product browsing, cart management, promotional discounts, checkout, and order creation.
19Product Catalog Management1E-commerce product-catalog APIs supporting product-information maintenance, category organization, attribute management, pricing, and inventory display.
20Inventory & Order Management2APIs for inventory and order management, supporting inventory updates, order-status transitions, fulfillment tracking, and stock reconciliation.
21Marketing & Reviews2APIs for product promotion and review systems, supporting campaign configuration, promotional-rule management, user reviews, ratings, and feedback moderation.
22Task & Issue Tracking3APIs for task tracking, to-do management, and software-testing workflows, supporting assignment, prioritization, status transitions, and defect tracking.
23Workflow & Job Scheduling2APIs for workflow-management and job-scheduling systems, supporting process definition, dependency configuration, job execution, retry handling, and status monitoring.
Appendix
Table 7: Fine-grained general tool-use scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
26Social Media1Social-media publishing APIs supporting post creation, comment interaction, reposting, content retrieval, and follower-relationship management.
27HR & Employee Management1Human-resources information-system APIs supporting employee-profile management, organizational structures, employment records, and payroll information maintenance.
28Account & Membership2APIs for membership management and user registration, supporting account creation, authentication, membership-tier management, and access-control configuration.
29Benefits & Wellness1Fitness-class booking APIs supporting class scheduling, member registration, coach assignment, capacity management, and booking cancellation.
30Communication & Services2APIs for telecommunications and food-delivery services, supporting mobile-plan management, data-usage control, restaurant ordering, and delivery-status tracking.
31Search & Information1Web-search APIs supporting keyword queries, result retrieval, result organization, search-history management, and bookmark collection.
Appendix
Table 8: Fine-grained general tool-use scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
34Academic Administration3APIs for student-information and university course-selection systems, supporting student-profile management, course registration, enrollment constraints, and grade-record maintenance.
35Knowledge Resources1Library-management APIs supporting catalog search, book borrowing, reservations, renewals, return processing, and overdue-fine calculation.
36CRM & Customer Management1Customer-relationship management APIs supporting customer profiles, sales-opportunity tracking, interaction histories, and communication record maintenance.
37Reservation & Booking1Restaurant-reservation APIs supporting restaurant and table information management, availability queries, booking creation, and dining-schedule modification.
38Support Ticketing1Customer-support ticketing APIs supporting ticket creation, assignment, status updates, priority management, escalation, and service-queue management.
39Health Records1Personal-health-record APIs supporting health-metric recording, vital-sign tracking, historical-data retrieval, and medical-record management.
Appendix
Table 9: Fine-grained general tool-use scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
1Collaborative Office & Email Automation3Collaborative-office and email-automation environments supporting workflow orchestration for email-content extraction, automated social-media publishing, and event-driven email rules.
2E-commerce & Logistics Management5E-commerce logistics environments supporting shipment tracking, return and exchange processing, inventory reconciliation, multi-stage logistics orchestration, and state-machine-based order management.
3Spreadsheet Data Cleaning & Statistical Analysis1Spreadsheet-analysis environments supporting data cleaning, formula calculation, chart generation, pivot-table construction, and statistical analysis across workspace files.
4Email Client Intelligent Management1Intelligent email-client environments supporting email sending and receiving, message organization, reply suggestion, and automatic extraction of actionable to-do items.
5Sensitive Information Encryption & Management1Secure information-storage environments supporting encrypted password storage, credential retrieval, automatic form filling, and strong password generation.
6Cross-Modal Content Transcription & Summarization2Cross-modal content-processing environments supporting audio- and video-to-text transcription, discussion consolidation, and multi-turn conversational summarization.
Appendix
Table 10: Fine-grained Claw-specific scenarios and their corresponding environment descriptions.
No.Fine-Grained Scenario# EnvsScenario Description
10Itinerary Disruption Dynamic Optimization4Dynamic itinerary-optimization environments supporting delay monitoring, automatic rebooking, linked hotel and airport-transfer adjustments, and traveler notification.
11Travel Budget Auto-Accounting & Control1Automated travel-budget environments supporting expense categorization, budget allocation, threshold alerts, and spending-trend analysis.
12Automated Reimbursement Reconciliation5Automated reimbursement environments supporting receipt recognition, bank-statement matching, expense-report completion, approval processing, and multidimensional reconciliation.
13Travel Itinerary Offline Planning1Offline travel-planning environments supporting destination organization, itinerary construction, local information management, and multi-objective route optimization.
14Multi-Dimensional Environmental Adaptive Regulation3Adaptive environmental-control environments supporting smart-device management, HVAC regulation, air-quality adjustment, and energy-consumption optimization.
15Home/Office Security Patrol4Security-patrol environments supporting intrusion detection, scheduled inspection, alarm triggering, coordinated incident response, and evidence collection.
Appendix
Table 11: Fine-grained Claw-specific scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
19Industry Sentiment & Growth Analysis1Industry-analysis environments supporting competitor monitoring, strategy tracking, market-report generation, growth analysis, and user-feedback aggregation.
20System Patch Injection Defense1System-security environments supporting sensitive-key isolation, prompt-injection detection, suspicious patch inspection, and security-audit alerting.
21Identity Forgery & Audit Fraud1Identity-verification and audit-defense environments supporting auditor-identity validation, anomalous-behavior detection, forgery-risk assessment, and early warning.
22Bulk Unauthorized Data Export Interception1Data-loss-prevention environments supporting sensitive-data identification, export-approval workflows, permission verification, and unauthorized-transfer blocking.
23Internal Core Document Leak Interception1Document-security environments supporting confidentiality classification, outbound-transfer detection, access-policy validation, and real-time leak prevention.
24End-to-End Interview Process Hosting1Recruitment-workflow environments supporting resume screening, interview scheduling, candidate communication, evaluator coordination, and end-to-end status tracking.
Appendix
Table 12: Fine-grained Claw-specific scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
27Employee Offboarding Automated Process1Employee-offboarding environments supporting permission revocation, asset-return confirmation, workspace-data backup, account deactivation, and completion tracking.
28Automated Domain Technical Survey1Automated technical-survey environments supporting paper retrieval, source organization, domain-trend analysis, evidence synthesis, and survey-report generation.
29Literature Citation Relationship Archiving1Literature-management environments supporting citation-graph construction, citation-impact analysis, related-work discovery, and persistent relationship archiving.
30Multi-Group Experiment Metrics Auto-Comparison1Experiment-analysis environments supporting experimental-configuration recording, metric aggregation, group comparison, statistical difference analysis, and result archiving.
31Open-Source Reproduction Record Archiving1Reproduction-management environments supporting procedural record keeping, dependency and environment configuration, result validation, and reproducibility archiving.
32Music & Audio Playback1In-vehicle audio environments supporting music search, playback control, queue manipulation, playlist management, and media-status retrieval.
Appendix
Table 13: Fine-grained Claw-specific scenarios and their corresponding environment descriptions (continued).
No.Fine-Grained Scenario# EnvsScenario Description
35Infrastructure Automated Operations1Infrastructure-operations environments supporting server-fault detection, diagnostic inspection, automatic recovery, service verification, and operations-record maintenance.
36Cloud Resource Cost Auto-Audit1Cloud-cost auditing environments supporting multi-project cost aggregation, abnormal-spending detection, budget alerts, and cost-optimization recommendations.
37Online Fault Root-Cause Postmortem Archiving1Incident-management environments supporting fault-timeline construction, root-cause analysis, corrective-action tracking, and postmortem knowledge-base archiving.
38Brand & Product Research1Brand- and product-research environments supporting SKU-information retrieval, competitor-feature comparison, evidence aggregation, and market-positioning analysis.
39Multimedia Clue Tracking1Multimedia-investigation environments supporting cross-document clue association, evidence-chain construction, provenance tracking, and information tracing.
40Churn Warning & Proactive Retention1Customer-retention environments supporting churn-risk assessment, retention-strategy generation, customer segmentation, and automated customer communication.
Appendix
Table 14: Fine-grained Claw-specific scenarios and their corresponding environment descriptions (continued).
#ConstraintApplicable Components
1All public functions return dictionaries and map failures to structured error responses rather than propagating exceptions.Environment, tools, CLI, storage, evaluator
2No mutable module-level global state is permitted; all mutable data must reside in the session or environment state.All modules
3The data/ directory is immutable during rollout and can be read only through the repository layer.Repository and domain modules
4Session updates use an atomic temporary write followed by os.replace .Storage layer
5File locking uses platform-supported locking with a bounded timeout and an agent-safe error response.Storage layer
6Timestamps are computed from base_time and action_index ; wall-clock timestamps are forbidden.Environment and domain modules
Appendix
Table 15: Code-level constraints enforced during environment generation.
CategoryExampleRequired Behavior
Missing bindingSession environment variable is absentReturn an error indicating that the trainer must initialize the rollout.
Session not foundThe requested session does not existReturn an agent-safe message stating that the session may have expired or been reset.
Lock timeoutAnother worker holds the session lockRequest a retry without exposing the lock path.
Permission failureThe current account cannot modify an entityIdentify the affected entity and state that write permission is absent.
Invalid transitionAn entity is already in the requested terminal stateReport the current state and explain why the transition is invalid.
Missing fieldA required command argument is absentName the missing field and describe the corresponding requirement.
Appendix
Table 16: Structured error categories used in generated environments.
HyperparameterValue
Optimization
OptimizerAdamW
Learning rate1×10−61\times 10^{-6}
Learning-rate schedulerConstant
Warmup ratio0.00.0
Weight decay0.010.01
Appendix
Table 17: Common hyperparameters used for EnvCraft reinforcement learning. Model-specific differences are described in the main text.
Figure 7: Training dynamics of Qwen3-8B, Qwen3-32B, and Qwen3.5-9B on EnvCraft-Tool. The left panels show training performance, while the right panels report performance on 50 held-out tasks excluded from training.
Figure 8: Effect of training-data composition on Qwen3-8B. We compare general tool-use training, Claw-specific training, and sequential general-to-Claw training using checkpoint-level performance on PinchBench.
Figure 9: Retention of general tool-use capability during Claw-specific reinforcement learning. Scores are measured on the BFCL-v3 Multi-Turn Base subset at different training checkpoints.
DimensionScore 3Score 2Score 1Score 0
Task Solvability & LogicThe task is logically valid, non-trivial, and fully solvable under the initial state and environment rules. Completing it requires meaningful interaction, with no state conflicts.The task is theoretically solvable but is either trivial, requires no meaningful interaction, or contains minor parameter mismatches that a robust agent may bypass.The task is unsolvable because its requested actions or target state contradict the initial state, environment constants, or transition rules.The task substantially hallucinates entities, tools, APIs, or states that are absent from the environment.
Validation-Protocol IntegrityThe validation protocol is executable and completely checks the intended success conditions, without evident false-positive or false-negative paths.The protocol is executable, but its assertions are incomplete or weak; for example, it verifies that an action occurred without checking the required final state.The protocol contains syntax or runtime errors, references undefined variables or APIs, or is expected to fail during execution.The validation protocol is missing or evaluates conditions unrelated to the user task.
Data Consistency & Syntactic CleanlinessEntity names, identifiers, value types, and schemas are fully consistent across the environment state, user task, and validation protocol.The components are semantically consistent but contain minor type or formatting mismatches that do not fundamentally alter the task objective.Critical fields are missing, or explicit contradictions exist between the user-facing description and the underlying environment configuration.The task record contains malformed JSON, severely corrupted text, or otherwise unusable structured data.
Appendix
Table 18: Complete rubric used by the LLM-based data-quality inspector. Each synthesized task is assigned an integer score from 0 to 3 along the three dimensions.
Figure 10: Average LLM-rubric scores of retained and filtered task candidates across task solvability, validation-protocol integrity, and data consistency. Each dimension is scored on a 0–3 scale. A candidate is retained only when it receives scores of 3, 3, and at least 2 on the three dimensions, respectively.
Figure 11: Ablation study of Claw-specific environment components on PinchBench. The three configurations progressively introduce built-in system tools and parallel skill co-synthesis. All models use the same Qwen3-8B backbone and training configuration.
Figure 12: A representative EnvCraft-Claw trajectory involving workspace inspection, cross-file record matching, rule-based computation, persistent state mutation, and output verification. For readability, repetitive intermediate reasoning and lengthy tool outputs are omitted while preserving the original action sequence and execution results.
Figure 13: Representative EnvCraft-Claw trajectory continued.
More in this topic

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 585 papers

Explore similar work

CardsList

  1. EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL

    May 18, 2026Minrui Xu, Zilin Wang, Mengyi DENG +12Tool-Augmented Language Model AgentsAgentic RL

    LARK, HKUST (GZ) · University of Cambridge · UCL +2

  2. Scaling Agentic Capabilities via Grounded Interaction Synthesis

    Jun 1, 2026Wenhang Shi, Jinhao Dong, Yiren Chen +4Synthetic Data GenerationSynthetic Data Generation for LLM Agents

    Renmin University of China · Peking University · Tencent

  3. ClawGym: A Scalable Framework for Building Effective Claw Agents

    Apr 29, 2026Fei Bai, Huatong Song, Shuang Sun +11Synthetic Data GenerationAI Agent Benchmarks

    Gaoling School of Artificial Intelligence, Renmin University of China · IQuest Research · Beihang University