AutoGym: Blueprint-First Generation of Verifiable Agent Gyms

Aarati Andrea Noronha; Kavya Ravikumar; Carly Xiaoyu Lin

cs.AISep 18, 2026

AutoGym: Blueprint-First Generation of Verifiable Agent Gyms

Authors: Aarati Andrea Noronha, Kavya Ravikumar, Carly Xiaoyu Lin

Organizations: Amazon AGI

Abstract

Training agents with reinforcement learning requires a gym, comprising a task, an executable environment in which the task can be attempted, and a verifier that reliably distinguishes success from failure. Constructing such gyms remains manual, expensive, and static. Task sets saturate as models improve and are increasingly exposed to contamination. Synthetic generation offers scale, but single-pass synthesis produces tasks whose difficulty is largely cosmetic. Models comparable in capability solve them despite convoluted phrasing, and correctness must be adjudicated post-hoc by unreliable LLM judges. We present AutoGym, a framework that generates complete gyms (tasks, executable environments, and verifiers) from a minimal domain seed or prior model trajectories. AutoGym introduces three mechanisms. (1) Blueprint-first generation specifies the valid solution space, environment requirements, and verification criteria before the environment is materialized, making solvability a construction prerequisite rather than a property verified after the fact. (2) Explicit generation parameters control task topology, interaction depth, capability axes, question obfuscation, and distractor composition, enabling fine-grained difficulty steering. (3) Active curriculum synthesis uses performance-informed calibration to adjust the distribution over these parameters as model capabilities evolve. Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models.

Figures & tables

Figure 1: Overview of the AutoGym pipeline. A blueprint specifying the task context, valid solution space, environment requirements, and verification criteria is constructed first, then materialized into an executable environment with controlled distractors, and finally paired with verifiers derived from the blueprint.

	Claude Opus 4.6			gpt-oss-120b
Gym	Hard (%)	Med (%)	Easy (%)	Hard (%)	Med (%)	Easy (%)
Human expert-authored, N = 300	21.5	12.4	66.1	40.0	20.0	40.0
Agent World Model, N = 217	4.0	6.0	90.0	9.0	13.0	78.0
AutoGym, C1 (low), N = 297	10.0	34.7	55.3	33.4	40.5	26.1
AutoGym, C2 (mid-to-hard), N = 280	39.0	10.0	51.0	70.0	17.0	13.0

Table 1: Difficulty distributions for the productivity gym (eight-rollout protocol). Tasks are categorized by mean verifier score: hard (

<0.30

), medium (

0.30

–

0.80

), easy (

\geq 0.80

). All conditions share the same 7-tool interface. C1/C2 settings in Appendix A.5 .

	Hard (%)	Medium (%)	Easy (%)
Before iterative repair (350 tasks)	57.0	13.0	30.0
After iterative repair (280 retained)	39.0	10.0	51.0

Table 2: Difficulty distribution before and after three repair iterations on gym generated with C2 configuration (productivity gym, Claude Opus 4.6).

	Scope compliance	Groundedness	Scalability	Task realism	Environment realism	Diversity	Interface realism
Score	92.7	91.4	90.0	89.0	86.0	80.0	78.0

Table 3: Certification scores for the productivity gym (0–100 scale).

	Claude Opus 4.6			gpt-oss-120b
	Hard (%)	Med (%)	Easy (%)	Hard (%)	Med (%)	Easy (%)
Temporal-reasoning suite	6.5	4.3	89.2	20.7	20.6	58.7

Table 4: Difficulty distributions for the failure-driven temporal-reasoning gym (eight-rollout protocol). Tasks are categorized by mean verifier score: hard (

<0.30

), medium (

0.30

–

0.80

), easy (

\geq 0.80

	Scope compliance	Groundedness	Scalability	Task realism	Environment realism	Diversity	Interface realism
Score	96.7	93.4	97.0	98.0	96.3	85.0	99.0

Table 5: Certification scores for the temporal reasoning gym (0–100 scale).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Pattern	Description	Configurable	Default Range
1	Direct Retrieval	Locate one entity, extract a value or subset	–	–
2	Multi-hop	Output of query A is the filter for query B	No of hops	2–4
2	Multi-constraint	Multiple filter criteria applied simultaneously	No of constraints	2–5
2	List Aggregation	Collect all matching items exhaustively	No of list elements	3–20
3	Cross-source Comparison	Same metric from 2+ independent sources, compare or rank	No of sources	2–4
3	Parallel Synthesis	N entities, different dimensions of same subject, combine into unified answer	No of entities to merge	3–5

Appendix

Table 6: Task topology patterns. Each pattern defines a structural type of reasoning. The configurable range controls within-pattern difficulty.

Axis	Description
Quantitative computation	Answer requires arithmetic (count, sum, ratio, percentage, weighted average), not just extraction
Exhaustive aggregation	Must collect all matching items across paginated results without stopping early
Ranking / comparison	Find max/min/top-N/outlier across a set, or compare entities on multiple dimensions
Temporal reasoning	Filter by time window, compare across periods, or handle versioned/dated records
Negative verification	Correct answer may be “none” / “zero” / “not found”; agent must report absence honestly
Implicit criteria	The question does not state the threshold or rule; the agent must discover it from a reference resource

Appendix

Table 8

Score	Level	What the request contains
0.0–0.3	Moderate	May name the general domain area, time scope, and regulatory frameworks. Specific entities and thresholds still omitted.
0.4–0.6	Vague	States only the goal. No scope hints, time references, or specific entities.
0.7–0.9	Very vague	1–2 sentences of business-intent language only.
1.0	Maximally vague	Contains zero nouns from the blueprint’s entity names. Agent must discover data sources entirely on its own.

Appendix

Table 9

Scenario and persona
Parameter	C1	C2
No of scenarios	5	5
No of personas per scenario	5	5
Task topology weights
Tier 1	0.50	0.10
Tier 2	0.30	0.40

Appendix

Table 7: Generation parameter settings for configurations C1 and C2. Both use the same 7-tool interface and domain seed. Capability axis weights are held at defaults across both configurations. The configurable ranges within individual task topology patterns are also constant across C1 and C2. Only topology tier weights, obfuscation, and distractor budgets differ.

	Expert authoring	AutoGym (mid-tier)	AutoGym (frontier)
Cost	$500–$ 1000	$2-5	$3-8
Wall-clock time	1-1.5 hours	15.6 min	15.6 min

Appendix

Table 8: Per-task cost

	Expert authoring	AutoGym (mid-tier)	AutoGym (frontier)
Cost	$25K–$ 50K	$100–150	$150–200
Wall-clock time	2–4 weeks	54 minutes	54 minutes

Appendix

Table 9: Gym-level cost (50 tasks, 20 parallelism)

Figure 2: Training reward over 500 RL steps on the instruction-following gym. The steady improvement confirms the generated tasks provide learnable signal.

Synthetic Benchmark Generation

Synthetic benchmark generation creates controlled, verifiable evaluation tasks and datasets to test models across diverse settings.

See all 287 papers

SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation
Sep 29, 2026Renxi Wang, Mingshan Hee, Fajri Koto +2Supervised Fine-Tuning LLM Agent Skill Learning
Mohamed bin Zayed University of Artificial Intelligence
CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
May 25, 2026Bowen Wang, Dunjie Lu, Junli Wang +11Computer-Use Agents Synthetic Data Generation
The University of Hong Kong · Qwen Team, Alibaba Inc. · University of California, San Diego +1
Gym-Anything: Turn any Software into an Agent Environment
Apr 7, 2026Pranjal Aggarwal, Graham Neubig, Sean WelleckComputer-Use Agents Computer-Use Agent Benchmarks
CMU

Sep 29, 2026J/K move · Enter open · S save

SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation

Renxi Wang, Mingshan Hee, Fajri Koto, Timothy Baldwin+1

Mohamed bin Zayed University of Artificial Intelligence

Supervised Fine-Tuning LLM Agent Skill Learning LLM Agent Trajectory Synthesis LLM Agent Training

Open paper AskPDF arXiv

Skills equip LLM agents with professional knowledge and guidance to complete long-horizon and complex tasks. Although skills have been widely adopted in recent agent paradigms and harnesses, how to synthesize reliable training data and how to train agents for skill use remain underexplored. In this work, we propose SkillGym, an automatic pipeline to build verifiable environments, collect trajectories, and train skill-use agents. SkillGym first crawls a large volume of skills from the internet, then keeps those whose workflows can run reproducibly offline. A builder-reviewer pipeline is used to construct difficulty-controlled tasks, spanning four task types, each with a reference solution and an executable verifier. With this pipeline, we build 6.8k environments and collect 19k verified successful trajectories for supervised finetuning. Finetuning on these trajectories improves LLMs of different families and sizes, from 2B to 122B parameters across four skill-use benchmarks; Our Qwen3.5-9B SFT model outperforms the 397B untrained model on two of them. Further analysis shows that training teaches agents to invoke skills, raising the rate of reading the relevant skill from 28% to 96%, and that the gains hold across reasoning structures, extending to task types that form a minority of the training data and to skills held out from training

Abstract

Synthetic Benchmark Generation

Explore similar work

SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation

CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

Gym-Anything: Turn any Software into an Agent Environment