WorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training
Organizations: IQuest Research · Zhejiang University · Gaoling School of Artificial Intelligence, Renmin University of China · Shenzhen Institutes of Advanced Technology ·
Abstract
Scaling supervised training for working agents requires diverse professional tasks that yield reliable expert demonstrations. However, missing or inconsistent task information can obstruct teacher execution and weaken the resulting supervision. We propose WorkFactory, a scalable framework for constructing coherent and executable professional tasks. Starting from professional intents, information-level planning builds an information support graph to guide task instantiation, explicitly representing the dependencies and locations of information required for the intended outputs. Probe executions then identify support failures and guide repairs while preserving the intended work. We construct 112,686 tasks spanning 612 occupations and use the collected teacher trajectories to fine-tune Qwen models at 4B, 27B, and 35B-A3B scales, yielding the WorkPilot family. All three models improve over their respective base models across four professional-work benchmarks, with WorkPilot-35B-A3B improving from 879 to 1565 Elo on GDPval-AA v2.1. Scaling analyses reveal model-dependent diminishing returns, with execution length first increasing and then decreasing. Controlled component analyses show that planning and repair reduce probe-diagnosed task environment defects and improve downstream performance. The data and models are available at https://github.com/IQuestLab/WorkFactory.
Figures & tables
| Model | GDPval-AA | JobBench | Workspace-Bench | OmegaUse-OfficeVal |
| Frontier Models | ||||
| GLM-5.3 | 1646 * | 66.35 | 61.80 | 31.09 |
| DeepSeek-V4.1-Flash | 1600 * | 67.69 | 63.06 | 31.43 |
| Qwen3.8-Max | 1596 * | 59.83 | 61.51 | 28.23 |
| GPT-5.6 Sol | 1588 * | 50.91 | 53.77 | 19.25 |
| Kimi-K3 | 1524 * | 55.83 | 54.05 | 23.09 |
| Artifact-level planning | + Information support graph | vs. left | + Execution-based repair | vs. left | |
| Probe-diagnosed hard-defect incidence | |||||
| Missing evidence | 481 (9.62%) | 182 (3.64%) | 5.98 pp | 101 (2.02%) | 1.62 pp |
| Cross-file inconsistency | 1,641 (32.82%) | 1,287 (25.74%) | 7.08 pp | 242 (4.84%) | 20.90 pp |
| Broken derivation | 1,243 (24.86%) | 343 (6.86%) | 18.00 pp | 163 (3.26%) | 3.60 pp |
| Underspecified request | 361 (7.22%) | 31 (0.62%) | 6.60 pp | 14 (0.28%) | 0.34 pp |
| Leakage or shortcut | 96 (1.92%) | 168 (3.36%) | +1.44 pp | 42 (0.84%) | 2.52 pp |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Node role | Meaning | Example |
| Facts / evidence | Records and observations | Demand forecasts and supplier quotations |
| Rules / definitions | Policies and conventions | Setup, freight, and discount rules |
| Intermediate | Values to derive | Carton volumes, annual spend, and vendor NPVs |
| Deliverable | Requested work product | Cost-comparison workbook and recommendation |
| Requirements | Goals and constraints | Compare lifecycle cost and vendor service |
| Information unit | Carrier / status | Support relation |
| Lifecycle-cost analysis and recommendation requirements | User request; input | Specify the analysis and workbook to produce. |
| Annual carton demand and standard/insulated mix | Quotation and forecast PDF; input | Determine annual carton volumes by type. |
| Supplier unit prices, setup fees, and freight terms | Quotation and forecast PDF; input | Supply vendor-specific cost inputs. |
| Discount and cost-timing rules | User request; input | Place setup fees in year 0 and discount years 1–3 at 8%. |
| Vendor quality, lead time, and capacity | Request and quotation; input | Constrain and contextualize the sourcing decision. |
| Carton volumes, year-0 setup, annual spend, and vendor NPVs | Cost-comparison workbook; to be derived | Transform demand, quotations, and rules into comparable lifecycle costs. |
| Category | Definition |
| Missing evidence | Required information is absent from all solver-accessible sources. |
| Cross-file inconsistency | Independent sources disagree on entities, values, periods, units, or definitions. |
| Broken derivation | A requested result cannot be derived from the supplied premises or rules. |
| Underspecified request | The request omits a decision needed to determine a valid deliverable. |
| Leakage or shortcut | Inputs reveal an answer or internal evaluation signal that should require work. |
| Execution feasibility | A hard constraint, corrupt artifact, or unavailable tool makes completion impossible. |
| Repair depth | ||
| Rounds | Tasks | Share |
| 0 | 3,479 | 69.58% |
| 1 | 1,012 | 20.24% |
| 2 | 385 | 7.70% |
| 3 (limit) | 124 | 2.48% |
| Paired probe execution statistics | ||
| Synthesis stage | Input | Output | Total |
| Information-level planning | 4.658 | 0.079 | 4.737 |
| Material synthesis | 3.219 | 0.056 | 3.275 |
| Execution and repair | 3.571 | 0.073 | 3.644 |
| Total | 11.448 | 0.208 | 11.656 |
| Setting | Value |
| Probe execution | |
| Probe model | DeepSeek-V4-Flash-0731 |
| Tool interface | Claude Code tools |
| Context window | 1M tokens |
| Context compaction | Disabled |
| Teacher trajectory generation | |
| Metric | GLM source | DeepSeek source |
| Teacher behavior on the 30K subset | ||
| Mean assistant turns | 60.69 | 37.97 |
| Mean tool calls | 65.76 | 44.92 |
| Mean reasoning tokens | 48,127 | 48,368 |
| Student evaluation on 220 shared tasks | ||
| GDPval-AA Elo | 1483 | 1248 |
| Model | Local Elo | Bootstrap 95% CI |
| Nex-N2.5-mini | 1127 | |
| Qwen3.5-4B | 661 | |
| WorkPilot-4B | 1375 | |
| Qwen3.6-27B | 975 | |
| WorkPilot-35B-A3B | 1565 | |
| WorkPilot-27B | 1612 |
| Pricing basis | OpenRouter endpoint / provider | Input | Cache | Output | Models assigned to this basis and mapping rule |
| WorkPilot-4B (scaled Nex-N2.5-mini) | nex-agi/nex-n2.5-mini ; Nex AGI | 0.00625 | 0.000625 | 0.025 | WorkPilot-4B; each Nex rate is multiplied by . |
| Nex-N2.5-mini | nex-agi/nex-n2.5-mini ; Nex AGI | 0.025 | 0.0025 | 0.10 | WorkPilot-35B-A3B, Nex-N2.5-mini, Qwen3.6-35B-A3B, and Apodex-1.1-mini; same-size alignment. |
| Qwen3.8-27B | qwen/qwen3.8-27b ; Wafer | 0.04 | 0.0198 | 2.30 | WorkPilot-27B, Qwen3.8-27B, and Qwen3.6-27B; same-size alignment. |
| Qwen3.8-Max-0902 | qwen/qwen3.8-max-0902 ; Alibaba Cloud Int. | 2.00 | 0.25 | 6.00 | Qwen3.8-Max; current endpoint used as a version proxy. |
| GLM-5.3 | z-ai/glm-5.3 ; Wafer | 0.07 | 0.05 | 4.80 | GLM-5.3; direct endpoint. |
| GLM-5.2 | z-ai/glm-5.2 ; Wafer | 0.06 | 0.059 | 7.00 | GLM-5.2; direct endpoint. |
LLM Agent Training
Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.
See all 586 papers