cs.AIOct 27, 2026
SaveWorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training
Organizations: IQuest Research · Zhejiang University · Gaoling School of Artificial Intelligence, Renmin University of China · Shenzhen Institutes of Advanced Technology ·
Abstract
Scaling supervised training for working agents requires diverse professional tasks that yield reliable expert demonstrations. However, missing or inconsistent task information can obstruct teacher execution and weaken the resulting supervision. We propose WorkFactory, a scalable framework for constructing coherent and executable professional tasks. Starting from professional intents, information-level planning builds an information support graph to guide task instantiation, explicitly representing the dependencies and locations of information required for the intended outputs. Probe executions then identify support failures and guide repairs while preserving the intended work. We construct 112,686 tasks spanning 612 occupations and use the collected teacher trajectories to fine-tune Qwen models at 4B, 27B, and 35B-A3B scales, yielding the WorkPilot family. All three models improve over their respective base models across four professional-work benchmarks, with WorkPilot-35B-A3B improving from 879 to 1565 Elo on GDPval-AA v2.1. Scaling analyses reveal model-dependent diminishing returns, with execution length first increasing and then decreasing. Controlled component analyses show that planning and repair reduce probe-diagnosed task environment defects and improve downstream performance. The data and models are available at https://github.com/IQuestLab/WorkFactory.
Figures & tables
Figure 1: WorkPilot-35B-A3B consistently outperforms its Qwen3.6-35B-A3B base model across four professional-work benchmarks, with representative models shown for context.
Figure 2: Overview of WorkFactory through a synthetic shipping-carton sourcing example. (1) An information support graph guides environment construction while leaving derived outputs to the solver. (2) Clean probe executions expose blocking issues and guide checked repairs before retained environments are used for independent teacher trajectory collection. Appendix C.4 provides the complete end-to-end walkthrough.
Figure 3: Selected high-coverage occupations from four of the 23 major occupational groups represented in the WorkFactory corpus.
| Model | GDPval-AA | JobBench | Workspace-Bench | OmegaUse-OfficeVal |
| Frontier Models | ||||
| GLM-5.3 | 1646 * | 66.35 | 61.80 | 31.09 |
| DeepSeek-V4.1-Flash | 1600 * | 67.69 | 63.06 | 31.43 |
| Qwen3.8-Max | 1596 * | 59.83 | 61.51 | 28.23 |
| GPT-5.6 Sol | 1588 * | 50.91 | 53.77 | 19.25 |
| Kimi-K3 | 1524 * | 55.83 | 54.05 | 23.09 |
Table 2: Results on professional-task benchmarks. For GDPval-AA, marks an official v2.1 Elo rating; unmarked values are local two-anchor estimates, with task-bootstrap 95% confidence intervals reported in Appendix Table 11 .
Figure 5: Scaling on GDPval with the number of WorkFactory training tasks. (a) GDPval-AA Elo across model scales. (b) Task-level mean execution length. (c) Tool-call failure rates for Qwen3.6-35B-A3B. Shading denotes 95% confidence intervals; the -axis uses a square-root scale.
| Artifact-level planning | + Information support graph | vs. left | + Execution-based repair | vs. left | |
| Probe-diagnosed hard-defect incidence | |||||
| Missing evidence | 481 (9.62%) | 182 (3.64%) | 5.98 pp | 101 (2.02%) | 1.62 pp |
| Cross-file inconsistency | 1,641 (32.82%) | 1,287 (25.74%) | 7.08 pp | 242 (4.84%) | 20.90 pp |
| Broken derivation | 1,243 (24.86%) | 343 (6.86%) | 18.00 pp | 163 (3.26%) | 3.60 pp |
| Underspecified request | 361 (7.22%) | 31 (0.62%) | 6.60 pp | 14 (0.28%) | 0.34 pp |
| Leakage or shortcut | 96 (1.92%) | 168 (3.36%) | +1.44 pp | 42 (0.84%) | 2.52 pp |
Table 3: Component analysis across three 5K-task synthesis conditions. Defect entries report task counts (percentages), with categories allowed to overlap. Deltas compare adjacent conditions in percentage points for defects and GDPval-AA Elo points for downstream utility.
Figure 6: GDPval-AA Elo versus standardized estimates of model-inference cost per task. The line marks the non-dominated frontier among the models shown.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Node role | Meaning | Example |
| Facts / evidence | Records and observations | Demand forecasts and supplier quotations |
| Rules / definitions | Policies and conventions | Setup, freight, and discount rules |
| Intermediate | Values to derive | Carton volumes, annual spend, and vendor NPVs |
| Deliverable | Requested work product | Cost-comparison workbook and recommendation |
| Requirements | Goals and constraints | Compare lifecycle cost and vendor service |
Table 4: Information-node roles.
| Information unit | Carrier / status | Support relation |
| Lifecycle-cost analysis and recommendation requirements | User request; input | Specify the analysis and workbook to produce. |
| Annual carton demand and standard/insulated mix | Quotation and forecast PDF; input | Determine annual carton volumes by type. |
| Supplier unit prices, setup fees, and freight terms | Quotation and forecast PDF; input | Supply vendor-specific cost inputs. |
| Discount and cost-timing rules | User request; input | Place setup fees in year 0 and discount years 1–3 at 8%. |
| Vendor quality, lead time, and capacity | Request and quotation; input | Constrain and contextualize the sourcing decision. |
| Carton volumes, year-0 setup, annual spend, and vendor NPVs | Cost-comparison workbook; to be derived | Transform demand, quotations, and rules into comparable lifecycle costs. |
Table 5: Information support for the synthetic shipping-carton sourcing example. Inputs are supplied in the request and reference PDF; carton volumes, spending, NPVs, and the recommendation remain solver work.
| Category | Definition |
| Missing evidence | Required information is absent from all solver-accessible sources. |
| Cross-file inconsistency | Independent sources disagree on entities, values, periods, units, or definitions. |
| Broken derivation | A requested result cannot be derived from the supplied premises or rules. |
| Underspecified request | The request omits a decision needed to determine a valid deliverable. |
| Leakage or shortcut | Inputs reveal an answer or internal evaluation signal that should require work. |
| Execution feasibility | A hard constraint, corrupt artifact, or unavailable tool makes completion impossible. |
Table 6: Defect categories assigned retrospectively to issues in execution-based quality reports.
| Repair depth | ||
| Rounds | Tasks | Share |
| 0 | 3,479 | 69.58% |
| 1 | 1,012 | 20.24% |
| 2 | 385 | 7.70% |
| 3 (limit) | 124 | 2.48% |
| Paired probe execution statistics | ||
Table 7: Repair depth and paired probe-execution statistics for the execution-based repair condition on the 5K-task analysis set.
| Synthesis stage | Input | Output | Total |
| Information-level planning | 4.658 | 0.079 | 4.737 |
| Material synthesis | 3.219 | 0.056 | 3.275 |
| Execution and repair | 3.571 | 0.073 | 3.644 |
| Total | 11.448 | 0.208 | 11.656 |
Table 8: Mean model-token usage per task by construction stage. Token counts are in millions.
Figure 7: Distribution of assistant turns per teacher trajectory in the WorkFactory corpus.
| Setting | Value |
| Probe execution | |
| Probe model | DeepSeek-V4-Flash-0731 |
| Tool interface | Claude Code tools |
| Context window | 1M tokens |
| Context compaction | Disabled |
| Teacher trajectory generation | |
Table 9: Probe execution, teacher trajectory generation, and shared WorkPilot training settings.
Figure 8: Teacher and student tool-use behavior. The top row uses teacher trajectories on the matched 30K-task training subset; the bottom row uses student rollouts on the same 220 GDPval evaluation tasks. (a,d) Empirical cumulative distributions of task-level mean assistant turns. (b,e) Tool-call composition, obtained by summing per-task mean call counts and normalizing across tool types; Other includes remaining tools. (c,f) Task-macro failure rates. For student results, the three evaluation rollouts are averaged within each task for turns and call composition; for failure rates, failures and calls from the three rollouts are pooled within a task before averaging rates over tasks invoking that tool. GLM denotes GLM-5.3 in the teacher row and its trained student in the student row; DeepSeek analogously denotes DeepSeek-V4-Flash-0731 and its student.
| Metric | GLM source | DeepSeek source |
| Teacher behavior on the 30K subset | ||
| Mean assistant turns | 60.69 | 37.97 |
| Mean tool calls | 65.76 | 44.92 |
| Mean reasoning tokens | 48,127 | 48,368 |
| Student evaluation on 220 shared tasks | ||
| GDPval-AA Elo | 1483 | 1248 |
Table 10: Trajectory and student-behavior statistics for the 30K-task teacher comparison. Student behavioral statistics are task-level means over the same 220 GDPval tasks after averaging three independent evaluation rollouts within each task. All token counts use the Qwen3.6-35B-A3B tokenizer. GLM denotes GLM-5.3; DeepSeek denotes DeepSeek-V4-Flash-0731.
Figure 9: Distribution of interaction turns for Qwen3.6-35B-A3B and WorkPilot-35B-A3B across four professional-work benchmarks. Histograms show within-model sample proportions; dashed lines and numeric labels denote mean interaction lengths.
| Model | Local Elo | Bootstrap 95% CI |
| Nex-N2.5-mini | 1127 | |
| Qwen3.5-4B | 661 | |
| WorkPilot-4B | 1375 | |
| Qwen3.6-27B | 975 | |
| WorkPilot-35B-A3B | 1565 | |
| WorkPilot-27B | 1612 |
Table 11: Task-bootstrap 95% confidence intervals for the locally estimated GDPval-AA v2.1 Elo ratings in Table 2 .
Figure 10: Predicted versus official GDPval-AA Elo. Blue circles show the seven-model validation set used to determine the estimator design and calibration; purple squares show the three-model test set evaluated after the procedure was frozen. Predicted Elo is on the horizontal axis and official Elo is on the vertical axis. The dashed line denotes equality and the shaded region denotes Elo. The independently evaluated GLM-5.3 rerun is distinct from the GLM-5.3 calibration node.
| Pricing basis | OpenRouter endpoint / provider | Input | Cache | Output | Models assigned to this basis and mapping rule |
| WorkPilot-4B (scaled Nex-N2.5-mini) | nex-agi/nex-n2.5-mini ; Nex AGI | 0.00625 | 0.000625 | 0.025 | WorkPilot-4B; each Nex rate is multiplied by . |
| Nex-N2.5-mini | nex-agi/nex-n2.5-mini ; Nex AGI | 0.025 | 0.0025 | 0.10 | WorkPilot-35B-A3B, Nex-N2.5-mini, Qwen3.6-35B-A3B, and Apodex-1.1-mini; same-size alignment. |
| Qwen3.8-27B | qwen/qwen3.8-27b ; Wafer | 0.04 | 0.0198 | 2.30 | WorkPilot-27B, Qwen3.8-27B, and Qwen3.6-27B; same-size alignment. |
| Qwen3.8-Max-0902 | qwen/qwen3.8-max-0902 ; Alibaba Cloud Int. | 2.00 | 0.25 | 6.00 | Qwen3.8-Max; current endpoint used as a version proxy. |
| GLM-5.3 | z-ai/glm-5.3 ; Wafer | 0.07 | 0.05 | 4.80 | GLM-5.3; direct endpoint. |
| GLM-5.2 | z-ai/glm-5.2 ; Wafer | 0.06 | 0.059 | 7.00 | GLM-5.2; direct endpoint. |
Table 12: Complete pricing assignment for the GDPval-AA cost–performance analysis. Input, cache-read, and output prices are USD per one million tokens, using the October 12, 2026 snapshot.
More in this topic
LLM Agent Training
Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.
See all 586 papers