Data World: Scaling Data Environments from Scratch

Zhanli Li; Zeyuan Xu; Chaoxu Pang; Zeyuan Liu; Anyang Ji; Te Sun; Kongcheng Zhang; Wenbo Xu

cs.AIOct 15, 2026

Data World: Scaling Data Environments from Scratch

Authors: Zhanli Li, Zeyuan Xu, Chaoxu Pang, Zeyuan Liu, Anyang Ji, Te Sun, Kongcheng Zhang, Wenbo Xu,

Organizations: Longcat Interaction Team, Meituan · Zhongnan University of Economics and Law · Hunan University · School of Artificial Intelligence, Nanjing University ·

Abstract

Data agents use code to process heterogeneous files and derive analytical insights. Scaling their training faces costly environment curation, privacy and intellectual-property restrictions, and difficulty verifying open-ended findings without known generative truth. Our core insight is to specify the world behind the files before generating the files themselves. We introduce Data World, a multi-agent engine that synthesizes environments, tasks, and rubrics from explicit mechanisms. Independent audits check target recoverability from observed files; rubrics support evaluation and RL rewards. Human preference-based curation for question realism selects 100 tasks across 50 held-out worlds and 16 domains for DataWorld-Bench. The engine also yields 2,135 training tasks from 1,037 disjoint worlds. Evaluation of ten frontier models shows that the benchmark remains challenging, particularly for insight discovery, with a consistent computation-insight gap. RL-zero training on the synthesized tasks produces DataWorld-35B-A3B from Qwen3.6-35B-A3B without trajectory distillation. Under matched evaluation, it improves over its base by 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard, and 12.13 on JobBench, a broader occupational benchmark.

Figures & tables

Figure 1: Supervision for data agents: exploration-based synthesis (top) versus Data World (bottom). Exploring existing files may leave valid findings without matching criteria; Data World specifies mechanisms first, aligning files and rubrics with analytical targets.

Figure 2: Data World synthesis and evaluation pipeline. A dependency DAG and world contract guide layer-wise canonical-state generation; observation projection produces analyst-visible files. An independent auditor checks relationships, invariants, and target recoverability. The generative record grounds tasks and rubrics; the data agent solves tasks from observed files, and a separate evaluator grades its report.

Figure 3: Taxonomy shares (%) of the DataWorld-Bench test tasks and Data World RL training tasks.

	Score			Interaction and length
Model	Data computation	Insight discovery	Avg	Rounds	Total tokens (k)	Resp. tokens (k)
GPT-6-Astra	$\mathbf{87.80}$ (16.38)	$\underline{66.45}$ (20.98)	$\mathbf{74.43}$ (15.85)	$28.7$ (8.7)	$121.9$ k (36.4)	$88.8$ k (18.9)
Qwen-3.8-Max-0902	$82.36$ (17.79)	$\mathbf{67.95}$ (22.92)	$\underline{73.43}$ (16.81)	$82.7$ (23.5)	$289.7$ k (81.9)	$125.8$ k (33.0)
GPT-6.1-Sol	$\underline{85.59}$ (17.26)	$66.02$ (21.02)	$73.30$ (16.44)	$31.2$ (10.6)	$135.2$ k (46.3)	$103.4$ k (30.4)
GLM 5.3	$75.08$ (21.22)	$63.18$ (21.56)	$67.69$ (17.67)	$89.9$ (28.0)	$238.7$ k (96.6)	$116.3$ k (49.4)
HY4-Preview	$77.27$ (20.63)	$60.00$ (21.38)	$66.55$ (17.72)	$70.7$ (21.3)	$177.9$ k (49.2)	$86.0$ k (23.3)

Table 1: DataWorld-Bench results (normalized rubric scores

\times 100

, with SD in parentheses). Avg pools all rubric items and need not equal the mean of the two section columns. Best/second-best means are bold / underlined . Rounds count model calls; tokens are prompt plus output of the final call (total) or outputs over the run (response).

	Neg. rubric items (%)
Model	0	1	2
GPT-6.1-Sol	86	11	2
GPT-6-Astra	83	14	2
HY4-Preview	74	20	5
Kimi-K3	70	20	9
Qwen-3.8-Max-0902	68	26	5

Table 2: Distribution of triggered negative rubric items across trajectories. Each column gives the percentage of trajectories triggering exactly that many negative items; 3+ is omitted.

	JobBench			DAComp			DABstep-Hard			DataWorld-Bench
Model	Score	Rounds	Resp. tokens (k)	Score	Rounds	Resp. tokens (k)	Score	Rounds	Resp. tokens (k)	Score	Rounds	Resp. tokens (k)
Qwen3.6-35B-A3B	$18.31$ (18.12)	$38.7$ (17.4)	$60.4$ k (22.1)	$+53.72$ (20.35)	$35.6$ (17.7)	$41.8$ k (19.6)	$+25.66$ (43.73)	$28.6$ (18.0)	$15.1$ k (15.3)	$37.64$ (11.62)	$36.5$ (13.9)	$38.4$ k (19.2)
DataMind-35B-A3B	$22.93$ (22.33) (+4.63)	$36.6$ (12.7)	$73.9$ k (28.0)	$+54.79$ (20.49) (+1.07)	$37.6$ (14.1)	$47.7$ k (19.0)	$+31.22$ (46.40) (+5.56)	$40.3$ (17.6)	$26.3$ k (16.8)	$37.36$ (11.14) (-0.29)	$34.4$ (9.6)	$42.0$ k (17.2)
BigBang-v1	$20.84$ (20.69) (+2.53)	$68.9$ (23.4)	$84.3$ k (30.1)	$+54.31$ (20.93) (+0.58)	$45.1$ (22.9)	$38.2$ k (18.4)	$+34.66$ (47.65) (+8.99)	$37.2$ (24.5)	$35.5$ k (34.0)	$40.05$ (12.06) (+2.41)	$58.4$ (27.8)	$46.7$ k (18.1)
Occamy-1.0	$30.18$ (23.79) (+11.87)	$79.2$ (41.7)	$96.4$ k (38.9)	$+64.86$ (20.89) (+11.13)	$49.1$ (22.5)	$45.4$ k (21.6)	$+44.44$ (49.76) (+18.78)	$31.0$ (21.3)	$26.3$ k (28.4)	$42.38$ (14.06) (+4.74)	$61.5$ (25.8)	$58.2$ k (28.3)
DataWorld-35B-A3B	$30.43$ (23.92) (+12.13)	$63.6$ (28.9)	$88.1$ k (29.1)	$+68.00$ (18.90) (+14.28)	$78.4$ (25.0)	$65.9$ k (21.7)	$+46.56$ (49.95) (+20.90)	$43.2$ (22.3)	$36.3$ k (28.0)	$44.70$ (13.94) (+7.06)	$102.8$ (22.6)	$68.4$ k (17.6)

Table 3: RL training utility: DataWorld-35B-A3B paired per task against its base Qwen3.6-35B-A3B (mean with SD in parentheses). Green/red badges indicate gains/losses relative to the base.

Model	MMLU-Pro	LCB-Hard	SimpleQA-V
Qwen3.6-35B-A3B	84.73	71.14	22.20
DataWorld-35B-A3B	84.73	70.29	24.00

Table 4: General-capability and factuality checks for DataWorld-35B-A3B versus Qwen3.6-35B-A3B. MMLU-Pro and LiveCodeBench (LCB)-Hard report accuracy (%); SimpleQA-Verified (SimpleQA-V) reports F1. Higher is better.

Figure 4: Performance as RL training scales over synthesized tasks. Base is the pre-RL model; scores are unscaled, with final gains over Base annotated in points and, where applicable, relative percentages.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

	Taxonomy category
Model	Com	Cus	Fin	Ops	HR	Risk	Clin	Epi	Bio	Sci	Env	Enrg	Pol	Edu	Soc	Tran	Avg
GPT-6-Astra	69.96	77.25	79.79	84.91	65.68	73.12	71.73	70.54	83.31	85.53	64.22	77.27	68.46	72.65	77.60	69.77	74.43
Qwen-3.8-Max-0902	78.58	81.23	73.15	80.35	77.81	65.23	76.70	73.68	76.08	77.98	60.82	88.12	68.99	62.23	76.15	58.28	73.43
GPT-6.1-Sol	76.20	77.60	79.59	76.93	57.45	69.65	74.14	69.81	74.41	92.71	66.50	80.69	68.25	67.10	75.00	67.90	73.30
GLM 5.3	71.95	69.50	71.62	75.45	73.97	69.53	62.54	68.52	78.43	68.81	50.27	75.91	64.35	58.02	75.16	47.68	67.69
HY4-Preview	67.65	64.90	76.25	73.26	65.97	54.39	63.98	62.10	70.51	68.05	65.86	79.36	62.50	59.23	75.29	53.88	66.55

Appendix

Table 5: Per-domain scores, following Table 1 . Avg pools all 100 test tasks. The bottom row averages domain offsets after removing each model’s mean. Each domain has only

6

–

8

tasks, so individual cells should not be used to rank models. Domain abbreviations are defined in Section A.3.3 .

	Score ( $\times 100$ )		Paired cause $-$ consequence test
Model	Cause	Consequence	$\Delta$	MDE
GPT-6-Astra	$76.32$	$72.54$	$+3.78$	$9.30$
Qwen-3.8-Max-0902	$75.32$	$71.53$	$+3.80$	$8.79$
GPT-6.1-Sol	$73.33$	$73.27$	$+0.07$	$8.87$
GLM 5.3	$66.05$	$69.33$	$-3.28$	$9.06$
HY4-Preview	$68.02$	$65.08$	$+2.94$	$9.81$

Appendix

Table 6: Cause versus consequence scores (50 tasks each), following Table 1 .

\Delta

= cause

-

consequence. MDE uses same-world pairs. No model’s

\Delta

reaches its MDE.

Setting	Value
Training / rollout GPUs	48 / 64 H20-3e GPUs (6 / 8 eight-GPU nodes)
Training parallelism	TP 2, CP 4, PP 1, EP 8, ETP 1
Derived data parallelism	DP $=48/(2\times 4\times 1)=6$
Rollout engines	64 one-GPU SGLang engines (TP 1)
Rollout batch size	24 prompts
Samples per prompt	8

Appendix

Table 7: RL training resources and settings for the training and rollout workers.

Synthetic Data Generation for LLM Agents

See all 28 papers

Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
Apr 20, 2026Guanting Dong, Junting Lu, Junjie Huang +17Lifelong Learning Agents AI Agent Benchmarks
Renmin University of China · ByteDance Seed
CompoWorld: Compositional Environment Scaling for General Agents
Sep 27, 2026Xiao-Wen Yang, Weiyi Xu, Wen Da +9Tool-Augmented Language Model Agents Tool-Using Agents
AllSpark Team
Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
May 20, 2026Zihao Cheng, Hongru Wang, Zeming Liu +6Tool-Augmented Language Model Agents Terminal Agents
School of Computer Science and Engineering, Beihang University, Beijing, China · Independent Researcher · 3Beijing Institute of Technology +2

Apr 20, 2026J/K move · Enter open · S save

Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence

Guanting Dong, Junting Lu, Junjie Huang, Wanjun Zhong+16

Renmin University of China · ByteDance Seed

Lifelong Learning Agents AI Agent Benchmarks LLM Agent Training RL for LLM Agents Self-Evolving Agents

Open paper AskPDF arXiv

Large language models are increasingly expected to serve as general-purpose agents that interact with external, stateful tool environments. The Model Context Protocol (MCP) and broader agent skills offer a unified interface for connecting agents with scalable real-world services, but training robust agents remains limited by the lack of realistic environments and principled mechanisms for life-long learning. In this paper, we present Agent-World, a self-evolving training arena for advancing general agent intelligence through scalable environments. Agent-World has two main components: (1) Agentic Environment-Task Discovery, which autonomously explores topic-aligned databases and executable tool ecosystems from thousands of real-world environment themes and synthesizes verifiable tasks with controllable difficulty; and (2) Continuous Self-Evolving Agent Training, which combines multi-environment reinforcement learning with a self-evolving agent arena that automatically identifies capability gaps through dynamic task synthesis and drives targeted learning, enabling the co-evolution of agent policies and environments. Across 23 challenging agent benchmarks, Agent-World-8B and 14B consistently outperforms strong proprietary models and environment scaling baselines. Further analyses reveal scaling trends in relation to environment diversity and self-evolution rounds, offering insights for building general agent intelligence.

Abstract

Synthetic Data Generation for LLM Agents

Explore similar work

Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence

CompoWorld: Compositional Environment Scaling for General Agents

Terminal-World: Scaling Terminal-Agent Environments via Agent Skills