DataPraxis: Scaling Analytics Tasks via Verified Workspace Evolution

Nan Jia; Wenxuan Shi; Jingqing Ruan; Zhizhao Zeng; Tianyi Zhai; Zhongda Su; Feng Hong; Zhengyu Chen

cs.AIOct 17, 2026

DataPraxis: Scaling Analytics Tasks via Verified Workspace Evolution

Authors: Nan Jia, Wenxuan Shi, Jingqing Ruan, Zhizhao Zeng, Tianyi Zhai, Zhongda Su, Feng Hong, Zhengyu Chen,

Organizations: Huazhong University of Science and Technology · Institute of Computing Technology, Chinese Academy of Sciences · Meituan · Tsinghua University ·

Abstract

General-purpose agents are increasingly expected to perform professional analytics work, which requires sustained reasoning, iterative investigation, and reliable evidence. Training these capabilities requires diverse and challenging tasks. However, generating tasks directly from raw data often overlooks deeper questions that emerge through analysis and provides limited evidence for verification. To address these limitations, we introduce DataPraxis, an analytics task synthesis framework based on verified workspace evolution. The framework progressively enriches workspaces with validated findings and artifacts, then inherits and combines these workspaces to uncover deeper questions. It constructs tasks from accumulated findings and their dependencies, verifying analytical results both during evolution and after task assembly. Using this framework, we construct DataPraxis-5K, a dataset of 5,000 verified trajectories spanning diverse domains and output formats, to train DataPraxis-27B and DataPraxis-35B-A3B. DataPraxis-27B achieves 55.11 on DABStep and 51.9 on JobBench, matching or surpassing frontier models such as GLM-5.2. DataPraxis-5K is available at huggingface.co/datasets/stargazerhust/DataPraxis-5k

Figures & tables

Figure 1: Up: Tasks are generated from the initial materials without workspace evolution. Down: DataPraxis inherits and combines verified analytical results through workspace evolution, providing new grounds for discovering deeper questions.

Figure 2: The DataPraxis framework. (a) The Profiler characterizes the initial data and domain. (b) Verified workspace evolution progressively discovers and validates analytical findings, which are converted into standalone tasks through backward synthesis. (c) Dual-evidence verification combines evidence from evolution with independent solving trajectories.

Figure 3: Question perplexity with initial workspace information alone or with accumulated findings.

Mode	Sources $\to$ Tasks	Description
deepen	$1\to 1$	Extend existing findings through further analysis.
search	$1\to 1$	Retrieve and verify relevant external materials.
decompose	$1\to k$	Split a compound objective into independent subtasks.
compose	$k\to 1$	Merge verified findings from complementary workspaces.

Table 1: Workspace evolution modes. The mapping

s\to t

denotes

s

source workspaces and

t

analytical directions, with

k\geq 2

Model / data	DABStep			JobBench	RideBench	Avg
	Easy	Hard	Overall	JobBench	RideBench	Avg
Claude Opus 5	87.50 †	48.68 †	54.89	69.50	82.02	68.80
Kimi-K3	84.72	69.58	72.00	57.20	79.78	69.66
GLM-5.2	87.50	49.47	55.55	51.20	78.65	61.80
DeepSeek-V4-Pro	84.72	44.71	51.11	49.20	75.28	58.53
GPT-5.6-Terra	83.33	45.50	51.55	33.80	68.54	51.30

Table 2: Performance on data analysis benchmarks. Scores marked with

\dagger

are taken from the DABStep leaderboard.

Figure 6

Figure 5: Teacher ablation for Qwen3.6-27B using 5K training trajectories per teacher. Benchmark values denote student scores, with the best in bold.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Workspace evolution in hotel pricing analysis. A–C denote selected evolution nodes. Arrows show workspace inheritance and subsequent task assembly. Descriptive filenames are used for readability.

Stage	Selected criterion and supporting evidence
Candidate	Report both co-occurrence frequency and lift. Identify Bakery + Coffee as frequent but with lift below one, and Bakery + Tea as having lift above one.
Validation	Bakery + Tea appears in 7,983 baskets. Its lift is 0.920 among all 116,790 baskets and 1.197 among 28,904 baskets containing multiple categories. Bakery + Coffee has lift below one under both populations (0.951 and 0.857).
Final criterion	Report both frequency and lift, recognize that Bakery + Coffee is frequent without a positive association, and identify at least one category pair with lift above one. The required positive pair is no longer fixed to Bakery + Tea.

Appendix

Table 4: Rubric refinement for coffee basket analysis. Criterion text is condensed and translated from the recorded rubric.

Figure 8: Task organization and output inspection on JobBench. (a) Number of tasks with explicit subgoals, expressed as a percentage of all 65 matched tasks. Final verification goals are excluded. (b) Mean output inspection attempts per 100 nonmanagement tool calls, with 95% bootstrap intervals over tasks. Output inspection is an observable proxy for self-verification. Both panels compare Qwen3.6-27B with DataPraxis-27B on the same task set.

Figure 9: Paired cases illustrating the two aggregate behavior patterns. (a) Explicit subgoals organize VPN incident analysis and its deliverables. (b) Inspecting workbook contents reveals an omission and guides its repair. Text summarizes recorded operations.

Data Analysis Agents

Data analysis agents use language models and tools to explore datasets, run analyses, and produce answers or reports grounded in evidence.

See all 106 papers

DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces
Aug 4, 2026Boyan Li, Zhuowen Liang, Yupeng Xie +11LLM Agent Evaluation Multimodal Agents
HKUST(GZ) · Tsinghua University
Data World: Scaling Data Environments from Scratch
Oct 15, 2026Zhanli Li, Zeyuan Xu, Chaoxu Pang +16Synthetic Data Generation for LLM Agents Data Analysis Agents
Longcat Interaction Team, Meituan · Zhongnan University of Economics and Law · Hunan University +7
GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis
Sep 30, 2026Qisheng Su, Hanchen Wang, Guanru Zhu +11AI Agent Benchmarks LLM Agent Training
University of Science and Technology of China · Shanghai Innovation Institute · Fudan University

Aug 4, 2026J/K move · Enter open · S save

DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces

Boyan Li, Zhuowen Liang, Yupeng Xie, Xiaotian Lin+10

HKUST(GZ) · Tsinghua University

LLM Agent Evaluation Multimodal Agents Agent Benchmarks Data Analysis Agents

Open paper AskPDF arXiv

Data agents enable natural-language analytics over organizational workspaces, where relevant evidence may be scattered across databases, structured files, long documents, and multimedia. Existing benchmarks largely isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic evaluation insufficiently unified. We introduce DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces. It contains 410 cross-language tasks and 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video. DataSpace also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition. Each agent receives only a question and workspace and returns the complete requested tabular result. We construct DataSpace with DataSpace-Builder, an execution-grounded framework comprising cross-language transformation, constraint-aware relational sampling, modality routing and artifact rendering, and human review and task repair by 11 domain experts. A deterministic evaluator performs header-invariant column alignment, type- and precision-aware normalization, and order-aware row comparison. Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, while harness choice creates a 15.36-point spread with the backbone fixed. Multimodal evidence integration and joins consistently reduce accuracy across all six backbones. These results show that DataSpace remains unsaturated and identify key challenges for improving data-agent reliability.

Abstract

Data Analysis Agents

Explore similar work

DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces

Data World: Scaling Data Environments from Scratch

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis