Data science workflows often require tools that presume non-trivial technical knowledge. Large language models (LLMs) reduce this barrier, but a gap remains between flexible general-purpose agents and statistically controlled pipelines. We introduce DSFlow, a LangGraph-based neuro-symbolic data-agent platform that combines reusable, LLM-selected skills with deterministic diagnostic guardrails. Skills provide extensible task knowledge. Symbolic validation can accept a proposed method, request clarification or replanning, override an unsuitable method, or supply a safe fallback. Regression and clustering case studies illustrate how DSFlow uses diagnostic evidence to correct method choices and documents those decisions in auditable reports.
Figures & tables
Figure 1: DSFlow workflow for the evaluated regression and clustering paths. The LLM proposes a plan. Schema validation can request clarification or replanning, and diagnostics determine whether to retain or override the proposed method. The examples pair ordinary least squares (OLS) with Huber robust regression and K -means with spectral clustering. A deterministic stage checks the executor’s outputs and formats the report. Classification is supported in the architecture and outside the scope of the evaluation.
System
Regression
Clustering
Criteria met
DSFlow
7.00±0.00
7.00±0.00
42/42
Manus
2.67±0.47
3.67±1.89
19/42
Gemini
2.00±0.00
1.33±0.47
10/42
ChatGPT
1.00±0.00
1.67±0.47
8/42
Table 1: Original descriptive case-study scores (maximum 7 per run), reported as mean ± population SD over three runs.
Figure 2: Original product outputs for the non-convex clustering case across three executions per system. Point colors denote the cluster labels assigned by each system and need not correspond across panels. The label correct denotes recovery of the central disc and both rings. Centroid solutions divide rings into angular sectors.
Policy
R-A
R-O
C-A
C-O
Total
Selective router
18/20
19/20
20/20
20/20
77/80
No override
20/20
0/20
20/20
0/20
40/80
Constant default
20/20
0/20
20/20
0/20
40/80
Constant complex
0/20
20/20
0/20
20/20
40/80
Table 2: Final route accuracy across four controlled conditions.
Model
Task
Target
Features
Plan
Clar. R
Fails
Llama 3.3 70B
24/25
12/12
22/22
21/21
4/4
0
Llama 3.1 8B
24/25
12/12
22/22
21/21
4/4
0
Table 3: Language-interface results. Plan is first-pass validity among requests that reached planning. Clar. R is clarification recall on the five ambiguity and guardrail prompts.
Condition
Generator
Rows
Preference
Expected method
R- Accept
Linear Gaussian
1,000
OLS
OLS
R- Override
Paired contaminated linear
1,000
OLS
Huber regression
C- Accept
Gaussian blobs
900
K -means
K -means
C- Override
Disc and two rings
1,000
K -means
Spectral clustering
Table 4: Controlled routing conditions. Each row describes one dataset per seed.
Multiplier
Correct
R- Accept
R- Override
Median RMSE
0.50
20/40
0/20
20/20
1.013
0.75
20/40
0/20
20/20
1.013
1.00
37/40
18/20
19/20
1.013
1.25
38/40
19/20
19/20
1.013
1.50
38/40
19/20
19/20
1.013
2.00
38/40
19/20
19/20
1.013
Table 5: Prespecified regression-threshold sensitivity. The multiplier scales the three influence cutoffs together. The coefficient-shift threshold is fixed.
Agreement cutoff
Correct
C- Accept
C- Override
0.50
40/40
20/20
20/20
0.75
40/40
20/20
20/20
0.90
40/40
20/20
20/20
0.95
40/40
20/20
20/20
0.975
40/40
20/20
20/20
0.99
40/40
20/20
20/20
Table 6: Clustering agreement-threshold sensitivity. The default 0.95 was retained; no threshold was selected from these outcomes.
Configuration
Regression
Clustering
Full deterministic core
Robust; RMSE 100,131
Spectral; sizes 250/250/500
Oracle plan
Same as full core
Same as full core
Constant complex
Same as full core
Same as full core
Override disabled
OLS; RMSE 127,139
K -means; six sectors
Execution guardrail off
Same as full core
Same as full core
LLM only
Failed before execution
K -means; six groups
Table 7: Component configurations evaluated on the two original cases.
Rubric criterion
Agreement
Cohen’s κ
Regression: detects influence
50.0%
0.08
Regression: identifies injected rows
83.3%
0.57
Regression: handles contamination
83.3%
0.64
Regression: predictive validation
100%
1.00
Regression: quantifies associations
100%
1.00
Regression: interprets limits
100%
1.00
Table 8: Inter-rater agreement before adjudication. Each criterion covers 12 runs and the overall row covers 168 binary judgments. Cohen’s kappa is undefined when both raters assign the same single category to every run.
Figure 3: Archived contaminated-regression interface overview showing the submitted request, uploaded dataset, target, retained feature set, and selected robust-regression method.
Figure 4: Original contaminated-regression sensitivity analysis. Holdout RMSE was 127,139 for OLS on all training rows, 100,131 for robust regression retaining all rows, and 100,122 for OLS after excluding the three highest-influence training rows.
Figure 5: Archived rendered sections of the original contaminated-regression report. Coefficients are conditional model estimates rather than causal effects.
Figure 6: Archived regression decision trace and generated diagnostics. The workflow used Cook’s distance, leverage, and studentized residuals to prioritize influential training rows before selecting robust regression.
Figure 7: Archived non-convex-clustering output. The local graph supported three connected components across neighbor counts 6–20, and the workflow selected spectral clustering.
Figure 8: Archived clustering report, symbolic-override trace, and final visualization. The spectral partition recovered the central disc and the inner and outer rings, with 250, 250, and 500 points, respectively.
Recent advances in large language models (LLMs) have established human-agent collaboration as a promising paradigm for data science. Effective collaboration requires a shared computational abstraction that enables agents to reason about processing logic while allowing human analysts to understand, guide, and refine the analysis. Although scripts written in Python or SQL provide precise control, their syntax and execution details can be difficult for analysts to understand and modify. In this paper, we argue that dataflows are an effective abstraction for human-agent collaboration in data science. We introduce BobFlow, a system that combines the dataflow abstraction with the ReAct paradigm. It enables agents to iteratively construct and execute dataflows for data science tasks while allowing analysts to inspect the analysis and provide feedback. Our user study shows that BobFlow helps analysts understand, inspect, and verify agent-generated analyses. A thorough experimental evaluation across proprietary and open-weight LLMs shows accuracy score gains of up to 18.6 points at just over 1/3 of a script-based ReAct agent's inference cost.
Jiadong Bai, Yicong Huang, Chen Li
University of California, Irvine Irvine, California, USA · University of Massachusetts Amherst Amherst, Massachusetts, USA
Large language models (LLMs) are increasingly used to automate data-processing workflows, yet coding agents typically produce scripts that are not automatically materialized as persistent, editable platform artifacts. We call this disconnect the NL2Pipeline gap. To bridge it, we introduce DataFlow-Harness, a platform that guides an LLM agent to construct platform-native directed acyclic graphs (DAGs) through typed, incremental mutations rather than free-form scripts. The platform combines DataFlow-Skills for procedural guidance, a Model Context Protocol (MCP) layer that exposes the live operator registry and current pipeline state, and DataFlow-WebUI, which synchronizes conversational authoring with a visual DAG editor. On a 12-task data-engineering benchmark, DataFlow-Harness achieves a 93.3% observed end-to-end pass rate. Relative to Vanilla Claude Code, it reduces measured monetary cost by 72.5% and generation latency by 49.9%; its observed pass rate is within 0.9 percentage points of the Context-Aware Claude Code baseline while its cost is 42.8% lower. Per-task analysis indicates that Skills are most useful when construction depends on implicit procedural knowledge. These results show that live platform grounding can produce persistent, editable workflow artifacts with an observed reliability close to script-generation baselines and with lower measured construction cost and latency.
Runming He, Zhen Hao Wong, Hao Liang +4
Peking University · Institute for Advanced Algorithms Research, Shanghai · Zhongguancun Academy
Large Language Model (LLM) agents have shown great potential in addressing real-world data science problems. LLM-driven data science agents promise to automate the entire machine learning pipeline, yet their real-world effectiveness remains limited. Existing frameworks depend on rigid, pre-defined workflows and inflexible coding strategies; consequently, they excel only on relatively simple, classical problems and fail to capture the empirical expertise that human practitioners bring to complex, innovative tasks. In this work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework that overcomes these deficiencies through three key advances: (1) a curated expert knowledge base that grounds the agent in domain expert knowledge, (2) an agentic knowledgeable tree search algorithm that strategically explores possible solutions, and (3) a self-adaptive coding strategy that dynamically tailors code generation to task complexity. Evaluations on two automated data science benchmarks demonstrate that AutoMind delivers superior performance versus state-of-the-art baselines. Additional analyses confirm favorable effectiveness, efficiency, and qualitative solution quality, highlighting AutoMind as an efficient and robust step toward fully automated data science. Code is at https://github.com/zjunlp/AutoMind.
Yujie Luo, Yixin Ou, Jingsheng Zheng +9
Zhejiang University · Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph · Ant Group