The Silent Failures in Agentic Data Science

Xue Li; Hongqian Xia; Daniel Gomm; Aécio Santos; Cornelius Wolff; Madelon Hulsebos

cs.AIOct 9, 2026

The Silent Failures in Agentic Data Science

Authors: Xue Li, Hongqian Xia, Daniel Gomm, Aécio Santos, Cornelius Wolff, Madelon Hulsebos

Organizations: Centrum Wiskunde & Informatica, Amsterdam, Netherlands

Abstract

Large language models have significantly lowered the barrier to writing code. Naturally, this has also led to the development of data science agents that promise automating the design and execution of data science from task formulation to model development. Existing evaluations of agentic data science (agentic DS) assess downstream predictive performance and code correctness, providing preliminary insight in agentic DS capabilities. We argue that these evaluations overlook the fundamental principles of data science: methodological data science- and domain knowledge. As such, only assessing downstream performance masks methodological failures such as data leakage and inappropriate validation, limiting the development of robust agentic DS systems. In this paper, we evaluate agentic DS beyond accuracy from three different angles: 1) a large-scale analysis of failure modes in common agentic DS frameworks and models across 220 data science tasks, 2) a reusable and extensible test suite with curated data science tasks that test for six specific failures in a controlled manner, and 3) a qualitative assessment of agentic DS pipelines against expert solutions on Kaggle competitions. We find that the majority of agentic solutions contain at least one failure mode despite the complexity of agentic harness or model capacity. Moreover, we identify a positive correlation between the occurrence of data leakage failures and predictive performance, confirming the necessity to evaluate agents beyond accuracy. Our qualitative analysis shows that DS agents generally lack problem solving creativity and domain knowledge, compared to competition-winning data scientists. We believe that principled failure mode analysis, as presented here, should be integral part of agentic DS evaluations, and that the insights we surface will be key in developing trustworthy agentic DS pipelines.

Figures & tables

Figure 1: Percentage of pipelines with at least one failure mode per agent-model configuration.

ID	Failure	Pipeline Stage	ID	Failure	Pipeline Stage
\otherbadge FM01	Wrong task formulation	Problem Formulation	\leakbadge FM09	Feature selection before splitting	Feature Selection
\leakbadge FM02	Peeking at test data during EDA	Exploratory Data Analysis	\otherbadge FM10	Wrong model type for task	Model Selection
\otherbadge FM03	Not cleaning data	Preprocessing	\otherbadge FM11	No regularisation in DL †	Model Training
\leakbadge FM04	Imputation before splitting	Preprocessing	\otherbadge FM12	Naive ensembling †	Model Optimization
\leakbadge FM05	Ignoring temporal order †	Data Splitting	\leakbadge FM13	Reusing test set across variants †	Model Evaluation
\otherbadge FM06	No train/val split	Data Splitting	\otherbadge FM14	Single eval of stochastic model †	Model Evaluation

Table 1: Failure modes used for the large-scale analysis. Identifier (ID) colors distinguish two categories: \leakbadge FM Data Leakage modes and \otherbadge FM Non-leakage modes. Failure modes marked with a dagger ( † ) are conditionally evaluated based on pipeline stage characteristics (e.g., whether deep learning or ensembling is used).

Failure Mode	Test Case	Assessment
\tfbadge TF01 Ignoring Class Imbalance	Dataset with high class-imbalance	Assess if the agent is alerting to the skew; classifier performing poorly on minority class
\tfbadge TF02 Not Normalizing Formatting	Inconsistent formatting of equivalent values (for example, string-encoded missing values)	Assess if an agent is aware of and reacts appropriately to the inconsistency during data preprocessing
\tfbadge TF03 Not Separating Correlation from Causation	Dataset includes a feature that strongly correlates with the target, without meaningful relationship	Assess if an agent is able to distinguish spurious relations by excluding spurious features
\tfbadge TF04 No Domain Feature Engineering	Prediction task requires a domain-specific feature transformation that is not apparent in the data	Assess if an agent is able to apply feature transformation utilizing its pre-trained domain knowledge.
\tfbadge TF05 Data Leakage in Feature Selection	Dataset includes features that are not expected to be used at training time.	Assess if an agent is deliberately excluding the features.
\tfbadge TF06 Wrong Metric Selection	Task with imbalanced classes yielding high accuracy if not evaluated with a proper class-balancing metric	Assess if an agent chooses the appropriate metric accounting for imbalance

Table 2: Six controlled failure-mode test suites. Each test introduces a targeted perturbation designed to expose a specific data science failure and is evaluated with a strict evaluation measure.

Agent	Model	Wrong task formulation	Peeking at test data during EDA	Not cleaning data	Imputation before splitting	Ignoring temporal order	No train/val split	No feature engineering	Scaling before splitting	Feature selection before splitting	Wrong model type for task	No regularisation in DL	Naive ensembling	Reusing test set across variants	Single eval of stochastic model	Using only one eval metric
ReAct	Qwen-4B	0.02	0.01	0.28	0.00	0.00	1.00	0.98	0.00	0.00	0.01	N/A	N/A	N/A	0.48	0.15
	Qwen-27B	0.01	0.12	0.04	0.13	0.36	0.05	0.50	0.00	0.03	0.00	N/A	0.49	0.00	0.95	0.89
	Gemini-3.7	0.02	0.07	0.30	0.06	0.48	0.40	0.38	0.00	0.08	0.00	N/A	0.90	N/A	0.04	0.14
AIDE	Qwen-4B	0.01	0.02	0.72	0.08	0.13	0.07	0.93	0.02	0.00	0.03	N/A	N/A	N/A	0.19	0.87
	Qwen-27B	0.03	0.08	0.21	0.03	0.15	0.01	0.71	0.00	0.02	0.00	N/A	0.81	N/A	0.08	0.85
	Gemini-3.7	0.01	0.15	0.45	0.13	0.47	0.00	0.12	0.03	0.14	0.00	N/A	0.46	0.00	0.03	0.88

Table 3: Occurrence rate per failure modes (FM) by agent and model over the pipelines for which the FM is applicable. N/A indicates that the number of applicable pipelines (as defined in Section 4.2 ) for a given FM is less than 20. Red indicates leakage modes and blue indicates non-leakage modes.

Figure 2: Adjusted effect of FMs on residual rank in the test set that is accessible to the agent. A high rank indicates worse performance, while a low rank indicates better performance (* indicates significance).

		Data Science Agents			AutoML	Baseline
Challenge	Metric	ReAct	AIDE	MLEvolve	AutoGluon	1st Place (Human)
Single-Cell Perturbations	MRRMSE $\downarrow$	1.1020 / 0.7690	0.8970 / 0.6600	0.8310 / 0.6170	– / –	0.7290 / 0.5230
Heart Disease Prediction	AUC-ROC $\uparrow$	0.9547 / 0.9531	0.9549 / 0.9533	0.9546 / 0.9531	0.9551 / 0.9536	0.9554 / 0.9542
Obesity Risk	Accuracy $\uparrow$	0.9065 / 0.9090	0.9059 / 0.9025	0.9071 / 0.9035	0.9074 / 0.9075	0.9116 / 0.9234
Problematic Internet Use	QWK $\uparrow$	0.4440 / 0.4680	0.4280 / 0.4640	0.4170 / 0.4480	– / –	0.4820 / 0.5010
Survival Predictions	C-index $\uparrow$	0.6502 / 0.6465	0.6536 / 0.6488	0.3155 / 0.3233	– / –	0.7013 / 0.6981

Table 4: Leaderboard performance across Kaggle challenges. Scores reported as private / public . Best overall score is bolded, and the second-best is underscored. AutoGluon solutions are applicable only for challenges with CSV uploads (i.e., heart disease prediction and obesity risk) which are playground tasks and hence more simplistic in nature.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Agent	$n$ (high)	Wilcoxon $p$ (high)	$n$ (low)	Wilcoxon $p$ (low)
ReAct	17	0.890	18	0.212
AIDE	16	0.231	16	0.782
MLEvolve	16	0.597	17	0.890

Appendix

Table 5: Wilcoxon signed-rank test of

\Delta_{alt}

vs. the rerun randomized

\Delta_{plc}

, computed separately for high- and low-memorization rate tasks across agentic frameworks. None of the six tests reaches significance at

\alpha=0.05

, so dataset memorization does not measurably influence performance.

ID	Failure Mode	Definition
FM01	Wrong task formulation	The agent frames the modelling problem in a way that discards the signal the task actually provides.
FM02	Peeking at test data during EDA	Exploring test data during initial analysis creates untestable assumptions baked into model design.
FM03	Not cleaning data	Ignoring data duplication, missing values, and outliers/inconsistencies before model building.
FM04	Imputation before splitting	Performing mean/median imputation before splitting – leaks test statistics into training.
FM05	Ignoring temporal order	Randomly splitting time-series data without respecting temporal order – introduces look-ahead bias.
FM06	No train/val split	Model trained and evaluated on the same data – performance is a misleading overestimate.

Appendix

Table 6: Failure modes definition.

Source	N	Regr.	Bin.-cls.	Multi-cls.	Med. rows	Med. feat.
Kaggle	44	18	17	8	100,820	24
MLE-bench	8	6	2	0	6,976	13
OpenML	36	10	15	11	3,104	18
STRABLE	108	76	13	19	7,710	17
UCI	24	12	7	5	11,805	13
Total	220	122	54	43	8,693	18

Appendix

Table 7: Dataset characteristics across the 220 benchmark tasks.

Domain	Tasks
Healthcare	50
Finance	47
Science	25
Retail	23
Social	21
Energy	17

Appendix

Table 8: Domain distribution across the 220 benchmark tasks.

FM	Failure Mode	$n$	Agreement	Precision	Recall	Human Present
FM01	Wrong task formulation †	19	0.95	0.00	–	0
FM02	Peeking at test data during EDA †	19	0.95	0.00	–	0
FM03	Not cleaning data	19	0.95	0.80	1.00	4
FM04	Imputation before splitting	19	0.79	0.50	0.25	4
FM05	Ignoring temporal order †	19	0.95	0.50	1.00	1
FM06	No train/val split	19	0.89	0.67	1.00	4

Appendix

Table 9: Human-vs-LLM-judge agreement on 19 randomly sampled, independently annotated pipelines. Scored as Present vs. Not-Present (all 15 failure modes per pipeline). Cohen’s

\kappa=0.62

(95% CI [0.50, 0.73]) indicates moderate, above-chance agreement. Precision exceeds recall generally (judge-as-predictor, human as ground truth), indicating the judge is conservative rather than strict: reported failure rates in the main results are more likely an under- than an over-estimate.

Agent	Model	$n$ judged	Ignoring temporal order	No regularisation in DL	Reusing test set across variants	Single eval of stochastic model	Naive ensembling
ReAct	Gemini-3.7	219	137	0	0	211	198
	Qwen-4B	220	138	0	0	219	1
	Qwen-27B	220	138	0	124	217	118
AIDE	Gemini-3.7	219	137	3	41	215	215
	Qwen-4B	211	133	0	0	190	3
	Qwen-27B	185	112	0	4	176	27

Appendix

Table 10: Number of applicable pipelines per failure mode, by agent and model.

Figure 5: Average failure modes per pipeline by agent-model configuration.

Figure 6: Failure mode distribution. 4,557 failures detected over 1,761 pipelines, averaging 2.58 failures per pipeline.

Figure 7: Effects of each failure mode on predictive performance measured in residual rank.

Agent	Model	Validation $\rightarrow$ train leakage	Test $\rightarrow$ train leakage	Flawed training protocol
ReAct	Qwen-4B	0.00	0.00	0.03
	Qwen-27B	0.08	0.02	0.07
	Gemini-3.7	0.15	0.15	0.12
AIDE	Qwen-4B	0.16	0.01	0.29
	Qwen-27B	0.07	0.02	0.03
	Gemini-3.7	0.13	0.35	0.01

Appendix

Table 11: Share of pipelines for which the judge reports the three most prominent failure modes beyond the taxonomy.

Figure 8: Exemplar pipelines of DS agents generated solutions using gemini-3.7-flash v.s. human expert solution.

Explore similar work

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems
Jan 20, 2026Maojun Sun, Yifei Xie, Yue Wu +5LLM Agent Evaluation Agent Evaluation
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Department of Applied Mathematics, The Hong Kong Polytechnic University
AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science
Mar 19, 2026An Luo, Jin Du, Xun Xian +12AI Agent Evaluation AI Agent Benchmarks
School of Statistics, University of Minnesota · AIScientists, Inc. · Data Science Institute, University of Chicago +4
Sanity Checks for Agentic Data Science
Apr 13, 2026Zachary T. Rewolinski, Austin V. Zane, Hao Huang +4AI Agent Reliability Data Analysis Agents
Dept. of Statistics, UC Berkeley · Dept. of EECS, UC Berkeley · Microsoft Research +1

Jan 20, 2026J/K move · Enter open · S save

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

Maojun Sun, Yifei Xie, Yue Wu, Ruijian Han+4

Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Department of Applied Mathematics, The Hong Kong Polytechnic University

LLM Agent Evaluation Agent Evaluation AI Agent Benchmarks

Open paper AskPDF arXiv

Recent LLM-based data agents aim to automate data science tasks ranging from data analysis to deep learning. However, the open-ended nature of real-world data science problems, which often span multiple taxonomies and lack standard answers, poses a significant challenge for evaluation. To address this, we introduce DSAEval, a benchmark comprising 641 real-world data science problems grounded in 285 diverse datasets, covering both structured and unstructured data (e.g., image and text). DSAEval incorporates three distinctive features: (1) Multimodal Environment Perception, which enables agents to interpret observations from multiple modalities, including text and vision; (2) Multi-Query Interactions, which mirror the iterative and cumulative nature of real-world data science projects; and (3) Multi-Dimensional Evaluation, which provides a holistic assessment across reasoning, code, and results. We systematically evaluate 13 recent advanced agentic LLMs using DSAEval. Our results show that Claude-Sonnet-4.5 achieves the strongest overall performance, MiMo-V2-Pro and GPT-5.2 lead in duration and step efficiency, respectively, and MiMo-V2-Flash is the most cost-effective. We further demonstrate that multimodal perception consistently improves performance on vision-related tasks, with gains ranging from 2.04% to 11.30%. Overall, while current data science agents perform well on structured data and routine data analysis workflows, substantial challenges remain in unstructured domains. Finally, we offer critical insights and outline future research directions.

Abstract

Explore similar work

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science

Sanity Checks for Agentic Data Science