The Silent Failures in Agentic Data Science

Xue Li; Hongqian Xia; Daniel Gomm; Aécio Santos; Cornelius Wolff; Madelon Hulsebos

cs.AIOct 9, 2026

The Silent Failures in Agentic Data Science

Authors: Xue Li, Hongqian Xia, Daniel Gomm, Aécio Santos, Cornelius Wolff, Madelon Hulsebos

Organizations: Centrum Wiskunde & Informatica, Amsterdam, Netherlands

Abstract

Large language models have significantly lowered the barrier to writing code. Naturally, this has also led to the development of data science agents that promise automating the design and execution of data science from task formulation to model development. Existing evaluations of agentic data science (agentic DS) assess downstream predictive performance and code correctness, providing preliminary insight in agentic DS capabilities. We argue that these evaluations overlook the fundamental principles of data science: methodological data science- and domain knowledge. As such, only assessing downstream performance masks methodological failures such as data leakage and inappropriate validation, limiting the development of robust agentic DS systems. In this paper, we evaluate agentic DS beyond accuracy from three different angles: 1) a large-scale analysis of failure modes in common agentic DS frameworks and models across 220 data science tasks, 2) a reusable and extensible test suite with curated data science tasks that test for six specific failures in a controlled manner, and 3) a qualitative assessment of agentic DS pipelines against expert solutions on Kaggle competitions. We find that the majority of agentic solutions contain at least one failure mode despite the complexity of agentic harness or model capacity. Moreover, we identify a positive correlation between the occurrence of data leakage failures and predictive performance, confirming the necessity to evaluate agents beyond accuracy. Our qualitative analysis shows that DS agents generally lack problem solving creativity and domain knowledge, compared to competition-winning data scientists. We believe that principled failure mode analysis, as presented here, should be integral part of agentic DS evaluations, and that the insights we surface will be key in developing trustworthy agentic DS pipelines.

Figures & tables

Figure 1: Percentage of pipelines with at least one failure mode per agent-model configuration.

ID	Failure	Pipeline Stage	ID	Failure	Pipeline Stage
\otherbadge FM01	Wrong task formulation	Problem Formulation	\leakbadge FM09	Feature selection before splitting	Feature Selection
\leakbadge FM02	Peeking at test data during EDA	Exploratory Data Analysis	\otherbadge FM10	Wrong model type for task	Model Selection
\otherbadge FM03	Not cleaning data	Preprocessing	\otherbadge FM11	No regularisation in DL †	Model Training
\leakbadge FM04	Imputation before splitting	Preprocessing	\otherbadge FM12	Naive ensembling †	Model Optimization
\leakbadge FM05	Ignoring temporal order †	Data Splitting	\leakbadge FM13	Reusing test set across variants †	Model Evaluation
\otherbadge FM06	No train/val split	Data Splitting	\otherbadge FM14	Single eval of stochastic model †	Model Evaluation

Table 1: Failure modes used for the large-scale analysis. Identifier (ID) colors distinguish two categories: \leakbadge FM Data Leakage modes and \otherbadge FM Non-leakage modes. Failure modes marked with a dagger ( † ) are conditionally evaluated based on pipeline stage characteristics (e.g., whether deep learning or ensembling is used).

Failure Mode	Test Case	Assessment
\tfbadge TF01 Ignoring Class Imbalance	Dataset with high class-imbalance	Assess if the agent is alerting to the skew; classifier performing poorly on minority class
\tfbadge TF02 Not Normalizing Formatting	Inconsistent formatting of equivalent values (for example, string-encoded missing values)	Assess if an agent is aware of and reacts appropriately to the inconsistency during data preprocessing
\tfbadge TF03 Not Separating Correlation from Causation	Dataset includes a feature that strongly correlates with the target, without meaningful relationship	Assess if an agent is able to distinguish spurious relations by excluding spurious features
\tfbadge TF04 No Domain Feature Engineering	Prediction task requires a domain-specific feature transformation that is not apparent in the data	Assess if an agent is able to apply feature transformation utilizing its pre-trained domain knowledge.
\tfbadge TF05 Data Leakage in Feature Selection	Dataset includes features that are not expected to be used at training time.	Assess if an agent is deliberately excluding the features.
\tfbadge TF06 Wrong Metric Selection	Task with imbalanced classes yielding high accuracy if not evaluated with a proper class-balancing metric	Assess if an agent chooses the appropriate metric accounting for imbalance

Table 2: Six controlled failure-mode test suites. Each test introduces a targeted perturbation designed to expose a specific data science failure and is evaluated with a strict evaluation measure.

Agent	Model	Wrong task formulation	Peeking at test data during EDA	Not cleaning data	Imputation before splitting	Ignoring temporal order	No train/val split	No feature engineering	Scaling before splitting	Feature selection before splitting	Wrong model type for task	No regularisation in DL	Naive ensembling	Reusing test set across variants	Single eval of stochastic model	Using only one eval metric
ReAct	Qwen-4B	0.02	0.01	0.28	0.00	0.00	1.00	0.98	0.00	0.00	0.01	N/A	N/A	N/A	0.48	0.15
	Qwen-27B	0.01	0.12	0.04	0.13	0.36	0.05	0.50	0.00	0.03	0.00	N/A	0.49	0.00	0.95	0.89
	Gemini-3.7	0.02	0.07	0.30	0.06	0.48	0.40	0.38	0.00	0.08	0.00	N/A	0.90	N/A	0.04	0.14
AIDE	Qwen-4B	0.01	0.02	0.72	0.08	0.13	0.07	0.93	0.02	0.00	0.03	N/A	N/A	N/A	0.19	0.87
	Qwen-27B	0.03	0.08	0.21	0.03	0.15	0.01	0.71	0.00	0.02	0.00	N/A	0.81	N/A	0.08	0.85
	Gemini-3.7	0.01	0.15	0.45	0.13	0.47	0.00	0.12	0.03	0.14	0.00	N/A	0.46	0.00	0.03	0.88

Table 3: Occurrence rate per failure modes (FM) by agent and model over the pipelines for which the FM is applicable. N/A indicates that the number of applicable pipelines (as defined in Section 4.2 ) for a given FM is less than 20. Red indicates leakage modes and blue indicates non-leakage modes.

Figure 2: Adjusted effect of FMs on residual rank in the test set that is accessible to the agent. A high rank indicates worse performance, while a low rank indicates better performance (* indicates significance).

		Data Science Agents			AutoML	Baseline
Challenge	Metric	ReAct	AIDE	MLEvolve	AutoGluon	1st Place (Human)
Single-Cell Perturbations	MRRMSE $\downarrow$	1.1020 / 0.7690	0.8970 / 0.6600	0.8310 / 0.6170	– / –	0.7290 / 0.5230
Heart Disease Prediction	AUC-ROC $\uparrow$	0.9547 / 0.9531	0.9549 / 0.9533	0.9546 / 0.9531	0.9551 / 0.9536	0.9554 / 0.9542
Obesity Risk	Accuracy $\uparrow$	0.9065 / 0.9090	0.9059 / 0.9025	0.9071 / 0.9035	0.9074 / 0.9075	0.9116 / 0.9234
Problematic Internet Use	QWK $\uparrow$	0.4440 / 0.4680	0.4280 / 0.4640	0.4170 / 0.4480	– / –	0.4820 / 0.5010
Survival Predictions	C-index $\uparrow$	0.6502 / 0.6465	0.6536 / 0.6488	0.3155 / 0.3233	– / –	0.7013 / 0.6981

Table 4: Leaderboard performance across Kaggle challenges. Scores reported as private / public . Best overall score is bolded, and the second-best is underscored. AutoGluon solutions are applicable only for challenges with CSV uploads (i.e., heart disease prediction and obesity risk) which are playground tasks and hence more simplistic in nature.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Agent	$n$ (high)	Wilcoxon $p$ (high)	$n$ (low)	Wilcoxon $p$ (low)
ReAct	17	0.890	18	0.212
AIDE	16	0.231	16	0.782
MLEvolve	16	0.597	17	0.890

Appendix

Table 5: Wilcoxon signed-rank test of

\Delta_{alt}

vs. the rerun randomized

\Delta_{plc}

, computed separately for high- and low-memorization rate tasks across agentic frameworks. None of the six tests reaches significance at

\alpha=0.05

, so dataset memorization does not measurably influence performance.

ID	Failure Mode	Definition
FM01	Wrong task formulation	The agent frames the modelling problem in a way that discards the signal the task actually provides.
FM02	Peeking at test data during EDA	Exploring test data during initial analysis creates untestable assumptions baked into model design.
FM03	Not cleaning data	Ignoring data duplication, missing values, and outliers/inconsistencies before model building.
FM04	Imputation before splitting	Performing mean/median imputation before splitting – leaks test statistics into training.
FM05	Ignoring temporal order	Randomly splitting time-series data without respecting temporal order – introduces look-ahead bias.
FM06	No train/val split	Model trained and evaluated on the same data – performance is a misleading overestimate.

Appendix

Table 6: Failure modes definition.

Source	N	Regr.	Bin.-cls.	Multi-cls.	Med. rows	Med. feat.
Kaggle	44	18	17	8	100,820	24
MLE-bench	8	6	2	0	6,976	13
OpenML	36	10	15	11	3,104	18
STRABLE	108	76	13	19	7,710	17
UCI	24	12	7	5	11,805	13
Total	220	122	54	43	8,693	18

Appendix

Table 7: Dataset characteristics across the 220 benchmark tasks.

Domain	Tasks
Healthcare	50
Finance	47
Science	25
Retail	23
Social	21
Energy	17

Appendix

Table 8: Domain distribution across the 220 benchmark tasks.

FM	Failure Mode	$n$	Agreement	Precision	Recall	Human Present
FM01	Wrong task formulation †	19	0.95	0.00	–	0
FM02	Peeking at test data during EDA †	19	0.95	0.00	–	0
FM03	Not cleaning data	19	0.95	0.80	1.00	4
FM04	Imputation before splitting	19	0.79	0.50	0.25	4
FM05	Ignoring temporal order †	19	0.95	0.50	1.00	1
FM06	No train/val split	19	0.89	0.67	1.00	4

Appendix

Table 9: Human-vs-LLM-judge agreement on 19 randomly sampled, independently annotated pipelines. Scored as Present vs. Not-Present (all 15 failure modes per pipeline). Cohen’s

\kappa=0.62

(95% CI [0.50, 0.73]) indicates moderate, above-chance agreement. Precision exceeds recall generally (judge-as-predictor, human as ground truth), indicating the judge is conservative rather than strict: reported failure rates in the main results are more likely an under- than an over-estimate.

Agent	Model	$n$ judged	Ignoring temporal order	No regularisation in DL	Reusing test set across variants	Single eval of stochastic model	Naive ensembling
ReAct	Gemini-3.7	219	137	0	0	211	198
	Qwen-4B	220	138	0	0	219	1
	Qwen-27B	220	138	0	124	217	118
AIDE	Gemini-3.7	219	137	3	41	215	215
	Qwen-4B	211	133	0	0	190	3
	Qwen-27B	185	112	0	4	176	27

Appendix

Table 10: Number of applicable pipelines per failure mode, by agent and model.

Figure 5: Average failure modes per pipeline by agent-model configuration.

Figure 6: Failure mode distribution. 4,557 failures detected over 1,761 pipelines, averaging 2.58 failures per pipeline.

Figure 7: Effects of each failure mode on predictive performance measured in residual rank.

Agent	Model	Validation $\rightarrow$ train leakage	Test $\rightarrow$ train leakage	Flawed training protocol
ReAct	Qwen-4B	0.00	0.00	0.03
	Qwen-27B	0.08	0.02	0.07
	Gemini-3.7	0.15	0.15	0.12
AIDE	Qwen-4B	0.16	0.01	0.29
	Qwen-27B	0.07	0.02	0.03
	Gemini-3.7	0.13	0.35	0.01

Appendix

Table 11: Share of pipelines for which the judge reports the three most prominent failure modes beyond the taxonomy.

Figure 8: Exemplar pipelines of DS agents generated solutions using gemini-3.7-flash v.s. human expert solution.

Explore similar work

Jan 20, 2026

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

Recent LLM-based data agents aim to automate data science tasks ranging from data analysis to deep learning. However, the open-ended nature of real-world data science problems, which often span multiple taxonomies and lack standard answers, poses a significant challenge for evaluation. To address this, we introduce DSAEval, a benchmark comprising 641 real-world data science problems grounded in 285 diverse datasets, covering both structured and unstructured data (e.g., image and text). DSAEval incorporates three distinctive features: (1) Multimodal Environment Perception, which enables agents to interpret observations from multiple modalities, including text and vision; (2) Multi-Query Interactions, which mirror the iterative and cumulative nature of real-world data science projects; and (3) Multi-Dimensional Evaluation, which provides a holistic assessment across reasoning, code, and results. We systematically evaluate 13 recent advanced agentic LLMs using DSAEval. Our results show that Claude-Sonnet-4.5 achieves the strongest overall performance, MiMo-V2-Pro and GPT-5.2 lead in duration and step efficiency, respectively, and MiMo-V2-Flash is the most cost-effective. We further demonstrate that multimodal perception consistently improves performance on vision-related tasks, with gains ranging from 2.04% to 11.30%. Overall, while current data science agents perform well on structured data and routine data analysis workflows, substantial challenges remain in unstructured domains. Finally, we offer critical insights and outline future research directions.

Maojun Sun, Yifei Xie, Yue Wu +5

Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Department of Applied Mathematics, The Hong Kong Polytechnic University

LLM Agent Evaluation Agent Evaluation AI Agent Benchmarks

Ask PDF

Mar 19, 2026

AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science

Data science plays a critical role in transforming complex data into actionable insights across numerous domains. Recent developments in large language models (LLMs) and artificial intelligence (AI) agents have significantly automated data science workflow. However, it remains unclear to what extent AI agents can match the performance of human experts on domain-specific data science tasks, and in which aspects human expertise continues to provide advantages. We introduce AgentDS, a benchmark and competition designed to evaluate both AI agents and human-AI collaboration performance in domain-specific data science. AgentDS consists of 17 challenges across six industries: commerce, food production, healthcare, insurance, manufacturing, and retail banking. We conducted an open competition involving 29 teams and 80 participants, enabling systematic comparison between human-AI collaborative approaches and AI-only baselines. Our results show that current AI agents struggle with domain-specific reasoning. AI-only baselines perform below the top quartile of competition participants, while the strongest solutions arise from human-AI collaboration. These findings challenge the narrative of complete automation by AI and underscore the enduring importance of human expertise in data science, while illuminating directions for the next generation of AI. Visit the AgentDS website here: https://agentds.org/ and open source datasets here: https://huggingface.co/datasets/lainmn/AgentDS .

An Luo, Jin Du, Xun Xian +12

School of Statistics, University of Minnesota · AIScientists, Inc. · Data Science Institute, University of Chicago +4

AI Agent Evaluation AI Agent Benchmarks Human-AI Collaboration Data Analysis Agents Benchmark Design

Ask PDF

Apr 13, 2026

Sanity Checks for Agentic Data Science

Agentic data science (ADS) pipelines have grown rapidly in both capability and adoption, with systems such as OpenAI Codex now able to directly analyze datasets and produce answers to statistical questions. However, these systems can reach falsely optimistic conclusions that are difficult for users to detect. To address this, we propose a pair of lightweight sanity checks grounded in the Predictability-Computability-Stability (PCS) framework for veridical data science. These checks use reasonable perturbations to screen whether an agent can reliably distinguish signal from noise, acting as a falsifiability constraint that can expose affirmative conclusions as unsupported. Together, the two checks characterize the trustworthiness of an ADS output, e.g. whether it has found stable signal, is responding to noise, or is sensitive to incidental aspects of the input. We validate the approach on synthetic data with controlled signal-to-noise ratios, confirming that the sanity checks track ground-truth signal strength. We then demonstrate the checks on 11 real-world datasets using OpenAI Codex, characterizing the trustworthiness of each conclusion and finding that in 6 of the datasets an affirmative conclusion is not well-supported, even though a single ADS run may support one. We further analyze failure modes of ADS systems and find that ADS self-reported confidence is poorly calibrated to the empirical stability of its conclusions.

Zachary T. Rewolinski, Austin V. Zane, Hao Huang +4

Dept. of Statistics, UC Berkeley · Dept. of EECS, UC Berkeley · Microsoft Research +1

AI Agent Reliability Data Analysis Agents Language Model Calibration

Ask PDF