cs.LG · 2604.25765 Copy arXiv ID · Apr 28, 2026 Save Measuring the Sensitivity of Classification Models with the Error Sensitivity Profile Authors: Andrea Maurino
Organizations: Università degli studi di Milano Bicocca Dipartimento di Informatica, Sistemistica e Comunicazione Viale Sarca 336,20126 Milano, Italy
Abstract The quality of training data is critical to the performance of machine learning models. In this paper, the Error Sensitivity Profile (ESP) is proposed. It quantifies the sensitivity of model performance to errors in a single feature or in multiple features. By leveraging ESP, data-cleaning efforts can be prioritized based on error types and features most likely to affect model performance. To support the computation of this metric, an integrated suite of tools, called \dirty, is created. We conduct an extensive experimental study on two widely used datasets using 14 classification models, revealing that performance degradation is not always predictable from simple correlations with the target variable.
Explore similar work Jun 10, 2026 · Jiale Deng, Yanyan Shen, Xiaogang Shi +1 Training Data Training Data Attribution
Jul 28, 2026 · Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz +1 Model Sensitivity Causal Reasoning
Jul 22, 2026 · Nicholas Chandler, Sebastian Jäger, Philipp Jung +1 Tabular Data Errors
Jun 10, 2026 · cs.LG J/K move · Enter open · S save
Jiale Deng, Yanyan Shen, Xiaogang Shi, Junjun Chai
High-quality training data is essential for the success of machine learning models. However, real-world datasets often contain mixed types of errors arising from systematic flaws in data preparation pipelines, including label errors, feature errors, and spurious correlations. Effective debugging of training data requires both detecting erroneous samples and identifying their specific error types to enable targeted repair, yet existing data cleaning and attribution methods fail to adequately address this dual requirement. In this paper, we propose DeMix, a novel framework that simultaneously diagnoses erroneous samples and their error types. Our key insight is that different error types produce distinct patterns on model behavior. DeMix captures such error-specific patterns by influence vectors that characterize how each training sample affects model predictions across all validation samples. We formulate training data debugging as a multi-label classification problem where a classifier is developed to predict error types directly from influence vectors. We further introduce an intervention-based learning strategy that guides the classifier to capture invariant rationales specific to each error type, ensuring the learned classifier generalizes effectively. Empirical evaluations on 11 tasks across tabular data prediction, recommendation systems, and LLM alignment demonstrate that DeMix significantly outperforms state-of-the-art approaches, achieving a 22.61% improvement in data debugging F1-score and a 9.32% gain in task model performance after data repair. Code is available at: https://github.com/SJTU-DMTai/DeMix.