Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification

Xabier de Juan; Santiago Mazuelas; Yilun Zhu; Clayton Scott

stat.MLSep 30, 2026

Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification

Authors: Xabier de Juan, Santiago Mazuelas, Yilun Zhu, Clayton Scott

Organizations: Basque Center of Applied Mathematics (BCAM) · IKERBASQUE-Basque Foundation for Science · Electrical and Computer Engineering, University of Michigan

Abstract

Modern machine learning depends heavily on massive datasets, but obtaining high-quality annotations at scale is often expensive. As a result, learning from noisily-labeled data has become common, making accurate estimation of the label-noise transition matrix crucial. However, existing transition matrix estimators rely on the fragile estimation of class-posteriors and do not provide finite-sample performance guarantees. In this work, we propose a novel methodology to estimate the transition matrix based on one-sided selective classification. This approach bypasses class-posterior estimation, provides finite-sample performance guarantees, and leverages flexible learning methods for binary classification. Moreover, we introduce effective algorithms to implement the proposed methodology and provide their refined finite-sample performance bounds.

Figures & tables

Panel of Figure 1 — Figure 1: Mean Absolute Error (MAE) of different estimators as the number of training samples $n$ increases. The errors of the presented algorithms decrease with the number of samples in a similar trend to the error of the oracle, whereas the error of the anchor-based method stagnates.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: The three regimes described above appear on CIFAR-10. Moreover, the value of the plateau is close to $T_{j,j}$ .

Panel of Figure 3 — Figure 3: The value of the plateau is close to $T_{j,j}=0.8$ and the noise is uniform. Moreover, with fewer samples the plateau is shorter and noisier.

Figure 4: The value of the plateau is close to $T_{j,j}=0.5$ and the noise is uniform.

	CIFAR-10	MNIST		Letter		Satellite
	CIFAR-10	NN	Logistic reg.	RF	Logistic reg.	RF	Logistic reg.
Anchor-based	$.009\pm.001$	$.010\pm.001$	$.016\pm.001$	$.044\pm.000$	$.047\pm.000$	$.022\pm.001$	$\mathbf{.067\pm.023}$
Algorithm 1	$.008\pm.001$	$.008\pm.001$	$\mathbf{.007\pm.000}$	$\mathbf{.005\pm.000}$	$\mathbf{.021\pm.001}$	$\mathbf{.019\pm.002}$	$.068\pm.010$
Algorithm 2	$\mathbf{.006\pm.001}$	$\mathbf{.004\pm.000}$	$.012\pm.001$	$.006\pm.000$	$.029\pm.001$	$.021\pm.004$	$.085\pm.003$

Appendix

Table 1: MAE of different estimators under 0.2-uniform noise.

	CIFAR-10	MNIST		Letter		Satellite
	CIFAR-10	NN	Logistic reg.	RF	Logistic reg.	RF	Logistic reg.
Anchor-based	$.029\pm.011$	$.016\pm.009$	$\mathbf{.023\pm.003}$	$.046\pm.001$	$.048\pm.002$	$\mathbf{.039\pm.011}$	$.085\pm.056$
Algorithm 1	$.036\pm.004$	$.033\pm.003$	$.033\pm.004$	$\mathbf{.014\pm.001}$	$\mathbf{.031\pm.000}$	$.044\pm.005$	$\mathbf{.066\pm.003}$
Algorithm 2	$\mathbf{.015\pm.005}$	$\mathbf{.008\pm.003}$	$.037\pm.006$	$\mathbf{.014\pm.001}$	$.039\pm.001$	$.045\pm.004$	$.083\pm.003$

Appendix

Table 2: MAE of different estimators under 0.45-flip noise.

	CIFAR-10	MNIST		Letter		Satellite
	CIFAR-10	NN	Logistic reg.	RF	Logistic reg.	RF	Logistic reg.
Anchor-based	$.009\pm.001$	$.011\pm.001$	$.017\pm.002$	$.041\pm.001$	$.045\pm.001$	$.021\pm.003$	$.102\pm.065$
Algorithm 1	$.009\pm.003$	$.008\pm.001$	$\mathbf{.008\pm.001}$	$\mathbf{.006\pm.001}$	$\mathbf{.021\pm.001}$	$\mathbf{.020\pm.002}$	$\mathbf{.068\pm.010}$
Algorithm 2	$\mathbf{.006\pm.001}$	$\mathbf{.005\pm.001}$	$.011\pm.003$	$.007\pm.000$	$.029\pm.001$	$.021\pm.003$	$.089\pm.011$

Appendix

Table 3: MAE of different estimators under random asymmetric uniform noise.

Figure 5: The presented methods remain insensitive to the dimension, whereas the error of the anchor-based method increases with the dimension.

Figure 6: The proposed algorithms have low computational overhead in practice as an execution requires at most a few minutes to run.

Explore similar work

Robust Recommendation from Noisy Implicit Feedback: A GMM-Weighted Bayes-label Transition Matrix Framework
May 20, 2026Zongyu Li, Xuanyu Liu, Gongce Cao +3Noisy Labels Gaussian Mixture Models
Guangdong University of Technology, Guangzhou, China · University of Chinese Academy of Sciences, Beijing, China · Capital Normal University, Dalian, China +2
Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction
Aug 30, 2026João L. P. Santana, Filipe R. CordeiroNoisy Labels Machine Unlearning
Department of Computing Universidade Federal Rural de Pernambuco, Brazil
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Active Learning Label Propagation
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company

May 20, 2026, cs.LGJ/K move · Enter open · S save

Robust Recommendation from Noisy Implicit Feedback: A GMM-Weighted Bayes-label Transition Matrix Framework

Zongyu Li, Xuanyu Liu, Gongce Cao, Shirui Sun+2

Guangdong University of Technology, Guangzhou, China · University of Chinese Academy of Sciences, Beijing, China · Capital Normal University, Dalian, China+2

Noisy Labels Gaussian Mixture Models Iterative Denoising Methods Feedback

Open paper AskPDF arXiv

Learning from implicit feedback in recommender systems is fundamentally challenged by pervasive label noise. While conventional denoising approaches often discard noisy instances to ensure robustness, this strategy inevitably suffers from low data utilization. Alternative methods that employ a Bayes-label transition matrix (BLTM) can leverage all available data, but their estimates tend to be biased in practical recommendation scenarios. To address these limitations, this paper proposes a Robust GMM-weighted Bayes-label Transition Matrix framework (RGBT). Our solution utilizes a Gaussian Mixture Model (GMM) to derive instance-specific reliability scores, which systematically calibrate the BLTM estimation to mitigate bias. Theoretical analysis confirms that our approach, by leveraging the BLTM framework with GMM calibration, simultaneously ensures full sample utilization, delivers consistent estimation, and critically, achieves a significant reduction in estimation variance. Extensive experiments on multiple real-world and synthetically flipped datasets demonstrate that RGBT not only utilizes noisy samples more effectively than mainstream reliable sample-based denoising methods, but also achieves significantly superior calibration capability of the transition matrix compared to state-of-the-art transition matrix-based denoising approaches.

Abstract

Explore similar work

Robust Recommendation from Noisy Implicit Feedback: A GMM-Weighted Bayes-label Transition Matrix Framework

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection