Label Noise Cleaning for Supervised Classification via Bernoulli Random Sampling

Yuxin Liu; Xiong Jin; Yang Han

stat.MEMar 15, 2026

Label Noise Cleaning for Supervised Classification via Bernoulli Random Sampling

Authors: Yuxin Liu, Xiong Jin, Yang Han

Organizations: Department of Mathematics, The University of Manchester, UK · Department of Mathematics, The University of Manchester, Oxford Road, Manchester, M13 9PL, UK

Abstract

Label noise - incorrect labels assigned to observations - can substantially degrade the performance of supervised classifiers. This paper proposes a label noise cleaning method based on Bernoulli random sampling. We show that the mean label noise levels of subsets generated by Bernoulli random sampling containing a given observation are identically distributed for all clean observations, and identically distributed, with a different distribution, for all noisy observations. Although the mean label noise levels are not independent across observations, by introducing an independent coupling we further prove that they converge to a mixture of two well-separated distributions corresponding to clean and noisy observations. By establishing a linear model between cross-validated classification errors and label noise levels, we are able to approximate this mixture distribution and thereby separate clean and noisy observations without any prior label information. The proposed method is classifier-agnostic, theoretically justified, and demonstrates strong performance on both simulated and real datasets.

Figures & tables

Figure 1: Empirical distribution of $\{\tilde{l}_{\bm{z}}\}_{\bm{z}\in D}$ obtained from a dataset with $N=1000$ observations ( $N_{1}=800$ clean and $N_{2}=200$ noisy). The values of $\tilde{l}_{\bm{z}}$ are computed under the Bernoulli random sampling scheme with $M=2\times 10^{6}$ subsets and inclusion probability $q=0.4$ . The histogram shows two well-separated laws corresponding to $L_{1}$ and $L_{2}$ , with a mean gap $\mathbb{E}\!\left[\tfrac{1-q}{(N-1)q+1}\right]\approx 0.0015$ .

Figure 2: Empirical distribution of the i.i.d. sequence $\{\tilde{l}^{\prime}_{\bm{z}}\}_{\bm{z}\in D}$ generated under the same settings as Figure 1 . The mixture shows two components, approximately corresponding to $L_{1}$ and $L_{2}$ , with a mean gap of $\tfrac{1-q}{(N-1)q+1}\approx 0.0015$ , consistent with Lemma 2.3 .

Figure 3: Linear model between $\tilde{l}_{\bm{z}}$ and $\tilde{e}_{\bm{z}}$ under Setting 1 with 20% label noise. Results are shown for classifiers trained with RBF SVM and 1-NN, as described in Section 4 .

Figure 4: Comparison between the empirical and model-estimated distributions of $\tilde{l}_{\bm{z}}$ . The estimated distribution $\hat{l}_{\bm{z}}$ closely matches the empirical $\tilde{l}_{\bm{z}}$ , and the estimated cut point $\hat{l}^{*}$ aligns well with the true $l^{*}$ .

Panel of Figure 5 — Figure 5: Three-iteration BRSLC performance of RBF SVM and 1-NN. The legend $<e^{*}$ denotes $\frac{\#\{\bm{z}\in D_{1}\mid\tilde{e}_{\bm{z}}<e^{*}\}}{\#\{\bm{z}\in D\mid\tilde{e}_{\bm{z}}<e^{*}\}}$ , $\geq e^{*}$ denotes $\frac{\#\{\bm{z}\in D_{2}\mid\tilde{e}_{\bm{z}}\geq e^{*}\}}{\#\{\bm{z}\in D\mid\tilde{e}_{\bm{z}}\geq e^{*}\}}$ .

Setting	$\bm{\ell(D)}$ (%)	$\bm{\ell_{\text{cleaned}}}$ (%)	$\bm{r_{\text{noise}}}$ (%)	$\bm{r_{\text{clean}}}$ (%)
1	10	0.54	94.6	96.7
	20	1.18	94.1	93.2
	30	2.62	91.3	92.0
	40	3.40	91.5	88.6
2	10	1.06	89.4	95.7
	20	2.20	89.0	92.6

Table 1: Summary of label noise cleaning performance across four simulated data settings. All values are expressed as percentages.

\ell(D)

: original label noise level;

\ell_{\text{cleaned}}

: residual label noise level after cleaning;

r_{\text{noise}}=100\times[1-\ell_{\text{cleaned}}/\ell(D)]

: proportion of label noise removed;

r_{\text{clean}}=100\times(n_{\text{cleaned}}/n_{\text{clean}})

: proportion of clean observations retained after cleaning.

Panel of Figure 6 — Figure 6: Test error rates of noisy and cleaned datasets across the four simulated data settings. Each panel corresponds to one setting, showing results for three classifiers: RBF SVM, CART, and 1-NN. Solid lines represent performance on after BRSLC cleaning, while dashed lines represent performance of original noisy data.

Panel of Figure 7 — Figure 7: Examples of letters B, H, R in dataset UCI letter recognition, and digits 4, 7, 9 in dataset MNIST digit recognition.

Experiment	Label 0	Label 1	$n_{\text{train}}$	$n_{\text{test}}$
WDBC	Malignant	Benign	400	100
Mushroom	Edible	Poisonous	1000	500
Letter-B	B	H and R	1000	200
Letter-H	H	B and R	1000	200
Letter-R	R	B and H	1000	200
MNIST-4	4	7 and 9	2000	1000

Table 2: Summary of the eight experiments in our study. Each row defines a binary classification task derived from an underlying dataset. For the Letter and MNIST subsets, the total sample sizes for Label 0 and Label 1 follow a 50:50 split. Within Label 1, the two constituent classes are balanced in a 50:50 ratio as well.

$\bm{\ell(D)}$ (%)	Method	Classifier	$\bm{\ell_{\text{cleaned}}}$ (%)	$\bm{r_{\text{clean}}}$ (%)
10	ALNR	Linear SVM	1.1	89.4
		RBF SVM	1.0	86.0
	BRSLC	Linear SVM	1.0	90.1
		RBF SVM	0.5	89.6
20	ALNR	Linear SVM	2.9	85.9
		RBF SVM	2.4	83.0

Table 3: Average results of ALNR and BRSLC across eight datasets under different noise levels

\ell(D)

\ell_{\text{cleaned}}

: residual label noise level after cleaning;

r_{\text{clean}}

: proportion of clean observations retained after cleaning, expressed as a percentage. All values are averaged over datasets and classifiers.

Panel of Figure 8 — Figure 8: Average test error rates of Linear SVM trained on noisy data, and data cleaned by ALNR and BRSLC.

Panel of Figure 9 — Figure 9: Examples of CIFAR-10H soft labels. Top: noisy soft label distributions. Bottom: cleaned soft label distributions obtained after applying the proposed method.

$n$	$n_{\text{cleaned}}$	$l_{cleaned}$
500	484.37	0.0085
3500	3285.89	0.0089
6500	6211.01	0.0086
9500	8852.58	0.0083

Table 4: Averaged ResNet-18 convolutional neural network label cleaning results over 20 random subsets from CIFAR-10H of sizes 500, 3500, 6500, and 9500.

n

is the size of the noisy subset,

n_{\text{cleaned}}

is the size after cleaning,

l_{\text{cleaned}}

is the purified noise rate.

Panel of Figure 10 — Figure 10: Comparison between the noisy data and the adaptive method proposed by M. Sesia in terms of coverage and size, computed over 10 CIFAR-10H random calibration sets of size 500–9500.

Learning with Noisy Labels

Learning with noisy labels studies how to train reliable models when annotations are incorrect, incomplete, subjective, or weakly supervised.

See all 150 papers

Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning
Aug 4, 2026Wenxiao Fan, Kan LiInstance Reweighting Noisy-Label Learning
School of Computer Science, Beijing Institute of Technology
An Adaptive Data cleaning Framework for Noisy Label Detection
Jun 5, 2026Chen-Hsuan Fang, Wei-Hsinag Chen, Pin-Hsuan Yu +2Noisy-Label Learning Learning with Noisy Labels
Department of Electrical Eng. National Taiwan Ocean University, Keelung City 20224, Taiwan · AI Research Center, National Taiwan Ocean University, Keelung City 20224, Taiwan
LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling
Aug 4, 2026Abhishek Moturu, Babak Taati, Anna GoldenbergMedical Image Classification Noisy-Label Learning
Department of Computer Science University of Toronto The Hospital for Sick Children UHN KITE Research Institute T-CAIREM Vector Institute · Department of Computer Science Institute of Biomedical Engineering University of Toronto Rehabilitation Sciences Institute UHN KITE Research Institute Vector Institute · Department of Computer Science Department of Laboratory Medicine and Pathobiology University of Toronto The Hospital for Sick Children T-CAIREM Vector Institute

Aug 4, 2026J/K move · Enter open · S save

Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning

Wenxiao Fan, Kan Li

School of Computer Science, Beijing Institute of Technology

Instance Reweighting Noisy-Label Learning Pseudo-Labeling

Open paper AskPDF arXiv

Refurbishment-based noisy-label learning mixes an observed label with a model-derived pseudo target, typically using one sample-wise cleanliness score to control both branches. This creates a hidden coupling: reducing trust in the observed label automatically increases trust in the pseudo target. We show that this complementarity can replace one unreliable signal with another because a pseudo target learned from corrupted supervision may reproduce the noise it is meant to correct. Our representation diagnostics provide a consistent account of this mismatch: noisy supervision redirects deeper layers more strongly, whereas shallower relations remain comparatively stable and provide information beyond the loss posterior. We therefore propose TRACE, a Two-Source Reliability Assessment framework for Label Correction and Sample Reweighting. TRACE assesses the observed label using loss fit, shallow relation stability, and prediction agreement, while separately assessing the pseudo target using model confidence. Its source-specific scores control target correction and supervision strength without assuming complementary reliability. Across synthetic and real-world noisy benchmarks, TRACE improves representative refurbishment baselines and yields more reliable pseudo supervision.

Abstract

Learning with Noisy Labels

Explore similar work

Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning

An Adaptive Data cleaning Framework for Noisy Label Detection

LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling