When Noise Meets Long-Tail: Feature-Threshold Dual Calibration for Robust Pseudo-Labeling

Ping Guo; Zhiqi Huang; Xinran Li

cs.CVSep 27, 2026

When Noise Meets Long-Tail: Feature-Threshold Dual Calibration for Robust Pseudo-Labeling

Authors: Ping Guo, Zhiqi Huang, Xinran Li

Organizations: Graduate School of Information, Production and Systems, Waseda University, Kitakyushu, Japan · School of Software, Dalian University of Technology, Dalian, China · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China

Abstract

Pseudo-labeling has become a cornerstone of learning from unlabeled data in semantic segmentation. Yet its effectiveness drops sharply in real-world scenarios where strong imaging noise and long-tailed class distributions occur together. We trace this failure to a vicious cycle of pseudo-label degradation. Imaging noise entangles foreground and background features, lowering prediction confidence across all classes, while long-tailed distributions leave tail classes with far fewer training samples and inherently lower confidence. Under fixed high-threshold filtering, these tail-class predictions are systematically filtered out, so they receive no supervision from unlabeled data and thus features keep degrading in subsequent iterations. Critically, noise and long-tail are not independent obstacles but mutually amplifying ones, and addressing either alone is insufficient. To break this cycle, we propose FTC-Seg, a Feature-Threshold dual-Calibration framework built on a standard teacher-student framework. At the feature level, Orthogonal Prototype Reconstruction (OPR) uses a set of learnable orthogonal prototypes to residually purify pixel-wise features, widening the margin between weak foreground targets and noisy backgrounds. At the threshold level, Adaptive Threshold Calibration (ATC) dynamically adjusts class-specific thresholds based on learning difficulty and prediction-distribution bias, rescuing low-confidence pseudo-labels of tail classes from systematic exclusion. Extensive experiments on four public benchmarks spanning three distinct noise modalities show that FTC-Seg achieves strong performance against state-of-the-art methods, with particularly substantial gains on tail classes. Our results establish that jointly calibrating features and thresholds is essential for robust pseudo-labeling under compounded noise and class imbalance.

Figures & tables

Figure 1 : Illustration of the pseudo-label degradation crisis and our FTC-Seg motivation. (a) Traditional fixed high-threshold strategies are severely disturbed by speckle noise, systematically filtering out low-confidence predictions of long-tailed targets such as standing bottle. (b) At the feature level, the Orthogonal Prototype Reconstruction (OPR) enlarges the margin between target and background noise in the feature space. (c) At the threshold level, the Adaptive Threshold Calibration (ATC) dynamically lowers the threshold for rare classes, successfully rescuing long-tailed targets.

Figure 2 : Controlled decoupling analysis of tail-class pseudo-label filtering. OB/OL use original images and CB/CL apply domain-aligned synthetic corruption (multiplicative speckle for FLSMD, underwater scattering for SUIM, fog for ACDC) to the same image lists, while balanced subsets are obtained by sub-sampling head classes to match tail-class frequencies; masks are unchanged across conditions. The highest filtered-rate under CL across all three modalities indicates that corruption and long-tail jointly aggravate tail-class exclusion.

Figure 3 : Overview of FTC-Seg framework. (a) Teacher-student architecture with OPR inserted between encoder and decoder, and ATC applied before pseudo-label filtering. (b) OPR computes similarities between features and learnable orthogonal prototypes, then aggregates foreground prototypes via a residual connection. (c) ATC combines a difficulty factor $\mathcal{M}_{\text{diff}}$ and a bias factor $\mathcal{M}_{\text{bias}}$ to produce class-specific thresholds.

Method	Venue	Encoder	FLSMD			FSSG
Method	Venue	Encoder	2%	5%	10%	2%	5%	10%
Labeled Only	–	RN-101	46.47 $\pm$ 1.17	55.15 $\pm$ 0.47	61.94 $\pm$ 0.83	17.64 $\pm$ 1.35	32.23 $\pm$ 0.66	52.27 $\pm$ 1.01
Labeled Only	–	DINOv2-S	51.08 $\pm$ 0.50	61.02 $\pm$ 0.95	68.48 $\pm$ 0.70	35.61 $\pm$ 0.89	41.67 $\pm$ 1.42	54.58 $\pm$ 0.32
AEL [ 10 ]	NIPS’21	RN-101	52.70 $\pm$ 1.54	64.89 $\pm$ 1.24	70.73 $\pm$ 0.88	53.51 $\pm$ 1.02	57.84 $\pm$ 0.98	63.08 $\pm$ 0.46
Dual Teacher [ 15 ]	NIPS’23	MiT-B2	56.23 $\pm$ 0.70	66.09 $\pm$ 0.56	69.61 $\pm$ 0.71	58.31 $\pm$ 0.80	60.16 $\pm$ 0.93	65.91 $\pm$ 1.01
UniMatch [ 25 ]	CVPR’23	RN-101	50.70 $\pm$ 1.47	56.14 $\pm$ 0.68	61.66 $\pm$ 1.34	58.39 $\pm$ 0.88	60.87 $\pm$ 0.70	63.15 $\pm$ 1.27

Table 1 : Quantitative comparison of mIoU (%) on the FLSMD and FSSG datasets. The best results are shown in bold , and the second best results are shown in italic . Results of FTC-Seg and all comparative methods are averaged over 3 random seeds (±std).

Figure 4 : Performance comparison on the rarest or second-rarest long-tailed categories from the FLSMD, FSSG, SUIM and ACDC datasets under the 2% labeled-data setting.

Method	Encoder	SUIM			ACDC
Method	Encoder	2%	5%	10%	2%	5%	10%
Labeled Only	RN-101	35.64 $\pm$ 1.44	45.47 $\pm$ 1.52	52.68 $\pm$ 0.97	36.62 $\pm$ 1.13	44.43 $\pm$ 0.91	48.47 $\pm$ 1.27
Labeled Only	DINOv2-S	56.61 $\pm$ 0.80	62.56 $\pm$ 1.29	64.26 $\pm$ 1.15	52.60 $\pm$ 0.45	56.57 $\pm$ 0.62	66.98 $\pm$ 0.97
UniMatch	RN-101	47.86 $\pm$ 2.17	49.17 $\pm$ 1.98	55.01 $\pm$ 1.71	45.09 $\pm$ 2.36	50.81 $\pm$ 1.49	51.66 $\pm$ 1.20
SemiVL	RN-101	55.99 $\pm$ 0.79	58.43 $\pm$ 0.60	60.15 $\pm$ 1.32	45.13 $\pm$ 0.57	50.65 $\pm$ 0.71	52.97 $\pm$ 0.97
FTC-Seg (Ours)	RN-101	54.71 $\pm$ 0.53	59.05 $\pm$ 1.46	62.85 $\pm$ 0.97	45.56 $\pm$ 1.35	51.80 $\pm$ 0.89	55.73 $\pm$ 1.04

Table 2 : Quantitative comparison of mIoU (%) on the SUIM and ACDC datasets.The best results are shown in bold , and the second best results are shown in italic . Results of FTC-Seg and all comparative methods are averaged over 3 random seeds (±std).

Method	Component			Pseudo-Label Acc. (%)		Global
Method	Res.	$\mathcal{L}_{o}$	$r_{k}$	Global	Tail	mIoU
Baseline	–	–	–	67.03	56.56	58.75
w/o Residual	✗	✓	✓	68.69	89.61	56.93
w/o $\mathcal{L}_{o}$	✓	✗	✓	71.30	81.04	59.26
w/o $r_{k}$	✓	✓	✗	64.32	83.16	59.84
OPR	✓	✓	✓	72.33	87.80	62.55

Table 3 : Ablation studies of the proposed OPR and ATC components.

Method	Pseudo-Label Acc (%)			Filtered-Rate (%)			Correctly Kept GT Rate (%)
Method	Global	Head	Tail	Global	Head	Tail	Global	Head	Tail
Baseline	67.03	73.06	56.56	13.15	9.50	17.83	58.22	67.04	46.47
+ OPR	72.33	74.08	87.80	9.04	7.04	11.24	63.98	71.34	77.93
+ ATC	69.73	79.60	88.65	11.63	10.38	10.39	64.80	68.13	79.44
FTC-Seg (ours)	73.83	84.13	89.03	8.26	6.75	8.43	67.15	78.21	80.61

Table 4 : Component-wise pseudo-label analysis on global, head, and tail classes.

OPR Layers	mIoU	Params (M)	FLOPs (G)	Infer. Time (ms)	Peak Mem. (MB)
0 (Baseline)	58.75	24.79	56.88	12.97	14279
1	61.92	24.80	56.90	13.15	14348
2	62.55	24.81	56.92	13.27	14398
3	61.47	24.82	56.93	13.43	14449
4	60.98	24.84	56.95	13.58	14497

Table 5 : Ablation studies of OPR layers and FTC-Seg components.

Prototypes 1–4	Prototypes 5–8	Prototypes 9–12	Prototypes 13–16
$\mu_{1}$ : SP $\to$ SP $\to$ TI $P{=}14.8,\ F{=}63.6,\ H{=}99.9$	$\mu_{5}$ : CC $\to$ CC $\to$ DV $P{=}16.1,\ F{=}39.8,\ H{=}99.6$	$\mu_{9}$ : CC $\to$ UR $\to$ CC $P{=}13.6,\ F{=}56.4,\ H{=}99.8$	$\mu_{13}$ : SP $\to$ SP $\to$ CG $P{=}15.3,\ F{=}53.2,\ H{=}99.8$
$\mu_{2}$ : CC $\to$ DV $\to$ BL $P{=}13.2,\ F{=}53.6,\ H{=}99.8$	$\mu_{6}$ : SF $\to$ SF $\to$ UR $P{=}18.0,\ F{=}79.2,\ H{=}99.7$	$\mu_{10}$ : SF $\to$ SF $\to$ PO $P{=}16.5,\ F{=}33.9,\ H{=}99.4$	$\mu_{14}$ : PO $\to$ PO $\to$ CC $P{=}17.1,\ F{=}60.8,\ H{=}99.7$
$\mu_{3}$ : SQ $\to$ UR $\to$ DV $P{=}14.9,\ F{=}29.7,\ H{=}99.2$	$\mu_{7}$ : PO $\to$ PO $\to$ DV $P{=}15.4,\ F{=}43.0,\ H{=}99.6$	$\mu_{11}$ : DV $\to$ DV $\to$ PO $P{=}14.0,\ F{=}34.2,\ H{=}99.8$	$\mu_{15}$ : SF $\to$ TI $\to$ CC $P{=}12.7,\ F{=}51.6,\ H{=}99.9$
$\mu_{4}$ : CC $\to$ BL $\to$ CC $P{=}17.6,\ F{=}66.4,\ H{=}99.6$	$\mu_{8}$ : PO $\to$ PO $\to$ SP $P{=}16.4,\ F{=}76.4,\ H{=}99.7$	$\mu_{12}$ : SQ $\to$ SQ $\to$ PO $P{=}14.4,\ F{=}37.7,\ H{=}99.4$	$\mu_{16}$ : TI $\to$ TI $\to$ PO $P{=}13.5,\ F{=}54.3,\ H{=}99.9$

Table 6 : Dominant-class responses of 16 OPR prototypes on FSSG with 2% labels. BL/CG/SF/CC/SP/PO/SQ/TI/UR/DV denote Block/Circle Cage/Steel Frame/Concrete Column/Steel Plate/Pot/SquareCage/Tire/Underwater Robot/Diver.

Panel of Figure 5 — Figure 5 : Qualitative segmentation results compared with state-of-the-art methods on the FSSG and FLSMD datasets with 2% labeled data.

Panel of Figure 6 — Figure 6 : Qualitative segmentation results compared with state-of-the-art methods on the SUIM and ACDC datasets with 2% labeled data.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Labels	M3	M4	M5	$\Delta$	Holm $p$
FLSMD	2%	59.80	60.17	63.07	+2.90	0.0436
	5%	66.72	66.49	68.84	+2.12	0.0225
	10%	71.24	70.99	73.49	+2.25	0.0356
FSSG	2%	60.87	61.24	63.20	+1.96	0.0448
	5%	62.73	62.46	66.31	+3.58	0.0160
	10%	63.97	64.16	68.01	+3.85	0.0086

Appendix

Table 7 : Joint calibration across four datasets and three labeled-data ratios. M3 uses EMA+OPR, M4 uses EMA+ATC, and M5 uses EMA+OPR+ATC. Results are mean mIoU (%) over three matched seeds shared by M3–M5 within each setting; these seeds differ from those used in the main comparison tables.

\Delta=\mathrm{M5}-\max(\mathrm{M3},\mathrm{M4})

is measured in percentage points. Two-sided paired

t

-tests compare M5 against the stronger individual module selected by mean mIoU. Reported

p

-values are Holm-adjusted across the 12 settings;

\dagger

denotes

p\geq 0.05

Method	mIoU	Tail mIoU	Pseudo-label mIoU
EMA baseline	58.78	5.33	53.49
Original OPR	62.55	17.19	73.86
Class-anchored OPR	54.87	8.93	58.25

Appendix

Table 8 : Prototype formulation comparison on FSSG with 2% labels using the same seed. Original OPR uses 16 learnable prototypes without predefined class identities. Class-anchored OPR uses one prototype per class, initialized from labeled class-mean features and assigned a fixed class identity. Tail mIoU measures segmentation on rare classes; pseudo-label mIoU evaluates accepted pseudo-labels against ground truth on unlabeled training pixels. All metrics are percentages.

Setting	$\mathcal{C}_{fg}$	mIoU	$\Delta$	Diver IoU
Correct	$\{1,\ldots,10\}$	62.55	–	17.50
Add background	$\{0,\ldots,10\}$	59.80	$-2.75$	10.11
Omit Diver	$\{1,\ldots,9\}$	58.42	$-4.13$	11.49
Add background + omit Diver	$\{0,\ldots,9\}$	53.09	$-9.46$	2.47

Appendix

Table 9 : Sensitivity to misspecifying the foreground set

\mathcal{C}_{fg}

on FSSG with 2% labels. All variants use EMA+OPR without ATC, a fixed pseudo-label threshold of 0.95, and the same seed. Background and Diver have category IDs 0 and 10, respectively. Only the gating set changes; segmentation output classes remain unchanged. IoU values are percentages, and

\Delta

denotes the mIoU change from the correct setting in percentage points.

Dataset / corruption	Mild: parameter / $I$	Medium: parameter / $I$	Severe: parameter / $I$
FLSMD / speckle	$\sigma=0.15$ / $+0.89$	$\sigma=0.35$ / $+1.83$	$\sigma=0.60$ / $+2.92$
SUIM / scattering	$t=0.80$ / $+1.04$	$t=0.60$ / $+1.56$	$t=0.40$ / $+1.22$
ACDC / fog	$(a,b)=(1.5,2.0)$ / $-0.11$	$(a,b)=(2.5,1.7)$ / $+1.55$	$(a,b)=(3.0,1.4)$ / $+2.23$

Appendix

Table 10 : Sensitivity of the noise–long-tail interaction to corruption severity. Each cell reports the corruption parameter and interaction

I=CL-OL-CB+OB

in percentage points. OB, OL, CB, and CL denote tail-class pseudo-label filtering rates under original-balanced, original-long-tail, corrupted-balanced, and corrupted-long-tail conditions, respectively. Positive

I

indicates super-additive exclusion. Larger speckle

\sigma

, smaller scattering transmission

t

, and larger

a

with smaller

b

for fog indicate stronger corruption. The checkpoint, image lists, class definitions, threshold, and random seed are held fixed across severity levels.

Method	Long-tail only	Long-tail + noise	Noise-induced change
Baseline	70.76	63.18	$-7.58$
FTC-Seg	75.36	69.11	$-6.25$
FTC-Seg gain	$+4.60$	$+5.93$	–

Appendix

Table 11 : Single-seed controlled evaluation on Pascal VOC using the public 1/16 semi-supervised split with 155 labeled and 5,507 unlabeled training images. A long-tailed distribution with imbalance ratio 20 is constructed, and Gaussian noise with

\sigma=0.18

is applied to all labeled and unlabeled training images in the noisy condition. Both conditions use the same training split and unchanged official validation set. Results are validation mIoU (%). Noise-induced change is the noisy result minus the long-tail-only result; gains and changes are reported in percentage points.

Method	Params (M)	FLOPs (G)	FPS	mIoU (%)
Method comparison
Labeled only	24.79	56.94	122.62	35.61
UniMatch V2	24.79	56.94	122.62	58.78
FTC-Seg	24.81	56.96	120.93	63.20
Component ablation
M2: EMA	24.79	56.88	122.62	58.75

Appendix

Table 12 : Inference cost of compared methods and individual modules with DINOv2-S and a DPT head. Parameters and FLOPs are measured for a single

1\times 3\times 518\times 518

input. FPS is measured after warm-up on an RTX 4090. The upper block reports method comparisons; the lower block reports component ablations. FSSG results with 2% labels are included as performance references. ATC operates during training and adds no inference-time computation.

Figure A1 : Sensitivity analysis of key hyperparameters in FTC-Seg framework on the FLSMD dataset with 2% labeled data.

Explore similar work

CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels
May 22, 2026Mengke Li, Haiquan Ling, Lihao Chen +3Noisy Labels Vision-Language Model Adaptation
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China
Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label
Apr 25, 2026Mengke Li, Haiquan Ling, Yiqun Zhang +2Noisy Labels Recent Vision-Language Models
CSSE, Shenzhen University · SCST, Guangdong University of Technology · INFORMATICS, Xiamen University
ThreshGuide: Class-Aware Labeled-Guided Thresholding for Semi-Supervised 3D Abdominal Multi-Organ Segmentation
Sep 14, 2026Hongyu Liu, Yinlong Wang, Lusha Li +13D Medical Image Segmentation Semi-Supervised Medical Image Segmentation
School of Intelligent Science and Technology, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China

May 22, 2026, cs.CVJ/K move · Enter open · S save

CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels

Mengke Li, Haiquan Ling, Lihao Chen, Yang Lu+2

College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

Noisy Labels Vision-Language Model Adaptation Rectification Long-Tailed Distribution

Open paper AskPDF arXiv

Learning from real-world data is frequently hindered by the compound challenge of long-tailed class distributions and noisy annotations. Existing methods partially address these issues but typically ignore the non-uniform impact of label noise across classes, resulting in ineffective correction for tail classes and over-regularization for head classes. To address this issue, we propose Class-Adaptive Rectification with Experts (CARE), a parameter-efficient framework that leverages three complementary supervision sources from vision-language models (VLM): observed noisy labels, VLM text embeddings, and visual features. CARE introduces a class-adaptive expert consensus mechanism that enforces stricter agreement for tail classes and more permissive agreement for head classes based on class frequency. By aggregating high-confidence predictions across these sources, CARE filters unreliable signals and recalibrates class distributions, yielding more reliable rectification under long-tailed distributions. Extensive experiments on both synthetic and real-world benchmarks demonstrate that CARE consistently outperforms state-of-the-art methods, achieving up to 3.0% performance gains. The source code is available at https://github.com/qwq123-study/CARE.

Abstract

Explore similar work

CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels

Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label

ThreshGuide: Class-Aware Labeled-Guided Thresholding for Semi-Supervised 3D Abdominal Multi-Organ Segmentation