Calibrated Disambiguation for Partial Multi-label Learning

Zhuoming Li; Yuheng Jia; Mi Yu; Zicong Miao

cs.LGOct 13, 2026

Calibrated Disambiguation for Partial Multi-label Learning

Authors: Zhuoming Li, Yuheng Jia, Mi Yu, Zicong Miao

Organizations: School of Computer Science and Engineering, Southeast University · China Telecom Cloud Computing Corporation, Beijing 100088, China

Abstract

Partial multi-label learning (PML) aims to train a classifier on datasets whose instances are over-annotated with not only relevant labels but also irrelevant labels, which is common when datasets are collected from crowd-sourcing platform. Existing works primarily approach it from a curriculum learning perspective, leveraging the memorization effect to disambiguate noisy labels and produce robust predictions. However, these methods are based on non-adaptive weighting functions and lack theoretical guidance for optimal weighting. To overcome these issues, a calibrated disambiguation model named PML-CD is proposed. We first formulate the optimal weighting function for curriculum-based disambiguation, which is equivalent to the calibration of the model's predicted confidences, thus providing guidance for curriculum designing. To obtain the optimal weighting function from PML dataset during the training, a transferable calibrator is designed, which takes the histogram of positive samples' confidences as input, and outputs the optimal curriculum weighting for training. Prototype alignment regularization is also proposed to promote the model's performance. Experiments conducted on Pascal VOC, MS-COCO, NUS-WIDE and CUB have verified that our method outperforms existing state-of-the-art PML methods.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of Calibrated Disambiguation. (a) shows the weighting function proposed in previous curriculum learning approaches, whose shape are heuristically designed; (b) is two histograms of predicted confidences on clean partial labels and corrupted partial samples. The optimal weighting curve, which is equivalent to the calibration curve, can be derived from the frequency statistics if the ground-truth information is given; (c) shows that the optimal weighting curve can be precisely predicted according to the histogram of predicted confidences on partial labels, perceive a general relations between confidences histogram and calibration curve according to the memorization effect.

Figure 2: An example of optimal curriculum weighting with histograms of model’s predicted confidences (left) and weights (right) by assigning

w_{ij}=P(y_{ij}=\tilde{y}_{ij}\mid t_{ij})

. It is shown that weighting is equivalent to calibrating model’s predicted confidences. The expected calibration error (ECE) of confidences and weights are presented.

Figure 3: Illustration of the proposed model PML-CD. During the training, the model is updated by both classification loss and prototype alignment loss. Predicted confidences on training set are collected and are fed into the calibrator to generate reliability of candidate labels, which guides the curriculum disambiguation training in the next epoch.

	VOC				COCO				NUS-WIDE				CUB
Method	20%	40%	60%	80%	20%	40%	60%	80%	20%	40%	60%	80%	20%	40%	60%	80%
PML-NI	$62.57$	$35.37$	$20.97$	$13.84$	$60.14$	$53.41$	$43.02$	$24.52$	$21.26$	$16.39$	$12.02$	$\hphantom{0}7.89$	$12.92$	$12.04$	$11.42$	$10.96$
PML-LRS	$79.96$	$63.45$	$32.86$	$16.66$	$61.56$	$53.80$	$42.85$	$24.23$	$21.52$	$16.23$	$11.85$	$\hphantom{0}7.76$	$14.56$	$12.91$	$11.83$	$11.19$
PLAIN	$80.76$	$72.45$	$54.59$	$27.24$	$60.13$	$53.71$	$44.99$	$21.56$	$19.22$	$17.85$	$13.72$	$\hphantom{0}7.44$	$14.98$	$11.38$	$10.55$	$10.37$
BCE	$87.73$	$84.26$	$75.27$	$48.86$	$73.66$	$69.62$	$63.52$	$50.47$	$50.81$	$45.78$	$37.90$	$27.06$	$29.66$	$27.01$	$22.19$	$17.39$
ASL	$88.63$	$84.43$	$77.23$	$52.95$	$73.19$	$68.53$	$62.56$	$50.46$	$46.27$	$37.86$	$31.67$	$21.97$	$28.58$	$25.75$	$22.66$	$17.35$

Table 1: Comparison of our model PML-CD with baseline methods, with noise rate

20\%

40\%

60\%

and

80\%

. mAP(

\%

) is adopted as metric. The best performances on each column are highlighted with background color.

weighting policy	ECE of confidences (epoch 1)	ECE of weights (epoch 1)	mAP per class (overall)
hard	$76.78$	$92.96$	$59.79$
soft	$76.78$	$61.68$	$60.08$
identity	$76.78$	$76.78$	$59.93$
calibrator	$76.78$	$\hphantom{0}3.74$	$61.48$
optimal	$76.78$	$\hphantom{0}3.74$	$66.42$

Table 2: Effectiveness of weighting policies on MS-COCO with noise rate

80\%

. ECE(

\%

) and mAP(

\%

) are presented.

		noise rate
dataset	$\mathcal{L}_{\text{proto}}$	$20\%$	$40\%$	$60\%$	$80\%$
VOC	neither	$89.28$	$87.93$	$85.05$	$66.91$
	2nd-phase only	$88.77$	$88.12$	$85.07$	$71.70$
	1st & 2nd-phase	$87.89$	$86.11$	$83.90$	$71.40$
COCO	neither	$75.54$	$73.86$	$69.78$	$60.13$
	2nd-phase only	$75.26$	$74.65$	$69.88$	$62.10$

Table 3: Ablation study of the prototype alignment regularization. mAP(

\%

) is adopted as metric.

Multi-Label Classification

Multi-label classification assigns several potentially related tags to each example, from medical findings and topics to objects and user interests.

See all 135 papers

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning
Dec 19, 2025Wei Tang, Yin-Fang Yang, Weijia Zhang +1Model Calibration Weakly Supervised Learning
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China, and the Key Laboratory of Computer Network and Information Integration (Southeast University), MoE, China · School of Computer and Information Sciences, The University of Newcastle, Callaghan, NSW 2308, Australia
Feature-Label Modal Alignment for Robust Partial Multi-Label Learning
Apr 10, 2026Yu Chen, Weijun Lv, Yue Huang +4Multi-Label Classification Learning with Noisy Labels
School of Automation, Guangdong University of Technology, Guangzhou 510006, China · Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen 518055, China · School of Computer Science and Engineering, Sun Yat-Sen University, Guangzhou 510275, China
Bringing Clustering to MLL: Weakly-Supervised Clustering for Partial Multi-Label Learning
Apr 10, 2026Yu Chen, Weijun Lv, Yue Huang +2Multi-Label Classification Learning with Noisy Labels
School of Automation, Guangdong University of Technology

Dec 19, 2025J/K move · Enter open · S save

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning

Wei Tang, Yin-Fang Yang, Weijia Zhang, Min-Ling Zhang

School of Computer Science and Engineering, Southeast University, Nanjing 210096, China, and the Key Laboratory of Computer Network and Information Integration (Southeast University), MoE, China · School of Computer and Information Sciences, The University of Newcastle, Callaghan, NSW 2308, Australia

Model Calibration Weakly Supervised Learning Multiple Instance Learning

Open paper AskPDF arXiv

Multi-instance partial-label learning (MIPL) is a weakly supervised framework that extends the principles of multi-instance learning (MIL) and partial-label learning (PLL) to address the challenges of inexact supervision in both instance and label spaces. However, existing MIPL approaches often suffer from poor calibration, undermining classifier reliability. In this work, we propose a plug-and-play calibratable disambiguation loss (CDL) for classification and calibration, which modulates a disambiguation objective by a top-vs-competitor prediction margin. The competitor is instantiated either as the second strongest candidate label or as the strongest non-candidate label, yielding two variants that respectively emphasize candidate-level separation and candidate-vs-non-candidate suppression. Theoretically, we analyze CDL as a margin-modulated momentum-based disambiguation loss (MDL) objective, derive a lower-bound and a pseudo-label confidence-alignment bound for calibration, and show through gradient and momentum analyses how margin shaping affects weight updates. Experimental results on benchmark and real-world MIPL datasets, together with representative PLL adaptation, confirm that our CDL significantly improves both classification accuracy and expected calibration error.

Abstract

Multi-Label Classification

Explore similar work

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning

Feature-Label Modal Alignment for Robust Partial Multi-Label Learning

Bringing Clustering to MLL: Weakly-Supervised Clustering for Partial Multi-Label Learning