Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learning

Chen Shu; Boyu Fu; Yiman Li; Ting Yin; Wenchuan Zhang; Jie Chen; Yuhao Yi; Hong Bu

cs.CVMay 17, 2025

Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learning

Authors: Chen Shu, Boyu Fu, Yiman Li, Ting Yin, Wenchuan Zhang, Jie Chen, Yuhao Yi, Hong Bu

Organizations: College of Computer Science, Sichuan University · Sichuan University-Pittsburgh Institute, Sichuan University · Department of Pathology West China Hospital, Sichuan University · Institute of Clinical Pathology, West China Hospital, Sichuan University

Abstract

Multiple Instance Learning is the predominant method for Whole Slide Image classification in digital pathology, enabling the use of slide-level labels to supervise model training. Although MIL eliminates the tedious fine-grained annotation process for supervised learning, whether it can learn accurate bag- and instance-level classifiers remains a question. To address the issue, instance-level classifiers and instance masks were incorporated to ground the prediction on supporting patches. These methods, while practically improving the performance of MIL methods, may potentially introduce noisy labels. We propose to bridge the gap between commonly used MIL and fully supervised learning by augmenting both the bag- and instance-level learning processes with pseudo-label correction capabilities elicited from weak to strong generalization techniques. The proposed algorithm improves the performance of dual-level MIL algorithms on both bag- and instance-level predictions. Experiments on public pathology datasets showcase the advantage of the proposed methods.

Figures & tables

Figure 1: Framework overview of the proposed method.

Methods	C16				TCGA-NSCLC
Methods	Bag		Instance		Bag
	AUC	ACC	AUC	ACC	AUC	ACC
Fully supervised	0.8961	0.8915	0.9759	0.9584	–	–
MaxPooling	0.6128	0.7083	0.6857	0.7244	0.9289	0.8462
MeanPooling	0.6012	0.6927	0.8637	0.8746	0.8770	0.7848
ABMIL	0.8508	0.8372	0.8067	0.7341	0.9261	0.8618

Table 1: Results on the CAMELYON16 and TCGA-NSCLC dataset

Table 3

Panel of Figure 2 — Figure 2: Visualization of instance-level prediction results. (a) shows one original CAMELYON16 slide thumbnail with ground-truth positive instances masked in green. The following are heatmaps overlapped on the slide image where deeper red indicates higher predicted probability of being positive and deeper blue indicates lower probability of being positive.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Methods	C16 (ResNet-18)		C16 (CONCH)		TCGA-NSCLC (PLIP)
	Bag	Instance	Bag	Instance	Bag
Fully supervised	0.8577	0.9684	0.9703	0.9922	–
MaxPooling	0.6527	0.6471	0.9823	0.7264	0.9428
MeanPooling	0.6309	0.8416	0.7753	0.5280	0.9286
ABMIL	0.7913	0.7290	0.9809	0.7925	0.9476
DSMIL	0.7591	0.9163	0.9809	0.9495	0.9383

Appendix

Table 4: Results (AUC) on the CAMELYON16 and TCGA-NSCLC with different pretrained encoders

Panel of Figure 3 — Figure 3: Instance-level AUC on CAMELYON16 test set throughout training.

Model		Bag		Instance
		AUC	ACC	AUC	ACC
without $L_{\text{inst}}$	ABMIL+Ours	0.9211	0.8984	0.9199	0.9147
without $L_{\text{inst}}$	DSMIL+Ours	0.9189	0.8944	0.9107	0.9154
without $L_{\text{self}}$	ABMIL+Ours	0.9134	0.8974	0.9118	0.9149
without $L_{\text{self}}$	DSMIL+Ours	0.9188	0.8803	0.9095	0.8931
without $L_{\text{attn}}$	ABMIL+Ours	0.9042	0.8977	0.8965	0.8752

Appendix

Table 5: Ablation study results on the CAMELYON16 dataset

Multiple Instance Learning

Multiple Instance Learning learns from labeled sets of instances when labels apply to bags, enabling prediction without dense annotations.

See all 98 papers

ProtoPur: Prototype-Purity Pseudo-Labeling for Weakly Supervised Instance Learning in Pathology
Oct 11, 2026Chen Shu, Chengyang Zhang, Boyu Fu +5Weakly Supervised Learning Computational Pathology
College of Computer Science, Sichuan University, Chengdu, China · Institute of Clinical Pathology, West China Hospital, Sichuan University, Chengdu, China · Pittsburgh Institute, Sichuan University, Chengdu, China +1
LRMIL: Efficient Low-Resolution Multiple Instance Learning via High-Resolution Knowledge Distillation for Whole Slide Image Classification
Jun 5, 2026Yonghan Shin, Won-Ki JeongHistopathology Image Classification Knowledge Distillation
Department of Computer Science and Engineering, Korea University, Seoul, Korea
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
Jul 16, 2026Mingxi Fu, Jiawen Li, Renao Yan +4Multi-Teacher Knowledge Distillation Multiple Instance Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University · University of Washington · City University of Hong Kong (Dongguan) +2

Oct 11, 2026J/K move · Enter open · S save

ProtoPur: Prototype-Purity Pseudo-Labeling for Weakly Supervised Instance Learning in Pathology

Chen Shu, Chengyang Zhang, Boyu Fu, Ting Yin+4

College of Computer Science, Sichuan University, Chengdu, China · Institute of Clinical Pathology, West China Hospital, Sichuan University, Chengdu, China · Pittsburgh Institute, Sichuan University, Chengdu, China+1

Weakly Supervised Learning Computational Pathology Pseudo-Labeling Multiple Instance Learning Whole-Slide Image Analysis Prototype-Based Learning

Open paper AskPDF arXiv

Weak bag-level labels are widely available in computational pathology, whereas reliable instance-level annotations for whole-slide images remain expensive and ambiguous. Weakly supervised multiple instance learning (MIL) achieves strong bag-level classification, but instance-level supervision from attention scores, model predictions, negative bags, or spatial refinement may fail to identify class-specific local evidence. We propose ProtoPur, a prototype-purity pseudo-labeling method for weakly supervised instance-level learning in pathology whole-slide images. ProtoPur estimates class-balanced local purity over prototype regions constructed from frozen foundation-model features, and assigns pseudo-labels to high-purity regions strongly associated with a target class. This purity-first design separates class-specific evidence from negative or shared morphology, with coverage used only to ensure sufficient pseudo-label assignment. We provide a PAC-style analysis showing that the risk of an instance classifier trained with ProtoPur pseudo-labels can be decomposed into a standard uniform-convergence term and a ProtoPur-specific pseudo-label noise term, comprising selected-region mismatch and a coverage residual. Experiments on three public whole-slide image datasets with two pathology foundation encoders show that ProtoPur consistently improves instance-level performance across multiple MIL backbones while maintaining competitive bag-level performance. Pseudo-label quality comparisons on two public datasets further demonstrate that prototype-purity selection provides stronger instance supervision. These results suggest that the foundation-model features contain exploitable instance-level supervision signals, and ProtoPur is a way to reveal and use these signals from weak bag labels without requiring dense instance annotations.

Abstract

Multiple Instance Learning

Explore similar work

ProtoPur: Prototype-Purity Pseudo-Labeling for Weakly Supervised Instance Learning in Pathology

LRMIL: Efficient Low-Resolution Multiple Instance Learning via High-Resolution Knowledge Distillation for Whole Slide Image Classification

Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models