Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images

Shrihari Dumbre; Bikash Santra

cs.CVOct 6, 2026

Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images

Authors: Shrihari Dumbre, Bikash Santra

Organizations: Indian Institute of Technology, Jodhpur

Abstract

Gastric adenocarcinoma (GA) is a leading cause of cancer-related mortality worldwide, and accurate histopathological subtype classification from whole-slide images (WSIs) is essential for effective treatment planning. While multimodal approaches that integrate pathology report text with WSIs can improve classification, existing methods often depend on computationally expensive transformer architectures and large language models. We propose a multimodal knowledge distillation (MKD) framework that combines a pretrained WSI image encoder and a clinical text encoder using Low-Rank Multimodal Fusion (LMF) to efficiently model cross-modal interactions during training. Each WSI is represented as a bag of patches paired with a slide-level diagnostic caption. The teacher model learns fused image-text representations for subtype classification, while the student model distills this knowledge to enable accurate image-only inference. We evaluate our method on the PatchGastric benchmark dataset and achieve at least 3.35% higher mean accuracy than state-of-the-art approaches, without relying on transformer-based fusion, multi-task learning, or large language models. The source code is available at https://github.com/helomelo1/MKD-LMF.

Figures & tables

Figure 1 : Overview of the proposed two-stage MKD-LMF framework. WSI patches are encoded and mean-pooled to obtain a bag-level image representation $(\mathbf{z}_{img})$ , while the corresponding caption is passed through a text encoder to extract text features $(\mathbf{z}_{txt})$ . In Stage 1 , the teacher network employs a Low-Rank Multimodal Fusion (LMF) module to combine $(\mathbf{z}_{img})$ and $(\mathbf{z}_{txt})$ , producing a fused representation $(\mathbf{z}_{fused})$ , which is used to train a linear classifier (shared with the student) for gastric adenocarcinoma (GA) subtype classification. In Stage 2 , the student network takes only $(\mathbf{z}_{img})$ as input and learns, via a linear knowledge distillation module, to approximate $\mathbf{z}_{fused}$ generated by the teacher’s frozen LMF module. The synthesized representation is then passed to the shared classifier to enable image-only GA subtype classification .

Figure 2 : Architecture of the proposed Knowledge Distiller, a residual multi-layer perceptron (MLP), comprising an input projection, stacked residual blocks, and an output projection for mapping image features to the fused feature space.

Method	Accuracy (%)	$\boldsymbol{\Delta}$
ABMIL [ 1 ]	$66.67\pm 2.63$	$-8.16$
CLAM [ 3 ]	$67.83\pm 1.12$	$-7.00$
DSMIL [ 4 ]	$69.02\pm 0.42$	$-5.81$
TransMIL [ 2 ]	$67.48\pm 1.16$	$-7.35$
CITE [ 18 ]	$69.63\pm 0.91$	$-5.20$
ILRA-MIL [ 19 ]	$70.16\pm 1.11$	$-4.67$

Table 1 : Comparison with state-of-the-art (SOTA) methods under image-only inference. Results are reported as mean accuracy (%)

\pm

standard deviation.

\boldsymbol{\Delta}

denotes the difference in mean accuracy of SOTA methods compared to our method (MKD-LMF). Red-colored negative values indicate lower performance than ours, while positive values indicate higher performance.

Study	Configuration	Accuracy (%)
Multimodal Inference	Proposed teacher model	$\mathbf{92.74}$
Distillation Loss	Feature loss ( $\mathcal{L}_{\text{feat}}$ )	$73.63$
	KD loss ( $\mathcal{L}_{\text{KD}}$ )	$73.13$
	Feature loss + KD loss	$\mathbf{74.83}$
Inference Modality	Text-only (unimodal)	$68.76$
	Image-only (unimodal)	$70.35$

Table 2 : Ablation study of the proposed MKD-LMF framework. The results of the experiments in the table are reported for a single run. The table analyzes (i) the performance of the teacher model using both image and text during inference, which is not clinically relevant, (ii) the effect of different distillation loss components in combination with the classification loss (cross-entropy) on the student-side and (iii) the impact of different inference modalities. Results show that combining both losses yields the best student performance. The student model in our multimodal knowledge distillation (KD) framework substantially outperforms unimodal (image-only or text-only) inference settings.

Panel of Figure 3 — Figure 3 : Results for different choices of (a) the coefficient of logit distillation loss, $\lambda_{KD}$ and (b) the rank in the low-rank multimodal fusion (LMF) approach

Panel of Figure 4 — Figure 4 : t-SNE visualization of learned embeddings with corresponding DBI scores: image-only (5.2641), text-only (2.3107), teacher (0.7309), and distilled student (1.1078). Blue represents "moderately differentiated tubular adenocarcinoma". Orange represents "poorly diffrentiated adenocarcinoma". Green represents "well diffrentiated tubular adenocarcinoma"

Explore similar work

MAGE: Color-Invariant and Spatial Knowledge Distillation for Gastric Neoplasm Classification
Jul 14, 2026Jiho Jun, Jeongwon Woo, Jaemin Song +6Colorectal Cancer Image Classification
Korea University, Korea · MEDAI, Korea · Seoul National University, Korea +3
SlideMix: Enhancing Whole Slide Image Analysis via Multimodal Shuffling
Aug 31, 2026Chad Wong, Sicheng Chen, Tianyi Zhang +4Whole-Slide Images Multiple Instance Learning
1Nhu Department of Electrical Engineering and Computer Science, University of California, Irvine · Department of Electrical & Computer Engineering, National University of Singapore · 3PuzzleLogic Pte Ltd, Singapore 229594, Singapore
Masked Feature Encoding for Large-Scale Whole Slide Image Representation
Oct 7, 2026Haoyu He, Basile Tessier-Cloutier, Yang Wang +1Whole-Slide Images Multiple Instance Learning
Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, Canada · McGill University, Montreal, Canada · Mila – Quebec AI Institute, Montreal, Canada

Jul 14, 2026, cs.CVJ/K move · Enter open · S save

MAGE: Color-Invariant and Spatial Knowledge Distillation for Gastric Neoplasm Classification

Jiho Jun, Jeongwon Woo, Jaemin Song, Thanh Bong Nguyen+5

Korea University, Korea · MEDAI, Korea · Seoul National University, Korea+3

Colorectal Cancer Image Classification Dataset Distillation Contrast

Open paper AskPDF arXiv

Accurate differentiation between gastric adenoma and carcinoma during endoscopy is critical for clinical decision-making. Yet, this task is highly challenging due to high inter-class similarity and ambiguous boundaries between the two classes. Existing ROI-based classification methods often suffer from detection/segmentation error propagation and loss of surrounding global context. In contrast, full-image classification lacks the necessary spatial focus. Furthermore, we observe that deep neural networks gravitate towards domain-specific texture biases(e.g. bleeding, lighting artifacts), often causing models to predict based on spurious correlations instead of intrinsic morphological features. To address these limitations, we propose a novel framework, Masked Achromatic Guidance Expert (MAGE). During training, we introduce an auxiliary local expert branch trained on masked achromatic views of the neoplasm. By suppressing background context and color, this branch is forced to learn highly discriminative, purely structural features. We then employ a dual-objective distillation strategy, transferring both classification logits and spatial attention maps to provide implicit spatial supervision to the main branch that receives full WLI as input. This dual-objective distillation forces the model to ground its predictions in morphology rather than relying on shortcuts, while still retaining clinically relevant color cues. At inference time, our deployable model operates on images without annotated masks, ensuring real-time deployability . Extensive experiments on a clinical gastric endoscopy dataset show that our method significantly outperforms existing detection-based methodologies (e.g. YOLO) and classification-based methodologies (e.g. Swin-Transformer), providing not only superior classification performance but also interpretable attention maps for clinical reliability.

Abstract

Explore similar work

MAGE: Color-Invariant and Spatial Knowledge Distillation for Gastric Neoplasm Classification

SlideMix: Enhancing Whole Slide Image Analysis via Multimodal Shuffling

Masked Feature Encoding for Large-Scale Whole Slide Image Representation