MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification

Jiahao Xu; Sheng Huang; Xin Zhang; Zhixiong Nan; Jiajun Dong; Nankun Mu

cs.CVFeb 24, 2026

MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification

Authors: Jiahao Xu, Sheng Huang, Xin Zhang, Zhixiong Nan, Jiajun Dong, Nankun Mu

Organizations: School of Big Data & Software Engineering, Chongqing University · School of Computer Science & Technology, Chongqing University

Abstract

In computational pathology, few-shot whole slide image classification is primarily driven by the extreme scarcity of expert-labeled slides. Recent vision-language methods incorporate textual semantics generated by large language models, but treat these descriptions as static class-level priors that are shared across all samples and lack sample-wise refinement. This limits both the diversity and precision of visual-semantic alignment, hindering generalization under limited supervision. To overcome this, we propose the stochastic MUlti-view Semantic Enhancement (MUSE), a framework that first refines semantic precision via sample-wise adaptation and then enhances semantic richness through retrieval-augmented multi-view generation. Specifically, MUSE introduces Sample-wise Fine-grained Semantic Enhancement (SFSE), which yields a fine-grained semantic prior for each sample through MoE-based adaptive visual-semantic interaction. Guided by this prior, Stochastic Multi-view Model Optimization (SMMO) constructs an LLM-generated knowledge base of diverse pathological descriptions per class, then retrieves and stochastically integrates multiple matched textual views during training. These dynamically selected texts serve as enriched semantic supervisions to stochastically optimize the vision-language model, promoting robustness and mitigating overfitting. Experiments on three benchmark WSI datasets show that MUSE consistently outperforms existing vision-language baselines in few-shot settings, demonstrating that effective few-shot pathology learning requires not only richer semantic sources but also their active and sample-aware semantic optimization. Our code is available at: https://github.com/JiahaoXu-god/CVPR2026_MUSE.

Figures & tables

Figure 1: Comparison of MUSE with existing VLM-based MIL methods. (a) VLM-based MIL methods incorporate pathological text and enable cross-modal interaction between text and image modalities. (b) Our method performs fine-grained modeling of semantics and enables interaction between text and image modalities, while enhancing text diversity through knowledge base retrieval and stochastic optimization.

Figure 2: Overview of the proposed MUSE framework. (DSR: Decompositional Semantic Refinement. SVTI: Sample-wise Vision-Text Interaction) (a) The input semantic information is decomposed and modeled in a fine-grained manner. We then leverage the refined semantic representations to extract sample-relevant visual-semantic information and facilitate cross-modal interaction. (b) The semantic-enhanced features are used to retrieve relevant texts from the multi-view text knowledge base, and these retrieved texts are subsequently leveraged through stochastic optimization to enrich semantic diversity.

Figure 3: Our pipeline of the category related text knowledge base generation. (a) We employ ChatGPT to analyze and decompose the concepts associated with class names. (b) We leverage ChatGPT to construct concrete examples for each aspect. (c) We randomly sample some examples and combine them with prompts to guide the locally deployed LLM in generating the category knowledge base.

Dataset	Model	4-shot			8-shot			16-shot
Dataset	Model	ACC	AUC	F1 Score	ACC	AUC	F1 Score	ACC	AUC	F1 Score
CAMELYON	Mean Pooling	59.07 ±3.03	51.19 ±4.81	47.17 ±5.19	64.05 ±4.14	61.53 ±5.07	55.14 ±2.90	65.83 ±1.96	62.50 ±4.56	55.04 ±7.53
	Max Pooling	62.93 ±4.20	62.50 ±8.88	55.58 ±7.61	70.37 ±5.00	74.63 ±4.38	67.64 ±4.44	74.46 ±6.46	78.38 ±8.12	72.15 ±6.45
	ABMIL [ 22 ]	62.04 ±2.77	58.98 ±7.37	49.61 ±8.85	71.52 ±10.41	75.05 ±11.03	69.69 ±10.15	82.37 ±7.10	85.48 ±6.22	81.04 ±7.36
	DSMIL [ 24 ]	59.44 ±9.00	54.45 ±10.00	53.01 ±9.16	73.19 ±6.93	74.63 ±9.69	70.20 ±8.79	84.72 ±5.11	87.80 ±4.69	83.60 ±5.35
	CLAM-SB [ 30 ]	57.59 ±6.18	55.42 ±9.81	53.67 ±6.09	71.67 ±7.50	73.56 ±10.40	67.16 ±10.43	81.30 ±9.97	84.65 ±10.30	79.91 ±10.34

Table 1: Few-shot weakly supervised learning results (presented in %) on CAMELYON, TCGA-NSCLC, and TCGA-BRCA under 4-shot, 8-shot, and 16-shot settings are presented. The best performance is highlighted in bold , and the second-best is underlined .

	BM	TI	SFSE	SMMO	ACC	AUC	F1 Score
16-shot	$\checkmark$	$\times$	$\times$	$\times$	82.37	85.48	81.04
	$\checkmark$	$\checkmark$	$\times$	$\times$	86.61	90.13	85.41
	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	86.72	90.20	85.33
	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	87.99	90.95	86.70
	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	89.70	92.52	88.59
8-shot	$\checkmark$	$\times$	$\times$	$\times$	71.52	75.05	69.69

Table 2: Ablation experiments (presented in %) of key modules of MUSE on on CAMELYON under few-shot learning settings.(BM: Base MIL. TI: Traditional Interaction.)

	Retrieval Method	ACC	AUC	F1 Score
16-shot	Random	88.17	90.86	87.08
	L2-norm	87.06	90.94	85.93
	Cosine	89.70	92.52	88.59
8-shot	Random	84.20	87.66	82.70
	L2-norm	84.49	88.40	83.07
	Cosine	84.01	88.32	82.42

Table 3: Performance comparison (presented in %) of different retrieval methods in SMMO on CAMELYON under few-shot learning settings.

	Optimization Strategy	ACC	AUC	F1 Score
16-shot	Multi-mean	87.36	90.23	86.23
16-shot	Stochastic	89.70	92.52	88.59
8-shot	Multi-mean	84.08	89.14	81.70
8-shot	Stochastic	84.01	88.32	82.42
4-shot	Multi-mean	70.33	73.53	65.01
4-shot	Stochastic	74.86	76.65	68.66

Table 4: Performance comparison (presented in %) of different optimization strategies in SMMO on CAMELYON under few-shot learning settings.

	Method	ACC	AUC	F1 Score
16-shot	Deepseek-R1-Distill-Qwen-7B	87.65	90.93	86.47
	Llama-3.2-1B	87.17	90.23	85.95
	Qwen2-7B	89.70	92.52	88.59
8-shot	Deepseek-R1-Distill-Qwen-7B	84.94	89.04	83.44
	Llama-3.2-1B	84.20	87.66	82.70
	Qwen2-7B	84.01	88.32	82.42

Table 5: Performance comparison (presented in %) of knowledge bases generated by different locally deployed LLMs under few-shot learning settings.

Figure 4: The visualization analysis of fine-grained semantic priors via SFSE and sample-based retrieval. (a) Visualization of fine-grained semantic in sample embedding space. (b) Comparison of text features retrieved by different samples. (Different colors represent different samples.)

	Category	Description
CAMELYON	Normal	In high-resolution imaging, a normal lymph node exhibits uniform nuclear staining with homogeneous chromatin that lacks coarse granularity or heterogeneity. The vascular distribution is regular, featuring well-organized capillaries and small vessels without any signs of abnormal dilation or neovascularization. There are no non-physiological speckles, such as pigment deposits, debris, or staining artifacts, and the boundaries between different tissue regions show smooth transitions without abrupt changes or protrusions.
	Tumor	In high-resolution imaging of a metastatic lymph node, cellular crowding is evident with densely packed cells often forming clusters or sheets, indicative of significant tumor burden. Concurrently, irregular or distorted vascular structures are observed, showing signs of compression, dilation, or irregular branching of blood vessels, suggesting compromised microvascular integrity. Additionally, increased nuclear basophilia is noted, characterized by darker purple nuclei due to elevated DNA content in the tumor cells, reflecting their proliferative activity. Lastly, diffuse infiltration patterns are present, where tumor cells spread along existing tissue frameworks, displacing normal lymphoid architecture and disrupting the orderly arrangement of lymphocytes and other immune cells.
TCGA-NSCLC	LUAD	Lung adenocarcinoma exhibits a complex morphology that can be systematically described through high-resolution imaging. Notably, some cells display vacuolated or mucinous cytoplasmic changes, indicating alterations in cellular metabolism and possibly reflecting the presence of secretory activity. Additionally, the tumor often features fibrovascular cores within its papillary structures, which provide structural support and potentially contribute to the tumor’s invasive capacity. Angiogenesis is another key feature, characterized by the presence of numerous, thin-walled blood vessels in the tumor stroma, which are crucial for tumor growth and survival. Lastly, there is significant heterogeneity in mucin content, with certain regions producing abundant mucin, while others show little or no production, suggesting diverse microenvironments and potential for different clinical behaviors.
	LUSC	In high-resolution imaging of lung squamous cell carcinoma, key features can be systematically described from four perspectives: Intercellular bridges manifest as visible desmosomal connections between tumor cells, indicating cellular cohesion. Polygonal tumor cells are characterized by well-defined, often angular shapes, reflecting the distinctive morphology of these neoplastic cells. The tumor-stroma interface exhibits heterogeneity, with irregular boundaries that highlight the complex interaction between the malignant cells and their microenvironment. Lastly, local variation in mitotic activity is evident, with hotspots of proliferating cells indicating areas of increased cellular division and potential for aggressive growth.
TCGA-BRCA	IDC	Invasive ductal carcinoma exhibits high cellular density with cells closely packed and minimal intervening stroma, leading to a crowded appearance. Ducts often show lumen formation with cellular debris accumulating within them. Surrounding these areas, peritumoral fibrosis is characterized by thickened fibrotic bands that encircle the invasive tumor regions. Additionally, desmoplastic stroma is evident as dense fibrous tissue frequently surrounds the tumor clusters, contributing to a robust and complex microenvironment.
	ILC	Invasive lobular carcinoma (ILC) can be characterized through high-resolution imaging from four distinct perspectives: In single-file arrangement, tumor cells often infiltrate in linear chains between collagen fibers, creating a linear or string-of-beads appearance. In diffuse infiltration, tumor cells spread in a more scattered, non-nodular manner, leading to a less organized and more widespread distribution. There is typically minimal fibroblast activation with fewer myofibroblasts compared to invasive ductal carcinoma (IDC), resulting in a less fibrotic stroma. Additionally, a targetoid pattern may be observed where tumor cells wrap around normal ductal structures, creating a concentric or target-like appearance around these structures.

Table 6: Examples of sampled descriptions from the multi-view category knowledge base for each category within each dataset.

FM	ACC	AUC	F1 Score
PLIP [ 20 ]	72.82	73.47	67.24
MUSK [ 49 ]	86.72	88.28	85.18
Prov-GigaPath [ 50 ]	87.32	89.73	84.26
CONCH [ 29 ]	89.70	92.52	88.59

Table 7: Performance comparison (presented in %) of different foundation models (PLIP, MUSK, Prov-GigaPath, and CONCH) on CAMELYON under 16-shot settings.

	number of retrieved texts	ACC	AUC	F1 Score
16-shot	10	87.88	91.33	86.75
	20	89.70	92.52	88.59
	80	88.43	91.63	87.26
8-shot	10	83.64	86.44	80.79
	20	84.01	88.32	82.42
	80	84.49	88.40	83.07

Table 8: Performance comparison (presented in %) of different number of retrieved texts on CAMELYON under few-shot learning settings.

	number of experts	ACC	AUC	F1 Score
16-shot	4	87.91	90.38	86.64
	8	89.70	92.52	88.59
	16	87.32	90.16	86.18
8-shot	4	82.88	87.49	81.52
	8	84.01	88.32	82.42
	16	84.31	87.60	81.94

Table 9: Performance comparison (presented in %) of different number of experts on CAMELYON under few-shot learning settings.

	Ratio	ACC	AUC	F1 Score
16-shot	10%	88.92	91.92	87.83
	20%	89.70	92.52	88.59
	50%	88.51	91.66	87.40
	100%	88.51	91.62	87.53
8-shot	10%	85.65	88.58	83.65
	20%	84.01	88.32	82.42

Table 10: Performance comparison (presented in %) of different top-patch ratio on CAMELYON under few-shot learning settings.

Figure 5: Visualization of the regions attended by individual experts.

Figure 6: The visualization analysis of semantic refinement via DSR. (a) Visual regions associated with text mapping through the cross-attention mechanism. (b) Relevant visual regions after mapping via DSR.

Figure 7: Retrieved textual descriptions from the knowledge base given a WSI query.

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Explore similar work

Mar 23, 2026

Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification

Whole Slide Images (WSIs) are giga-pixel in scale and are typically partitioned into small instances in WSI classification pipelines for computational feasibility. However, obtaining extensive instance level annotations is costly, making few-shot weakly supervised WSI classification (FSWC) crucial for learning from limited slide-level labels. Recently, pre-trained vision-language models (VLMs) have been adopted in FSWC, yet they exhibit several limitations. Existing prompt tuning methods in FSWC substantially increase both the number of trainable parameters and inference overhead. Moreover, current methods discard instances with low alignment to text embeddings from VLMs, potentially leading to information loss. To address these challenges, we propose two key contributions. First, we introduce a new parameter efficient prompt tuning method by scaling and shifting features in text encoder, which significantly reduces the computational cost. Second, to leverage not only the pre-trained knowledge of VLMs, but also the inherent hierarchical structure of WSIs, we introduce a WSI representation learning approach with a soft hierarchical textual guidance strategy without utilizing hard instance filtering. Comprehensive evaluations on pathology datasets covering breast, lung, and ovarian cancer types demonstrate consistent improvements up-to 10.9%, 7.8%, and 13.8% respectively, over the state-of-the-art methods in FSWC. Our method reduces the number of trainable parameters by 18.1% on both breast and lung cancer datasets, and 5.8% on the ovarian cancer dataset, while also excelling at weakly-supervised tumor localization. Code at https://github.com/Jayanie/HIPSS.

Jayanie Bogahawatte, Sachith Seneviratne, Saman Halgamuge

AI, Optimization and Pattern Recognition Research Group, Dept. of Mechanical Eng., University of Melbourne, Australia

Whole-Slide Image Analysis Parameter-Efficient Fine-Tuning Visual Prompt Learning

Ask PDF

Dec 19, 2025

PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology

While Vision-Language Models (VLMs) have achieved notable progress in computational pathology (CPath), the gigapixel scale and spatial heterogeneity of Whole Slide Images (WSIs) continue to pose challenges for multimodal understanding. Existing alignment methods struggle to capture fine-grained correspondences between textual descriptions and visual cues across thousands of patches from a slide, compromising their performance on downstream tasks. In this paper, we propose PathFLIP (Pathology Fine-grained Language-Image Pretraining), a novel framework for holistic WSI interpretation. PathFLIP decomposes slide-level captions into region-level subcaptions and generates text-conditioned region embeddings to facilitate precise visual-language grounding. By harnessing Large Language Models (LLMs), PathFLIP can seamlessly follow diverse clinical instructions and adapt to varied diagnostic contexts. Furthermore, it exhibits versatile capabilities across multiple paradigms, efficiently handling slide-level classification and retrieval, fine-grained lesion localization, and instruction following. Extensive experiments demonstrate that PathFLIP outperforms existing large-scale pathological VLMs on four representative benchmarks while requiring significantly less training data, paving the way for fine-grained, instruction-aware WSI interpretation in clinical practice.

Fengchun Liu, Songhan Jiang, Linghan Cai +2

Harbin Institute of Technology, Shenzhen, School of Computer Science and Technology · National University of Singapore, Department of Electronic and Computer Engineering

Computational Pathology Vision-Language Grounding Medical VLMs Whole-Slide Image Analysis

Ask PDF

Feb 21, 2026

Initialization matters in few-shot adaptation of vision-language models for histopathological image classification

Vision language models (VLM) pre-trained on datasets of histopathological image-caption pairs enabled zero-shot slide-level classification. The ability of VLM image encoders to extract discriminative features also opens the door for supervised fine-tuning for whole-slide image (WSI) classification, ideally using few labeled samples. Slide-level prediction frameworks require the incorporation of multiple instance learning (MIL) due to the gigapixel size of the WSI. Following patch-level feature extraction and aggregation, MIL frameworks rely on linear classifiers trained on top of the slide-level aggregated features. Classifier weight initialization has a large influence on Linear Probing performance in efficient transfer learning (ETL) approaches based on few-shot learning. In this work, we propose Zero-Shot Multiple-Instance Learning (ZS-MIL) to address the limitations of random classifier initialization that underperform zero-shot prediction in MIL problems. ZS-MIL uses the class-level embeddings of the VLM text encoder as the classification layer's starting point to compute each sample's bag-level probabilities. Through multiple experiments, we demonstrate the robustness of ZS-MIL compared to well-known weight initialization techniques both in terms of performance and variability in an ETL few-shot scenario for subtyping prediction.

Pablo Meseguer, Rocío del Amor, Valery Naranjo

Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano (HUMAN-Tech), Universitat Politècnica de Valencia, Valencia, Spain · Valencian Graduate School and Research Network for Artificial Intelligence (valgrAI), Valencia, Spain

Histopathology Image Classification VLM Adaptation Few-Shot Image Classification Whole-Slide Image Analysis Multiple Instance Learning

Ask PDF