Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification

Jayanie Bogahawatte; Sachith Seneviratne; Saman Halgamuge

cs.CVMar 23, 2026

Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification

Authors: Jayanie Bogahawatte, Sachith Seneviratne, Saman Halgamuge

Organizations: AI, Optimization and Pattern Recognition Research Group, Dept. of Mechanical Eng., University of Melbourne, Australia

Abstract

Whole Slide Images (WSIs) are giga-pixel in scale and are typically partitioned into small instances in WSI classification pipelines for computational feasibility. However, obtaining extensive instance level annotations is costly, making few-shot weakly supervised WSI classification (FSWC) crucial for learning from limited slide-level labels. Recently, pre-trained vision-language models (VLMs) have been adopted in FSWC, yet they exhibit several limitations. Existing prompt tuning methods in FSWC substantially increase both the number of trainable parameters and inference overhead. Moreover, current methods discard instances with low alignment to text embeddings from VLMs, potentially leading to information loss. To address these challenges, we propose two key contributions. First, we introduce a new parameter efficient prompt tuning method by scaling and shifting features in text encoder, which significantly reduces the computational cost. Second, to leverage not only the pre-trained knowledge of VLMs, but also the inherent hierarchical structure of WSIs, we introduce a WSI representation learning approach with a soft hierarchical textual guidance strategy without utilizing hard instance filtering. Comprehensive evaluations on pathology datasets covering breast, lung, and ovarian cancer types demonstrate consistent improvements up-to 10.9%, 7.8%, and 13.8% respectively, over the state-of-the-art methods in FSWC. Our method reduces the number of trainable parameters by 18.1% on both breast and lung cancer datasets, and 5.8% on the ovarian cancer dataset, while also excelling at weakly-supervised tumor localization. Code at https://github.com/Jayanie/HIPSS.

Figures & tables

Figure 1: Illustration of HIPSS framework. (a) : Two text descriptions are generated considering the entire WSI and the regions in WSI. A set of task-specific parameters: $\gamma$ and $\beta$ are attached to a set of selected layers of the text encoder backbone, up-to a pre-defined depth . (b) : WSI representation learning using two attention pooling mechanisms with textual guidance. Region encoder creates each region embedding considering the instances in the specific region. WSI encoder creates the WSI embedding by aggregating region embeddings. Attention weights are refined based on the cosine similarity between instance (or region) embeddings and the text embeddings . Contrastive learning based loss is calculated between image and text embeddings.

Dataset	Method	16-shot setting	8-shot setting	4-shot setting	2-shot setting	1-shot setting
Camelyon16	Max Pooling - CLIP	$0.5947_{\pm 0.0135}$	$0.5814_{\pm 0.0112}$	$0.5417_{\pm 0.0126}$	$0.5401_{\pm 0.0197}$	$0.5268_{\pm 0.0154}$
	Mean Pooling - CLIP	$0.5816_{\pm 0.0127}$	$0.5627_{\pm 0.0098}$	$0.4930_{\pm 0.0122}$	$0.3235_{\pm 0.0178}$	$0.3357_{\pm 0.0173}$
	Max Pooling	$0.6709_{\pm 0.0182}$	$0.6725_{\pm 0.0095}$	$0.6263_{\pm 0.0128}$	$0.5684_{\pm 0.0406}$	$0.5732_{\pm 0.0191}$
	Mean Pooling	$0.6843_{\pm 0.0193}$	$0.6330_{\pm 0.0449}$	$0.5959_{\pm 0.0141}$	$0.5637_{\pm 0.0318}$	$0.5716_{\pm 0.0259}$
	ABMIL (ICML’ 18)	$0.7820_{\pm 0.0455}$	$0.7195_{\pm 0.0314}$	$0.6086_{\pm 0.0587}$	$0.5969_{\pm 0.0431}$	$0.5706_{\pm 0.0426}$
	ViLa-MIL (CVPR’ 24)	$0.7997_{\pm 0.0383}$	$0.7599_{\pm 0.0474}$	$0.5617_{\pm 0.0543}$	$0.4601_{\pm 0.0365}$	$0.4565_{\pm 0.0280}$

Table 1: Few Shot Classification results for Camelyon16, TCGA-Lung, and UBC-OCEAN Datasets. Mean AUC values averaged over 3 different folds, each evaluated using 20 random seeds are reported (Refer to Section 4.1 for further details). Performance improvement compared to the second best method is reported within brackets with (+). Bold: best results, Underline: second best results.

Dataset	H	T	S	16-shot	4-shot	1-shot
Camelyon16	$\times$	$\times$	$\times$	$0.7820_{\pm 0.0455}$	$0.6086_{\pm 0.0587}$	$0.5706_{\pm 0.0426}$
	$\checkmark$	$\times$	$\times$	$0.8666_{\pm 0.0085}$	$0.6903_{\pm 0.0170}$	$0.6433_{\pm 0.0169}$
	$\checkmark$	$\checkmark$	$\times$	$0.8724_{\pm 0.0086}$	$0.7552_{\pm 0.0173}$	$0.6512_{\pm 0.0089}$
	$\times$	$\times$	$\checkmark$	$0.8853_{\pm 0.0267}$	$0.7798_{\pm 0.0033}$	$0.7071_{\pm 0.0070}$
	$\checkmark$	$\times$	$\checkmark$	$0.9006_{\pm 0.0116}$	$0.8034_{\pm 0.0139}$	$0.7217_{\pm 0.0081}$
	$\checkmark$	$\checkmark$	$\checkmark$	$\bm{0.9105_{\pm 0.0104}}$	$\bm{0.8159_{\pm 0.0165}}$	$\bm{0.7429_{\pm 0.0087}}$

Table 2: Ablation Studies on 3 datasets. Mean AUC values are reported. Components are denoted as H: Hierarchical WSI representation learning, T: Text-guided attention refinement strategy, and S: SSF-based prompt tuning in text encoder. Full results are provided in the Appendix.

Method	16-shot	4-shot	1-shot
LoRA-based prompt tuning	$0.8513_{\pm 0.0162}$	$0.7668_{\pm 0.0258}$	$0.5667_{\pm 0.0289}$
HIPSS (Ours)	$\bm{0.9105_{\pm 0.0104}}$	$\bm{0.8159_{\pm 0.0165}}$	$\bm{0.7429_{\pm 0.0087}}$

Table 3: Performance comparison with LoRA-based prompt tuning on C16 dataset. Mean AUC values are reported. Number of trainable parameters of LoRA-based prompt tuning is 0.7053 M.

Figure 2: Variation of AUC values based on the number of tuned blocks

d_{s}

in the text encoder with SSF. Mean AUC values are reported. The

d_{s}

values with the highest AUC are denoted by red dots.

Figure 3: Variation of AUC values based on (a): the attention weights refinement factor (

\lambda

) and (b): threshold value (

\alpha

). We report the mean AUC averaged across three datasets. The

\lambda

and

\alpha

values with the highest AUC are denoted by red dots.

Method	Training Parameters		Avg Inference
Method	C16/ TCGA-Lung	UBC-OCEAN	time per slide (C16)
TOP [ 24 ]	$0.3472$	$0.3702$	$58.81$
FOCUS [ 7 ]	$1.3276$	$1.3783$	$328.49$
ViLa-MIL [ 28 ]	$2.6516$	$2.6747$	$27.30$
HIPSS (Ours)	$\bm{0.2844}$ ( $\downarrow 18.1\%$ )	$\bm{0.3489}$ ( $\downarrow 5.8\%$ )	$\bm{23.85}$ ( $\downarrow 12.6\%$ )

Table 4: Computational efficiency comparison. Total number of trainable parameters (in millions) and average inference time per slide (in ms) are reported. The reduction compared to the second efficient FSWC method is indicated by

\downarrow

Method Type	Method	Dice Coefficient
Conventional MIL methods	CLAM-SB	$0.459$
	TransMIL	$0.103$
	DTFD-MIL	$0.525$
	DSMIL	$0.259$
	CAMIL	$0.515$
Few-shot methods	ABMIL [ 12 ]	$0.164$

Table 5: Weakly-supervised tumor localization results on C16 Dataset as an evaluation protocol on the quality of learned feature representations. Dice coefficient is reported. Results of conventional MIL methods are cited from [ 5 ] .

Figure 4: Attention map visualizations from HIPSS compared with ground truth annotation. (b) : High attention regions predicted by the WSI Encoder. (c) : High attention instances predicted by the region encoder. Attention maps are overlaid where red indicates instances of higher attention. Quantitative results in Table 5 further demonstrates the tumor localization capabilities of our method.

Explore similar work

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images
Sep 25, 2026Tiffanie Godelaine, Manon Dausort, Karim El Khoury +3Histopathology Image Classification Few-Shot Image Classification
ICTEAM, Universit´e Catholique de Louvain, Belgium
MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification
Feb 24, 2026Jiahao Xu, Sheng Huang, Xin Zhang +3Computational Pathology Medical VLMs
School of Big Data & Software Engineering, Chongqing University · School of Computer Science & Technology, Chongqing University
Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis
Mar 16, 2026Umar Marikkar, Muhammad Awais, Sara AtitoWhole-Slide Image Analysis Computational Pathology
Institute for People-Centered AI, University of Surrey · Centre of Vision, Speech and Signal Processing, University of Surrey

Sep 25, 2026J/K move · Enter open · S save

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

Tiffanie Godelaine, Manon Dausort, Karim El Khoury, Benoît Gérin+2

ICTEAM, Universit´e Catholique de Louvain, Belgium

Histopathology Image Classification Few-Shot Image Classification Whole-Slide Image Analysis Few-Shot Learning

Open paper AskPDF arXiv

Automating the analysis of whole-slide images (WSIs), a key step in cancer diagnosis, has high clinical value, as it can reduce pathologist's workload while improving diagnosis accuracy. Recently, vision-language models have shown promising performance for patch-level classification without requiring any annotation, yet these zero-shot (ZS) predictions remain noisy on fine-grained tasks and must be further refined. A promising direction is to refine all predictions jointly, i.e., a transductive approach. However, most existing methods are not tailored to WSIs. We thus propose SlideTIM, an adaptation to WSIs of the recent transductive approach LC-TIM, which introduces a combined spatial--latent regularizer together with a prior on the patch class distribution. The former enforces spatially and semantically close patches to receive the same predictions, while the prior calibrates the predicted class proportions. Together, they address the complex spatial organization and the strong class imbalance of WSIs. Evaluated on four histology datasets, SlideTIM consistently outperforms all TIM variants, improving the macro-F1 by +8.1pp over the best competing baseline at 1 shot. Compared to the ZS, it raises the macro-F1 by +19.4pp at 1 shot. The code will be made available after submission.

Abstract

Explore similar work

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification

Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis