Jigsaw Regularization in Whole-Slide Image Classification

So Won Jeong; Veronika Ročková

cs.CVMar 20, 2026

Jigsaw Regularization in Whole-Slide Image Classification

Authors: So Won Jeong, Veronika Ročková

Organizations: Booth School of Business, The University of Chicago, IL, USA.

Abstract

Computational pathology involves the digitization of stained tissues into whole-slide images (WSIs) that contain billions of pixels arranged as contiguous patches. Statistical analysis of WSIs largely focuses on classification via multiple instance learning (MIL), in which slide-level labels are inferred from unlabeled patches. Most MIL methods treat patches as exchangeable, overlooking the rich spatial and topological structure that underlies tissue images. This work builds on recent graph-based methods that aim to incorporate spatial awareness into MIL. Our approach is new in two regards: (1) we deploy vision foundation-model embeddings to incorporate local spatial structure within each patch, and (2) achieve across-patch spatial awareness using graph neural networks together with a novel {\em jigsaw regularization}. We find that a combination of these two features markedly improves classification over state-of-the-art attention-based MIL approaches on benchmark datasets in breast, head-and-neck, and colon cancer.

Figures & tables

Figure 1: Jigsaw-regularized graph ABMIL framework. There is a shared encoder backbone (green).

Figure 2: Jigsaw auxiliary task pipeline integrated into graph-MIL.

Figure 3: Spatial permutation of patches alters their embeddings $\tilde{h}_{ij}$ under our spatially-aware framework.

Method	BRCA	HNSC	COAD
ABMIL	0.884	0.809	0.822
ABMIL+Jigsaw	0.880	0.820	0.806
Graph-MIL	0.900	0.830	0.901
Graph-ABMIL	0.932	0.810	0.841
Graph-ABMIL + Jigsaw	0.952	0.833	0.885

Table 1: Performance across Cancer Types on small TCGA benchmarks. 3-fold cross-validated ROC-AUC is reported.

Figure 4: Visualization of attention map. Here, the attention weight ( $a_{ij}$ ) is computed following ( 1 ). Each point corresponds to each patch. The trained model is evaluated on the test data points. It is known that ductal and lobular breast cancer has visually different morphology 2 2 2 https://lobularbreastcancer.org/its-not-just-breast-cancer-its-lobular-breast-cancer/ , which serves as a reference point in the third column.

Method	GigaPath	RetCCL	UNI	Virchow2
ABMIL	0.941	0.919	0.941	0.936
Graph-MIL	0.949	0.911	0.954	0.948
Graph-ABMIL	0.948	0.921	0.954	0.936
Graph-ABMIL + Jigsaw	0.952	0.921	0.954	0.950

Table 2: Performance of TCGA-BRCA dataset across different foundation embeddings. 10-fold cross-validated ROC-AUC is reported.

$\lambda$	BRCA	HNSC	COAD
0.01	0.929 $\pm$ 0.020	0.843 $\pm$ 0.042	0.848 $\pm$ 0.099
0.1	0.941 $\pm$ 0.031	0.832 $\pm$ 0.086	0.837 $\pm$ 0.043
0.5	0.952 $\pm$ 0.046	0.833 $\pm$ 0.06	0.814 $\pm$ 0.139
1.0	0.929 $\pm$ 0.026	0.832 $\pm$ 0.060	0.846 $\pm$ 0.107
2.0	0.910 $\pm$ 0.096	0.822 $\pm$ 0.083	0.817 $\pm$ 0.097
5.0	0.879 $\pm$ 0.108	0.820 $\pm$ 0.060	0.860 $\pm$ 0.053

Table 3: Performance across Cancer Types on small TCGA benchmarks across different regularization strength

\lambda

compared to the Bayesian updating scheme introduced in Section 6 . 3-fold cross-validated test ROC-AUC is reported.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	UNI	gigapath	retccl	Virchow2
Embedding dim	1024	1536	2048	2560
Num of patches (avg)	3278.1	1310.5	1033.1	3287.1
Num of patches (min)	50	11	11	50
Num of patches (max)	12050	6631	5313	12050

Appendix

Table 4: Summary statistics fo TCGA-BRCA dataset across four different foundational embeddings.

$k$	BRCA	HNSC	MSI
8	0.914 $\pm$ 0.029	0.855 $\pm$ 0.056	0.775 $\pm$ 0.145
24	0.893 $\pm$ 0.103	0.832 $\pm$ 0.069	0.8305 $\pm$ 0.0931
50	0.930 $\pm$ 0.055	0.821 $\pm$ 0.067	0.815 $\pm$ 0.113

Appendix

Table 5: Effect of

k

-NN graph connectivity on classification performance. All models use spatial-GAT convolution with ABMIL aggregation, without jigsaw regularization. Mean

\pm

std AUC over 3-fold cross-validation is reported.

	BRCA	HNSC	COAD
Spatial-GAT (Ours)	0.9304 $\pm$ 0.0554	0.8206 $\pm$ 0.067	0.8152 $\pm$ 0.1131
GCN ( Kipf & Welling, 2017 )	0.8247 $\pm$ 0.0909	0.8325 $\pm$ 0.095	0.8091 $\pm$ 0.1298
GIN ( Xu et al., 2018 )	0.8394 $\pm$ 0.0425	0.796 $\pm$ 0.0654	0.8327 $\pm$ 0.1074
SAGE ( Hamilton et al., 2017 )	0.8774 $\pm$ 0.056	0.8163 $\pm$ 0.0842	0.7775 $\pm$ 0.105
ABMIL ( Ilse et al., 2018 )	0.8844 $\pm$ 0.1138	0.8094 $\pm$ 0.0615	0.8223 $\pm$ 0.0853

Appendix

Table 6: Ablation over graph convolution type on small TCGA benchmarks. All methods use

k{=}50

nearest-neighbor graphs with Gaussian edge weights and no jigsaw regularization. Bold : best per column; underline : second best. 3-fold cross-validated test ROC-AUC is reported (mean

\pm

std).

Histopathology Image Classification

Histopathology image classification uses computational models to identify tissue types, tumors, grades, and biomarkers from stained pathology images.

See all 134 papers

Exploiting Label-Independent Regularization from Spatial Dependencies for Whole Slide Image Analysis
Feb 23, 2026Weiyi Wu, Xinwen Xu, Chongyang Gao +3Whole-Slide Image Analysis Multiple Instance Learning
1Dartmouth College · 2Massachusetts General Hospital · 3Northwestern University
Masked Feature Encoding for Large-Scale Whole Slide Image Representation
Oct 7, 2026Haoyu He, Basile Tessier-Cloutier, Yang Wang +1Whole-Slide Image Analysis Medical Image Representation Learning
Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, Canada · McGill University, Montreal, Canada · Mila – Quebec AI Institute, Montreal, Canada
Geometry-Aware State Space Model: A New Paradigm for Whole-Slide Image Representation
May 6, 2026Enhui Chai, Sicheng Chen, Tianyi Zhang +4Histopathology Image Classification Multiple Instance Learning
PuzzleLogic Pte Ltd, Singapore 229594, Singapore · Nhu Department of Electrical Engineering and Computer Science, University of California, Irvine · Department of Electrical & Computer Engineering, National University of Singapore

Feb 23, 2026J/K move · Enter open · S save

Exploiting Label-Independent Regularization from Spatial Dependencies for Whole Slide Image Analysis

Weiyi Wu, Xinwen Xu, Chongyang Gao, Xingjian Diao+2

1Dartmouth College · 2Massachusetts General Hospital · 3Northwestern University

Whole-Slide Image Analysis Multiple Instance Learning Histopathology Image Classification

Open paper AskPDF arXiv

Whole slide images, with their gigapixel-scale panoramas of tissue samples, are pivotal for precise disease diagnosis. However, their analysis is hindered by immense data size and scarce annotations. Existing MIL methods face challenges due to the fundamental imbalance where a single bag-level label must guide the learning of numerous patch-level features. This sparse supervision makes it difficult to reliably identify discriminative patches during training, leading to unstable optimization and suboptimal solutions. We propose a spatially regularized MIL framework that leverages inherent spatial relationships among patch features as label-independent regularization signals. Our approach learns a shared representation space by jointly optimizing feature-induced spatial reconstruction and label-guided classification objectives, enforcing consistency between intrinsic structural patterns and supervisory signals. Experimental results on multiple public datasets demonstrate significant improvements over state-of-the-art methods, offering a promising direction.

Abstract

Histopathology Image Classification

Explore similar work

Exploiting Label-Independent Regularization from Spatial Dependencies for Whole Slide Image Analysis

Masked Feature Encoding for Large-Scale Whole Slide Image Representation

Geometry-Aware State Space Model: A New Paradigm for Whole-Slide Image Representation