MINT: Molecularly Informed Training with Spatial Transcriptomics Supervision for Pathology Foundation Models

Minsoo Lee; Jonghyun Kim; Juseung Yun; Sunwoo Yu; Jongseong Jang

cs.CVMar 9, 2026

MINT: Molecularly Informed Training with Spatial Transcriptomics Supervision for Pathology Foundation Models

Authors: Minsoo Lee, Jonghyun Kim, Juseung Yun, Sunwoo Yu, Jongseong Jang

Organizations: LG AI Research

Abstract

Pathology foundation models learn morphological representations through self-supervised pretraining on large-scale whole-slide images, yet they do not explicitly capture the underlying molecular state of the tissue. Spatial transcriptomics technologies bridge this gap by measuring gene expression in situ, offering a natural cross-modal supervisory signal. We propose MINT (Molecularly Informed Training), a fine-tuning framework that incorporates spatial transcriptomics supervision into pretrained pathology Vision Transformers. MINT appends a learnable ST token to the ViT input to encode transcriptomic information separately from the morphological CLS token, preventing catastrophic forgetting through DINO self-distillation and explicit feature anchoring to the frozen pretrained encoder. Gene expression regression at both spot-level (Visium) and patch-level (Xenium) resolutions provides complementary supervision across spatial scales. Trained on 577 publicly available HEST samples, MINT achieves the best overall performance on both HEST-Bench for gene expression prediction (mean Pearson r = 0.440) and EVA for general pathology tasks (0.803), demonstrating that spatial transcriptomics supervision complements morphology-centric self-supervised pretraining.

Figures & tables

Figure 1: MINT framework. The student ViT, augmented with a learnable ST token, outputs CLS, ST, and patch token representations. Gene expression regression from the ST token ( $\mathcal{L}_{\text{ST}}$ ) and patch tokens ( $\mathcal{L}_{\text{pST}}$ ) provides transcriptomic supervision, while DINO self-distillation ( $\mathcal{L}_{\text{DINO}}$ ) and feature anchoring ( $\mathcal{L}_{\text{distill}}$ ) preserve morphological representations. Only the student is updated via backpropagation.

Model	IDC	PRAD	PAAD	SKCM	COAD	READ	ccRCC	LUAD	LYMPH	Avg
CTransPath [ 15 ]	0.511	0.343	0.438	0.511	0.229	0.110	0.228	0.499	0.235	0.345
Phikon [ 19 ]	0.533	0.342	0.443	0.536	0.259	0.152	0.242	0.547	0.237	0.366
CONCH [ 5 ]	0.536	0.355	0.448	0.579	0.253	0.167	0.218	0.531	0.251	0.371
REMEDIS [ 22 ]	0.529	0.347	0.464	0.582	0.286	0.115	0.265	0.534	0.247	0.374
CONCH 1.5 [ 5 ]	0.544	0.381	0.457	0.552	0.280	0.160	0.218	0.551	0.270	0.379
GigaPath [ 16 ]	0.551	0.371	0.477	0.554	0.301	0.186	0.239	0.540	0.249	0.385

Table 1: HEST-Bench results (PCA+Ridge with 256 PCA components; Pearson correlation; higher is better). Best in bold , second-best underlined .

Model	BHis	CRC	Glea	MHIST	PCam	Cam16	PANDA	CoNS	MoNuS	Average
Phikon-v2 [ 24 ]	0.713	0.939	0.757	0.777	0.894	0.805	0.625	0.627	0.644	0.753
Phikon [ 19 ]	0.717	0.940	0.729	0.803	0.920	0.798	0.644	0.627	0.635	0.757
CONCH [ 5 ]	0.681	0.953	0.729	0.832	0.922	0.841	0.628	0.618	0.632	0.760
hibou-L [ 25 ]	0.735	0.932	0.764	0.839	0.955	0.823	0.634	0.646	0.669	0.777
UNI [ 2 ]	0.785	0.944	0.750	0.843	0.937	0.833	0.659	0.628	0.659	0.782
Pv-GigaPath [ 16 ]	0.827	0.951	0.724	0.829	0.945	0.815	0.653	0.626	0.680	0.783

Table 2: EVA evaluation framework results (higher is better). Best in bold , second-best underlined . Average is the arithmetic mean over 9 benchmarks.

Backbone	Training	Representation	HEST-Bench	EVA-CLS
H-opt-0	No training	CLS-only	0.415	0.830
	$L_{\mathrm{ST}}$ on [CLS] w/o $L_{\mathrm{distill}}$	CLS-only	0.424	0.811
	$L_{\mathrm{ST}}$ on [CLS] + $L_{\mathrm{distill}}$	CLS-only	0.426	0.827
	MINT	CLS-only	0.413	0.828
		ST-only	0.428	0.823
		CLS $\oplus$ ST (sum)	0.431	0.837

Table 3: Representation variants across backbones, including pretrained baselines and ablations. HEST-Bench reports mean Pearson correlation; EVA-CLS averages over classification and whole-slide tasks (7 benchmarks). Best in bold , second-best underlined .

Explore similar work

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology
Oct 13, 2026Koushik Howlader, Tirtho Roy, Md Tauhidul Islam +1Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images
Iowa State University, Iowa, USA · Stanford University, CA, USA
Spatial Transcriptomics-Guided Alignment Enhances Molecular Profiling in Pathology Foundation Model
May 29, 2026Fengtao Zhou, Yingxue Xu, Zhengyu Zhang +20Spatial Transcriptomics Cross-Modal Alignment
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China · Department of Pathology, Nanfang Hospital, Southern Medical University, Guangzhou, China · Department of Pathology, School of Basic Medical Sciences, Southern Medical University, Guangzhou, China +16
A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction
Apr 4, 2026Jinxi Xiang, Siyu Hou, Yuchen Li +17Spatial Transcriptomics Gene Expression Prediction from Histopathology Images
Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA · Department of Statistics and Data Science, Yale University, New Haven, CT, USA · Department of Medicine (Oncology), Stanford University School of Medicine, Stanford, CA, USA +4

Oct 13, 2026J/K move · Enter open · S save

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

Koushik Howlader, Tirtho Roy, Md Tauhidul Islam, Wei Le

Iowa State University, Iowa, USA · Stanford University, CA, USA

Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images Spatial Transcriptomics Multi-Scale Attention Spatial Reasoning

Open paper AskPDF arXiv

Predicting spatial gene expression from standard H&E images could extend molecular analysis to tissues without spatial transcriptomic measurements. A standard Transformer applies self-attention over spots but ignores where each spot sits in the tissue, whereas a spot's molecular state depends on tissue context at multiple spatial scales. Building on this insight, we introduce MIST, a Multi-context Spatial Transformer that augments the self-attention backbone with two context pathways applied in every layer: a coordinate-invariant local pathway that attends over spatial k-nearest-neighbor spots, and a slide pathway that broadcasts a whole-slide summary to every spot; the model predicts all spots in a single forward pass. Because coordinates enter only through pairwise distances, the local pathway is invariant to translations, rotations, and reflections when image features are fixed. In intra-cohort evaluation across six human organ cohorts from HEST-1k, MIST improves mean Pearson correlation over the strongest evaluated baseline by 9.7%, 11.6%, and 11.4% for 10-, 50-, and 100-gene panels, respectively, with the lowest average MSE and MAE. MIST further improves leave-one-organ-out transfer by 6.0% over the next strongest baseline, attains the best expression-derived domain recovery (highest mean ARI and NMI at every clustering resolution, and higher NMI than the closest baseline on 84% of held-out slides), and reproduces spatial expression patterns of selected cancer-marker genes. Together, these results show that adding local and slide context to a vanilla Transformer backbone is an effective approach to spatial gene expression prediction from histology.

Abstract

Explore similar work

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

Spatial Transcriptomics-Guided Alignment Enhances Molecular Profiling in Pathology Foundation Model

A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction