BiTro: Bidirectional Transfer Learning Enhances Bulk and Spatial Transcriptomics Prediction in Cancer Pathological Images

Jingkun Yu; Guangkai Shang; Changtao Li; Xun Gong; Tianrui Li; Yazhou He; Zhipeng Luo

cs.LGMar 16, 2026

BiTro: Bidirectional Transfer Learning Enhances Bulk and Spatial Transcriptomics Prediction in Cancer Pathological Images

Authors: Jingkun Yu, Guangkai Shang, Changtao Li, Xun Gong, Tianrui Li, Yazhou He, Zhipeng Luo

Organizations: SWJTU-Leeds Joint School, Southwest Jiaotong University, Chengdu, China · Yu-Yue Pathology Scientific Research Center, Jinfeng Laboratory, Chongqing, China · Department of Epidemiology and Medical Statistics, West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, China · School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China ·

Abstract

Cancer pathological analysis requires modeling tumor heterogeneity across multiple modalities, primarily through transcriptomics and whole slide imaging (WSI), along with their spatial relations. On one hand, bulk transcriptomics and WSI images are largely available but lack spatial mapping; on the other hand, spatial transcriptomics (ST) data can offer high spatial resolution, yet facing challenges of high cost, low sequencing depth, and limited sample sizes. Therefore, the data foundation of either side is flawed and has its limit in accurately finding the mapping between the two modalities. To this end, we propose BiTro, a bidirectional transfer learning framework that can enhance bulk and spatial transcriptomics prediction from pathological images. Our contributions are twofold. First, we design a universal and transferable model architecture that works for both bulk+WSI and ST data. A major highlight is that we model WSI images on the cellular level to better capture cells' visual features, morphological phenotypes, and their spatial relations; to map cells' features to their transcriptomics measured in bulk or ST, we adopt multiple instance learning. Second, by using LoRA, our model can be efficiently transferred between bulk and ST data to exploit their complementary information. To test our framework, we conducted comprehensive experiments on five cancer datasets. Results demonstrate that 1) our base model can achieve better or competitive performance compared to existing models on bulk or spatial transcriptomics prediction, and 2) transfer learning can further improve the base model's performance.

Figures & tables

Figure 1. Overview of the BiTro framework. There are four sequential phases: (a) WSI preprocessing : segment a WSI into cells by HoverNet or CellViT; (b) Cell feature extraction : extract cells’ visual features via Dinov3, capture cell phenotypes by K-means clustering, and add spatial coordinates; (c) Spatial enhancement : enhance cell features with both local and global spatial relations; (d) MIL and Bidirectional transfer learning : map cellular features to their bulk or spot-level transcriptomics by multiple instance learning. The learning process is enhanced by both levels of data via bidirectional transfer learning.

Figure 2. Model architecture. Inputs are cellular visual features

\mathbf{H}

and their local spatial graph

\mathcal{G}

. The GAT module and the Transformer encoder are to enhance

\mathbf{H}

with spatial relations both locally and globally. The enhanced features

\mathbf{H}_{\mathrm{cell}}

are used to predict gene expression

\hat{\mathbf{y}}

by multiple instance pooling.

Method	BRCA		COAD		PRAD		LIHC (HCC)
	$\text{PCC}_{\text{overall}}\uparrow$	JS $\downarrow$	$\text{PCC}_{\text{overall}}\uparrow$	JS $\downarrow$	$\text{PCC}_{\text{overall}}\uparrow$	JS $\downarrow$	$\text{PCC}_{\text{overall}}\uparrow$	JS $\downarrow$
Original	0.502 $\pm$ 0.198	0.181 $\pm$ 0.046	0.869 $\pm$ 0.015	0.040 $\pm$ 0.015	0.659 $\pm$ 0.148	0.092 $\pm$ 0.034	0.662 $\pm$ 0.248	0.214 $\pm$ 0.076
UNI+MLP	0.652 $\pm$ 0.275	0.132 $\pm$ 0.059	0.920 $\pm$ 0.098	0.021 $\pm$ 0.015	0.806 $\pm$ 0.127	0.053 $\pm$ 0.021	0.735 $\pm$ 0.177	0.174 $\pm$ 0.062
HE2RNA	0.710 $\pm$ 0.032	0.086 $\pm$ 0.002	0.911 $\pm$ 0.005	0.024 $\pm$ 0.002	0.830 $\pm$ 0.157	0.051 $\pm$ 0.005	0.827 $\pm$ 0.234	0.119 $\pm$ 0.084
SEQUOIA	0.738 $\pm$ 0.199	0.094 $\pm$ 0.059	0.932 $\pm$ 0.043	0.020 $\pm$ 0.022	0.820 $\pm$ 0.068	0.056 $\pm$ 0.019	0.738 $\pm$ 0.109	0.149 $\pm$ 0.014
BiTro (w/o trans.)	0.711 $\pm$ 0.174	0.107 $\pm$ 0.081	0.933 $\pm$ 0.007	0.021 $\pm$ 0.009	0.814 $\pm$ 0.127	0.050 $\pm$ 0.022	0.811 $\pm$ 0.263	0.139 $\pm$ 0.093

Table 1. Performance comparison on bulk transcriptomics prediction. Best results are bolded, and the second best underlined.

Model	BRCA		COAD		PRAD		HCC		CSCC
	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$
HisToGene	0.541 $\pm$ 0.079	0.114 $\pm$ 0.079	0.515 $\pm$ 0.161	0.102 $\pm$ 0.062	0.449 $\pm$ 0.120	0.110 $\pm$ 0.087	0.282 $\pm$ 0.020	0.072 $\pm$ 0.005	0.560 $\pm$ 0.049	0.157 $\pm$ 0.086
Hist2ST	0.574 $\pm$ 0.090	0.141 $\pm$ 0.084	0.621 $\pm$ 0.080	0.133 $\pm$ 0.132	0.438 $\pm$ 0.160	0.071 $\pm$ 0.057	0.274 $\pm$ 0.013	-0.001 $\pm$ 0.005	0.610 $\pm$ 0.109	0.131 $\pm$ 0.061
THItoGene	0.558 $\pm$ 0.099	0.094 $\pm$ 0.089	0.614 $\pm$ 0.076	0.075 $\pm$ 0.081	0.484 $\pm$ 0.104	0.134 $\pm$ 0.106	0.247 $\pm$ 0.051	0.005 $\pm$ 0.003	0.599 $\pm$ 0.039	0.241 $\pm$ 0.044
M2OST	0.558 $\pm$ 0.660	0.117 $\pm$ 0.081	0.514 $\pm$ 0.057	0.122 $\pm$ 0.117	0.515 $\pm$ 0.094	0.256 $\pm$ 0.070	0.124 $\pm$ 0.002	0.102 $\pm$ 0.030	0.450 $\pm$ 0.084	0.142 $\pm$ 0.057
BiTro (w/o trans.)	0.587 $\pm$ 0.101	0.154 $\pm$ 0.136	0.565 $\pm$ 0.066	0.103 $\pm$ 0.092	0.502 $\pm$ 0.129	0.186 $\pm$ 0.075	0.341 $\pm$ 0.012	0.128 $\pm$ 0.025	0.667 $\pm$ 0.041	0.429 $\pm$ 0.154

Table 2. Performance comparison on spatial transcriptomics prediction. Best results are bolded, and the second best underlined.

Model	BRCA		COAD		PRAD		HCC
	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$	$\text{PCC}_{\text{overall}}\uparrow$	$\text{PCC}_{\text{gene}}\uparrow$
GHIST	0.499 $\pm$ 0.024	0.348 $\pm$ 0.038	0.450 $\pm$ 0.092	0.209 $\pm$ 0.056	0.241 $\pm$ 0.010	0.141 $\pm$ 0.009	0.509 $\pm$ 0.070	0.204 $\pm$ 0.108
iStar	0.562 $\pm$ 0.037	0.223 $\pm$ 0.018	0.639 $\pm$ 0.024	0.153 $\pm$ 0.012	0.289 $\pm$ 0.020	0.094 $\pm$ 0.016	0.699 $\pm$ 0.033	0.166 $\pm$ 0.016
BiTro (w/o trans.)	0.512 $\pm$ 0.026	0.089 $\pm$ 0.034	0.513 $\pm$ 0.041	0.084 $\pm$ 0.042	0.328 $\pm$ 0.031	0.051 $\pm$ 0.021	0.587 $\pm$ 0.042	0.084 $\pm$ 0.017
BiTro (trans)	0.544 $\pm$ 0.041	0.135 $\pm$ 0.024	0.533 $\pm$ 0.047	0.103 $\pm$ 0.041	0.342 $\pm$ 0.019	0.082 $\pm$ 0.019	0.542 $\pm$ 0.036	0.113 $\pm$ 0.012

Table 3. Performance comparison on cellular transcriptomics prediction. Best results are bolded, and the second best underlined.

Figure 3. Gene expression visualization on the SPA148 sample of BRCA for gene in log1p space. A visualization showing the gene expression heatmaps for GNAS on the SPA148 dataset.

Figure 4. Visualization of BiTro’s inference on super-resolved spatial gene expression for gene EPCAM in the COAD sample TENX139 at both spot and cellular levels in the log1p space.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Note: CSCC has no Bulk transcriptomics data in this study.
Set	BRCA	COAD	LIHC (HCC)	PRAD
Training	200	246	104	152
Test	50	59	27	38
Total	250	305	131	190
Genes	734	388	185	175

Appendix

Table 4. Dataset statistics of bulk transcriptomics and WSI cases.

Metric	BRCA	COAD	CSCC	LIHC (HCC)	PRAD
Technology	ST	Visium	ST	Visium	Visium
Samples	36	15	12	2	23
Total Spots	13,620	44,823	8,671	4248	62,710
Min Spots/Sample	178	328	462	1987	1,418
Genes	785	388	469	200	200

Appendix

Table 5. Dataset statistics of spot-level ST datasets.

Metric	BRCA	COAD	LIHC (HCC)	PRAD
Technology	Xenium	Xenium	Xenium	Xenium
ID	NCBI785	TENX139	TENX120	TENX157
Total cells	167,780	388,175	162,628	193,000
Median transcripts/cell	166.0	78.0	114.0	144.0
Transcripts per $\mathrm{1}\mathrm{0}\mathrm{0}\mathrm{\SIUnitSymbolMicro}\mathrm{m}\mathrm{{}^{2}}$	106.0	182.8	85.0	371.7
Genes	278	465	474	300

Appendix

Table 6. Dataset statistics of super-resolution (cellular) ST datasets.

Configuration	BRCA		COAD		PRAD		LIHC
	O.	G.	O.	G.	O.	G.	O.	G.
BiTro.	0.743 $\pm$ 0.186	0.091 $\pm$ 0.058	0.946 $\pm$ 0.007	0.017 $\pm$ 0.014	0.831 $\pm$ 0.112	0.043 $\pm$ 0.012	0.844 $\pm$ 0.194	0.117 $\pm$ 0.102
w/o Trans.	0.711 $\pm$ 0.174	0.107 $\pm$ 0.065	0.937 $\pm$ 0.009	0.019 $\pm$ 0.008	0.821 $\pm$ 0.143	0.047 $\pm$ 0.024	0.823 $\pm$ 0.247	0.131 $\pm$ 0.074
w/o Clu+Trans.	0.708 $\pm$ 0.183	0.104 $\pm$ 0.062	0.933 $\pm$ 0.007	0.021 $\pm$ 0.009	0.814 $\pm$ 0.127	0.050 $\pm$ 0.022	0.811 $\pm$ 0.263	0.139 $\pm$ 0.093

Appendix

Table 7. Ablation study on bulk transcriptomics prediction. We compare our base and transfer models under three configurations: standard (Trans+Cluster), without clustering (w/o Clu.), without Clustering and Trans (w/o Clu+Trans). O. and G. represent Overall and Gene Pearson correlations. The best result for each dataset is bolded.

Type	Config.	BRCA		COAD		PRAD		HCC		CSCC
		O.	G.	O.	G.	O.	G.	O.	G.	O.	G.
Base	Norm+Clu.	0.578 $\pm$ 0.11	0.136 $\pm$ 0.13	0.566 $\pm$ 0.07	0.094 $\pm$ 0.10	0.572 $\pm$ 0.15	0.100 $\pm$ 0.12	0.340 $\pm$ 0.01	0.128 $\pm$ 0.03	0.574 $\pm$ 0.01	0.231 $\pm$ 0.08
	w/o Clu.	0.587 $\pm$ 0.10	0.154 $\pm$ 0.14	0.565 $\pm$ 0.07	0.103 $\pm$ 0.09	0.502 $\pm$ 0.13	0.186 $\pm$ 0.08	0.341 $\pm$ 0.02	0.128 $\pm$ 0.03	0.667 $\pm$ 0.04	0.429 $\pm$ 0.15
	w/o Norm.	0.618 $\pm$ 0.12	0.120 $\pm$ 0.09	0.485 $\pm$ 0.13	0.073 $\pm$ 0.07	0.526 $\pm$ 0.15	0.091 $\pm$ 0.10	0.074 $\pm$ 0.03	0.133 $\pm$ 0.04	0.202 $\pm$ 0.01	0.250 $\pm$ 0.02
	w/o Norm+Clu.	0.589 $\pm$ 0.09	0.088 $\pm$ 0.09	0.456 $\pm$ 0.13	0.104 $\pm$ 0.14	0.514 $\pm$ 0.12	0.127 $\pm$ 0.07	0.081 $\pm$ 0.04	0.134 $\pm$ 0.04	0.598 $\pm$ 0.06	0.424 $\pm$ 0.16
Trans.	Norm+Clu.	0.588 $\pm$ 0.10	0.162 $\pm$ 0.14	0.573 $\pm$ 0.06	0.116 $\pm$ 0.10	0.516 $\pm$ 0.13	0.204 $\pm$ 0.13	0.342 $\pm$ 0.01	0.143 $\pm$ 0.01	—	—

Appendix

Table 8. Ablation study on spatial transcriptomics prediction. We compare our base and transfer models under three configurations: standard (Norm+Cluster), without clustering (w/o Clu.), without normalization (w/o Norm.), and without clustering and normalization (w/o Norm+Clu.). O. and G. represent Overall and Gene Pearson correlations. The best result for each dataset is bolded.

Gene Expression Prediction from Histopathology Images

Gene expression prediction from histopathology images links H&E tissue morphology to molecular profiles, enabling virtual spatial transcriptomics without sequencing every sample.

See all 44 papers

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images
Sep 25, 2026Tiffanie Godelaine, Manon Dausort, Karim El Khoury +3Histopathology Image Classification Few-Shot Image Classification
ICTEAM, Universit´e Catholique de Louvain, Belgium
A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction
Apr 4, 2026Jinxi Xiang, Siyu Hou, Yuchen Li +17Spatial Transcriptomics Gene Expression Prediction from Histopathology Images
Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA · Department of Statistics and Data Science, Yale University, New Haven, CT, USA · Department of Medicine (Oncology), Stanford University School of Medicine, Stanford, CA, USA +4
Spatial Transcriptomics as Images for Large-Scale Pretraining
Mar 13, 2026Yishun Zhu, Jiaxin Qi, Jian Wang +2Spatial Transcriptomics
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Hangzhou Institute for Advanced Study, University of the Chinese Academy of Sciences, Zhejiang, China

Sep 25, 2026J/K move · Enter open · S save

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

Tiffanie Godelaine, Manon Dausort, Karim El Khoury, Benoît Gérin+2

ICTEAM, Universit´e Catholique de Louvain, Belgium

Histopathology Image Classification Few-Shot Image Classification Whole-Slide Image Analysis Few-Shot Learning

Open paper AskPDF arXiv

Automating the analysis of whole-slide images (WSIs), a key step in cancer diagnosis, has high clinical value, as it can reduce pathologist's workload while improving diagnosis accuracy. Recently, vision-language models have shown promising performance for patch-level classification without requiring any annotation, yet these zero-shot (ZS) predictions remain noisy on fine-grained tasks and must be further refined. A promising direction is to refine all predictions jointly, i.e., a transductive approach. However, most existing methods are not tailored to WSIs. We thus propose SlideTIM, an adaptation to WSIs of the recent transductive approach LC-TIM, which introduces a combined spatial--latent regularizer together with a prior on the patch class distribution. The former enforces spatially and semantically close patches to receive the same predictions, while the prior calibrates the predicted class proportions. Together, they address the complex spatial organization and the strong class imbalance of WSIs. Evaluated on four histology datasets, SlideTIM consistently outperforms all TIM variants, improving the macro-F1 by +8.1pp over the best competing baseline at 1 shot. Compared to the ZS, it raises the macro-F1 by +19.4pp at 1 shot. The code will be made available after submission.

Abstract

Gene Expression Prediction from Histopathology Images

Explore similar work

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

Spatial Transcriptomics as Images for Large-Scale Pretraining