Predicting Single-Cell Spatial Gene Expression from H&E using Pathology Foundation Models
Authors: Ashley P. Tsang, Evan Liu, Conrad Foo, Bob Chen, Hassaan Maan, Donny Chan, Elisa Penna, Veronica Ibarra Lopez, Hector Corrada Bravo, Aicha BenTaieb, Lisa M. McGinnis, Runmin Wei, Kenneth T. Gao
Organizations: AI for Biology & Translation, Computational Sciences Center of Excellence, Genentech · Gilbert S. Omenn Department of Computational Medicine & Bioinformatics, University of Michigan · Computational Biology & Medicine, Computational Sciences Center of Excellence, Genentech · Digital and Spatial Pathology, Translational Genomics and Proteomics, gRED, Genentech
Predicting gene expression directly from hematoxylin and eosin (H&E) images offers a scalable and cost-effective approach to deriving spatial molecular insights from routine histology. Pathology foundation models, typically pretrained using tile-level objectives, may not adequately capture the fine-grained cellular features required for single-cell molecular prediction. In this work, we systematically evaluate and establish domain-informed and task-specific strategies for adapting pathology foundation models to single-cell spatial gene expression prediction. Using paired H&E and Xenium 5K data, our approach improves mean gene-wise Pearson correlation by 2.67x over the standard zero-shot baseline and achieves state-of-the-art performance on this task. We demonstrate strong gene-detection performance across the full gene panel (AUROC = 0.800) and importantly identify that predictive performance varies substantially across biological contexts, with our model achieving its highest Pearson correlation for spatially variable genes (r = 0.643). Our findings demonstrate that, despite the challenges of sparse single-cell measurements, H&E-derived representations can recover biologically meaningful spatial expression patterns, highlighting both the opportunities and limitations of morphology-based molecular inference at cellular resolution.
Figures & tables
Figure 1: Single-cell predictions from our model in a tertiary lymphoid structure (TLS) region. Our model predicts expression (log1p shown) of the B-cell marker MS4A1 within the TLS, immune-cell recruitment chemokines CCL21 and CXCL9 , and plasma-cell marker MZB1 at the periphery.
Figure 2: Framework for predicting single-cell gene expression from H&E images. We integrate domain-informed design choices including high-resolution inputs, parameter-efficient encoder fine-tuning, cell-localized features, and a Poisson-trained MLP to predict expression across 5K genes.
Model
SVG50
IBD
HVG50
5K
OmiCLIP Chen et al. (2025)
0.207
0.072
0.057
0.029
BLEEP Xie et al. (2023)
0.349
0.160
0.117
0.065
GHIST Fu et al. (2025)
0.505
0.167
0.087
0.081
SpatialEx Liu et al. (2026)
0.509
0.207
0.158
0.094
Baseline †
0.469
0.182
0.151
0.065
+ MLP
0.525 ( +12.0% )
0.235 ( +28.8% )
0.195 ( +29.5% )
0.110 ( +69.0% )
Table 1: Mean per-gene PCC across cumulative model adaptations. Values in parentheses indicate relative improvement over the preceding configuration. † Configuration: linear probe, class-token embeddings, 20 × input, MSE loss, and frozen embeddings.
Figure 3: Baseline performance with frozen pathology foundation model embeddings at pseudo-spot and single-cell resolutions: (a) across gene panels; (b) per-gene over increasing expression sparsity.
Figure 4: Our domain-driven model adaptations generalize across different foundation models.
Model
Native granularity
PCC (per-gene)
PCC (per-cell)
AUROC
OmiCLIP Chen et al. (2025)
Spot-level
0.029 ± 0.006
0.203 ± 0.026
0.568 ± 0.016
BLEEP Xie et al. (2023)
Spot-level
0.065 ± 0.041
0.124 ± 0.078
0.578 ± 0.062
GHIST Fu et al. (2025)
Single-cell
0.081 ± 0.007
0.242 ± 0.027
0.605 ± 0.012
SpatialEx Liu et al. (2026)
Single-cell
0.094 ± 0.009
0.242 ± 0.031
0.642 ± 0.005
Ours (frozen)
Single-cell
0.161 ± 0.016
0.271 ± 0.034
0.786 ± 0.005
Ours (LoRA)
Single-cell
0.174 ± 0.014
0.280 ± 0.033
0.800 ± 0.009
Table 2: Benchmarking single-cell gene expression predictions from H&E on the 5K panel. We report mean per-gene PCC, mean per-cell PCC, and AUROC. Native granularity indicates the resolution-level the model was trained on; spot-level methods were evaluated on the single-cell prediction task.
Figure 5: Cell-type-specific effects of single-cell modeling adaptations. (a) Gains in performance across adaptations. (b) Performance stratified by cell type. (c) Example H&E image regions.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
PCC (5K)
Δ
Cell mask (reference)
0.161 ± 0.016
—
Disc r=5 µm, Gaussian σ=2.5 µm
0.154 ± 0.016
− 0.007 ( − 4.6%)
Disc r=5 µm, unweighted mean
0.149 ± 0.015
− 0.012 ( − 7.3%)
Class-token (baseline)
0.128 ± 0.013
− 0.035 ( − 20.2%)
Appendix
Table 3: Comparison of cell-token pooling strategies within 40 × tiles. Performance is reported as mean per-gene PCC across the 5K panel; Δ denotes the absolute and relative change from mask-guided pooling.
Figure 9
Figure 7: H&E input configurations evaluated across spatial resolutions and fields of view. Each image is centered on the target cell, with its Xenium-derived boundary shown in green and patch-token boundaries overlaid as a grid.
Input
Pixel size
FOV
Token width
Tokens/cell
PCC (5K) Δ
( μ m/px)
( μ m)
( μ m)
median (mean)
40×
0.2125
47.6
2.98
4 (5.1)
0.1611 ± 0.0158 —
20×
0.5
112
7.00
1 (1.2)
0.1521 ± 0.0125 − 5.6%
10×
1.0
224
14.0
1 (1.0)
0.1292 ± 0.0107 − 19.8%
Single-cell crop
0.2125
14.0
0.88
50 (56.8)
0.1361 ± 0.0186 − 15.5%
Appendix
Table 5: Ablation of H&E input resolution and field of view. Performance is reported as mean gene-wise PCC across the 5K panel; Δ denotes the absolute and relative change from 40 × .
Figure 8: UMAP embeddings across adaptations, colored by cell type annotations.
Figure 9: Predicted and observed single-cell expression of representative epithelial genes in a healthy region (left), and a diseased region (right).
Figure 10: Per-gene PCC as a function of gene expression sparsity for the baseline linear probe vs. our adapted model. (a) Performance is evaluated at single-cell resolution and after aggregating predicted and measured single-cell expression into (b) 55 μm and (c) 100 μm pseudo-spots.
Figure 11: Pathway enrichment analysis was done using g:Profiler and the REACTOME database on the (a) 500 most improved genes and (b) 500 least improved genes.
Predicting gene expression from H&E-stained histology images offers a scalable alternative to costly spatial transcriptomics, yet most existing methods operate at the spot level, where signals from multiple cells are aggregated and critical cellular heterogeneity is obscured. Extending this paradigm to single-cell resolution is non-trivial. Naively applying pathology foundation models faces a scale mismatch: their patch-level representations mix multiple cells, whereas per-cell cropping or resizing distorts morphology and removes local context. Conversely, segmentation-based models without strong pretrained visual encoders often lack the morphological representation capacity needed for accurate molecular prediction and inherit errors from imperfect cell boundary masks. Here, we present CELLO, an efficient end-to-end framework that performs a single pathology foundation model forward pass per image and uses grid sampling to extract location-specific features for all cells simultaneously. We further introduce a distance-decay cross-attention module that refines each cell representation using spatially biased local morphological context. Using 52 public Xenium-H&E pairs from HEST-1k that span 12 organs and approximately 10 million cells, CELLO improves the average predictive accuracy over the evaluated baselines while reducing the mean whole-slide inference time compared to DeepSpot2Cell, a 14.0x speed-up on average that excludes upstream cell segmentation. Our work establishes a scalable foundation for single-cell gene expression prediction from H&E images.
Zijun Gao, Chunbin Gu, Jinxi Xiang +2
The Chinese University of Hong Kong · Stanford University School of Medicine
Spatial transcriptomics can resolve gene expression at single-cell resolution, but it is costly, limited to targeted panels of a few hundred to a few thousand genes, and applicable to only a small number of samples. H&E imaging, by contrast, is cheap and collected routinely at scale. This makes predicting single-cell expression directly from morphology a practical way to bring molecular analysis to large tissue archives. We therefore present VOICE, a multimodal foundation model that predicts single-cell gene expression from H&E images using paired Xenium data. VOICE first aligns cell centered H&E morphology from a pathology foundation model with single-cell expression embeddings from a transcriptome foundation model, trained using contrastive learning over 23 million cells. Next it predicts expression through two branches. One branch directly regresses expression from morphology. The other branch retrieves measured expression from similar reference cells, recovering genes that do not have morphological signal. Because genes vary in morphological predictability, VOICE fuses the two branches with a per-gene weight. After training, VOICE generalizes to heldout patients, slides, and partially overlapping gene panels from Xenium, and it consistently outperforms prior single-cell expression prediction methods on seven metrics.
Xin Luo, Yicheng Tao, Haoxuan Zeng +6
University of Michigan, Ann Arbor, Michigan, USA · Weill Cornell Medicine, New York, New York, USA
Predicting spatial gene expression from routine hematoxylin and eosin (H&E) images provides a practical complement to experimental spatial transcriptomics. Existing approaches focus on local or multi-scale visual features and often treat pretrained gene representations as fixed priors, although the interpretation of local morphology and the relevance of gene priors depend on tissue context. We propose HistoGPA, a context-conditioned gene-prior attention framework that uses a shared slide-level representation in two parallel pathways: one modulates local morphological features, whereas the other conditions pretrained gene embeddings and retrieves gene-prior information through cross-attention. This design enables each spatial location to retrieve context-adapted gene-prior information using its local morphology, position, and slide context. Across ten cancer types in HEST-1k, HistoGPA achieves the highest macro-averaged gene-wise Pearson correlation coefficient among the compared methods under the same evaluation protocol for both the top-50 and top-1,500 highly variable gene sets. Additional analyses show that HistoGPA better recovers the spatial expression patterns of cancer-associated genes and yields greater agreement between clusters derived independently from predicted and ground-truth expression profiles. Together, these findings motivate a context-dependent view of histology-to-expression prediction, in which local morphological representations and gene priors are jointly adapted to the broader tissue context.
Ziang Liu, Xinhai Chen, Yigui Feng +3
Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology · National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China