Path2ST: Hierarchical Cell-Tissue Grounded Cross-Modal Translation for Spatial Transcriptomics
Authors: Ruochen Liu, Wei Lou
Organizations: Faculty of Science and Engineering University of Liverpool Liverpool, United Kingdom · College of Mathematical Medicine Zhejiang Normal University Jinhua, China
Predicting spatial gene expression from hematoxylin and eosin (H&E)-stained images offers a cost-effective alternative to spatial transcriptomics (ST). However, existing methods treat H&E images as generic visual inputs and ignore their intrinsic biological hierarchy, where spatially organized cell types collectively form functional tissue microenvironments that govern local gene expression programs. To bridge this gap, we formulate H&E-to-ST prediction as a cross-modal semantic translation task and propose Path2ST, a hierarchically grounded autoregressive framework featuring three key components: (i) a Hierarchical Cell-Tissue Conditioning mechanism that fuses explicit and implicit cellular features with tissue-level semantic representations to construct hierarchical conditioning signals; (ii) a Scale-Adaptive Autoregressive Generation process over a hierarchical semantic vocabulary, enabling coarse-to-fine, biologically consistent expression synthesis; and (iii) SpectraLoss, a full-spectrum objective that jointly enforces ordinal fidelity, models transcriptional bursts, and aligns semantic structures with cell types. Extensive experiments on three datasets demonstrate state-of-the-art performance, validating that Path2ST generates highly accurate and spatially coherent transcriptomic profiles. The related code is released at https://github.com/RuochenLiu23/Path2ST.
Figures & tables
Figure 1. Pathological images exhibit an intrinsic hierarchical structure highly analogous to natural language, framing spatial transcriptomics generation as a translation task facilitates comprehensive modeling of biologically meaningful hierarchical associations, grounding the generation process in authentic biological semantics.
Figure 2. Overview of the Path2ST framework. The proposed framework comprises three key components: Hierarchical Cell-Tissue Conditioning for constructing biologically grounded generation conditions, Scale-Adaptive Autoregressive Generation for coarse-to-fine multi-scale gene expression generation, and SpectraLoss, a composite objective combining GKL, ZINB, and semantic contrastive losses.
Figure 3. The Hierarchical Cell-Tissue Conditioning module integrates cellular and tissue information via two components: a Hybrid Explicit-Implicit Cross-Attention that captures tissue-context-aware cellular semantic features, and a Dynamic Gating mechanism that adaptively controls the injection strength of fused cellular signals.
Method
PRAD
HER2ST
Healthy Mouse Brain
PCC-10 ↑
PCC-50 ↑
PCC-200 ↑
MSE ↓
MAE ↓
PCC-10 ↑
PCC-50 ↑
PCC-200 ↑
MSE ↓
MAE ↓
PCC-10 ↑
PCC-50 ↑
PCC-200 ↑
MSE ↓
MAE ↓
BLEEP
0.580
0.510
0.316
2.475
1.091
0.773
0.714
0.565
1.243
0.833
0.342
0.280
0.156
1.591
0.987
TRIPLEX
0.620
0.544
0.423
1.319
0.836
0.783
0.714
0.586
1.212
0.857
0.501
0.445
0.312
1.157
0.822
M2OST
0.602
0.551
0.442
1.290
0.862
0.810
0.759
0.660
1.151
0.820
0.456
0.387
0.231
1.148
0.861
STEM
0.636
0.555
0.403
1.457
0.857
0.831
0.770
0.625
1.199
0.787
0.526
0.452
0.331
1.235
0.864
GenAR
0.702
0.650
0.512
1.191
0.771
0.842
0.784
0.663
1.082
0.745
0.568
0.503
0.367
1.138
0.805
Table 1. Comparison with state-of-the-art methods across PRAD, HER2ST, and Healthy Mouse Brain datasets. ↑ indicates higher is better, ↓ indicates lower is better. Best results are in bold .
Method
PCC-10 ↑
PCC-50 ↑
PCC-200 ↑
MSE ↓
MAE ↓
Baseline
0.736
0.659
0.519
1.148
0.753
+ Hierarchical Conditioning
0.748
0.673
0.549
1.080
0.738
+ Scale-Adaptive Autoregressive Generation
0.752
0.679
0.554
1.069
0.736
+ ZINB Loss
0.756
0.686
0.558
1.058
0.733
+ Soft-Positive Semantic Contrastive Loss (Ours)
0.767
0.701
0.578
1.005
0.725
Table 2. Ablation study on PRAD dataset.
Query
PCC-10 ↑
PCC-50 ↑
PCC-200 ↑
MSE ↓
MAE ↓
1
0.767
0.701
0.578
1.005
0.725
2
0.757
0.685
0.553
1.048
0.733
3
0.751
0.671
0.535
1.069
0.765
4
0.765
0.689
0.557
1.068
0.739
5
0.753
0.668
0.514
1.096
0.771
Table 3. Hyperparameter analysis on the query number Nq on PRAD dataset.
Figure 4. Visualization of cell classification and neoplastic cell clustering on the MEND145 sample from PRAD, alongside spatial expression maps of cancer biomarker genes SORD, TSPAN1, and TMPRSS2, where darker colors indicate lower gene expression at the corresponding spot.
Spatial transcriptomics (ST) links gene expression with tissue morphology but remains expensive and low-throughput, motivating surrogates that infer expression from routine histology. Whole-slide H&E-to-ST inference pairs a gigapixel image with gene measurements at a sparse, irregular set of locations, making multiscale modeling challenging without incurring dense-grid overhead or quadratic token mixing. We propose HiST, a hierarchical sparse transformer that treats measured locations as a lattice-indexed sparse field and builds a dyadic encoder--decoder directly on the active tissue footprint. HiST combines sparse window attention for local geometric correspondence with resolution-changing operators for rapid multiscale context integration. For a fixed window size, the dominant runtime and memory scale with the number of observed locations rather than the dense slide area. To mitigate slide-specific acquisition variation, HiST adds a bottlenecked global conditioning pathway via a slide calibration token that summarizes slide-level context and conditions local representations. On a multi-organ benchmark spanning diverse tissues and acquisition sources, HiST improves predictive performance over recent baselines while reducing runtime and peak memory.
Weiyi Wu, Xinwen Xu, Xingjian Diao +4
Dartmouth College · Mass General Hospital · New Jersey Institute of Technology +1
Spatial Transcriptomics (ST) measures gene expression while preserving spatial context, but its high cost and low throughput leave public datasets small. Inferring expression directly from widely available Hematoxylin and Eosin (H&E) stained histology offers a cost-effective alternative. However, existing approaches face several limitations: regression methods over-smooth toward the conditional mean, while generative methods are faithful but require slow multi-step inference; most methods treat genes as independent and equally important, ignoring inter-gene dependencies and heterogeneous gene informativeness; and most are tailored to a single resolution, either spot-level or cell-level. To address these issues, we propose DriftST, a unified framework for inferring spatially resolved gene expression from H&E images. DriftST builds on a Cellular Drifting generative model that learns a direct drift from a histology-conditioned source to the expression distribution, retaining generative expressiveness while enabling efficient one-step generation. To capture gene structure, we introduce the STransformer, which combines a co-expression attention module for inter-gene dependencies with a gene residual gate for differential gene importance. Operating on a generic gene-panel representation, DriftST applies directly to both spot-level and cell-level data in one framework, and extensive experiments across diverse tissues and platforms show that it achieves state-of-the-art performance at both resolutions.
Yuhang Yang, Yonggan Bu, Shengyuan Zhou +2
University of Science and Technology of China, Hefei, China · Peking University Cancer Hospital, Beijing, China
Spatial Transcriptomics (ST) has transformed biomedical research by enabling the spatial mapping of gene expression across tissue sections. However, high operational costs, specialized equipment requirements, and sensitivity to experimental noise limit the accessibility and scalability of ST. Recent computer vision approaches aim to overcome these limitations by predicting spatial gene expression directly from histopathology images. While effective, current approaches often suffer from gene expression over-smoothing and overly uniform predictions across tissue regions, suggesting that further progress depends on learning representations that reflect the hierarchical and asymmetric structure of gene regulation and tissue morphology. To address these issues, we propose Hyperbolic Contrastive Learning with Entailment for Spatial Transcriptomics (HyCLoST), a hyperbolic contrastive learning model that captures the intrinsic hierarchical relationships within ST data. By leveraging hyperbolic geometry and a gene-to-image entailment loss, HyCLoST learns structured, biologically grounded representations that improve gene expression prediction accuracy, achieving a 6% reduction in MSE and an 8% increase in PCC across 26 ST datasets, over previous methods. Our source code is publicly available at https://github.com/BCV-Uniandes/HyCLoST
Daniela Vega, Paula Cárdenas, Hannah Ceballos +2
Center for Research and Formation in Artificial Intelligence Universidad de los Andes, Colombia