3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology
Authors: Mohammad Vali Sanian, Arshia Hemmat, Amirhossein Vahidi, Jonas Maaskola, Jimmy Tsz Hang Lee, Martin Prete, Stanislaw Makarchuk, Yeliz Demirci, Nana-Jane Chipampe, Muzlifah Haniffa, Omer Bayraktar, Lassi Paavolainen, Mohammad Lotfollahi
Organizations: Computer Science Department, University of Helsinki · Institute for Molecular Medicine Finland, University of Helsinki · Wellcome Sanger Institute · Computer Science Department, University of Oxford · Cambridge Center for AI in Medicine, University of Cambridge · Department of Medicine, University of Cambridge · Cambridge Stem Cell Institute, University of Cambridge
A 3D transcriptomic profile of a tissue can give a holistic view of its organization in health and disease, but spatial transcriptomics (ST) is costly, so usually only some serial sections are profiled, while H&E is available for all. Most methods that infer ST from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flow-matching framework that imputes spot-level gene expression on an unprofiled section from its H&E and the measured expression of adjacent profiled sections. Our key idea is to take the measured expression at the same registered position on neighboring sections and feed this cross-section context to the flow model through a lightweight ControlNet, so that predictions follow anatomical continuity along z. To capture the discrete, zero-inflated nature of counts, the flow starts from a learned zero-inflated negative binomial (ZINB) prior that, on an unprofiled section, is predicted from the measured expression of the nearest profiled section. A global attention block with a small set of inducing tokens adds whole-slide context at a cost linear in the number of spots, so training and inference scale to large 3D ST datasets. Across three ST datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. We envision HoloTea advancing the creation of accurate 3D virtual tissues, ultimately accelerating biomarker discovery and deepening our understanding of disease. Code is available at https://github.com/Lotfollahi-lab/HoloTea.
Figures & tables
Figure 1: Method overview. For each query spot on section z , HoloTea takes the nearest spot by position on sections z±1 as adjacent tokens and uses a ControlNet to condition the flow-matching denoiser with ZINB-based priors.
HER2
ST-Data (images)
In-house embryo
Model
MSE ( ↓ )
MAE ( ↓ )
PCC ( ↑ )
MSE ( ↓ )
MAE ( ↓ )
PCC ( ↑ )
MSE ( ↓ )
MAE ( ↓ )
PCC ( ↑ )
SPOT
GENE
SPOT
GENE
SPOT
GENE
UNI
1.093
0.859
0.509±94
0.259±231
0.769
0.699
0.569±77
0.332±149
0.261
0.296
0.506±224
0.613±203
CIGA
0.865
0.738
0.494±135
0.465±143
0.783
0.706
0.541±83
0.247±137
0.449
0.438
0.399±194
0.427±229
Gigapath
0.611
0.615
0.581±126
0.576±161
0.674
0.649
0.597±76
0.364±145
0.314
0.322
0.457±223
0.554±225
STFlow
0.477
0.534
0.740±125
0.620±146
0.443
0.532
0.775±80
0.540±142
0.255
0.270
0.557±248
0.663±166
Table 1: Gene-expression imputation on held-out sections (HVG-250 panel). HER2 and ST-Data use split (i) (one ST-labeled section per stack), averaged over its four folds; the in-house embryo uses split (ii) (even-slice hold-out). PCC SPOT is the Pearson correlation across genes within each spot, averaged over spots; PCC GENE is the Pearson correlation across spots for each gene, averaged over genes. ± is the standard deviation across spots (PCC SPOT) or across genes (PCC GENE) within an evaluation, averaged over folds; it is not a variation across splits. Our Method (HoloTea) is the full model: learned ZINB prior (frozen, defines the flow start), ControlNet conditioning and the cross-slide branch. STFlow uses a fixed global ZINB prior without these components and shares HoloTea’s backbone and hyperparameters; UNI, CIGA and GigaPath are image-encoder baselines trained with the same protocol ( appendix S3 ). Best results in bold.
Figure 2: HoloTea imputed expression preserves anatomical and biological structure. ( a ) Spatial expression of four marker genes on section 70 (GT top, imputed bottom); rightmost column: GT anatomical annotation (top) and classifier prediction from imputed expression (bottom). ( b )–( d ) Joint UMAP of GT and imputed expression colored by source, section, and anatomical region. ( e ) Leave-one-section-out anatomy classification accuracy (logistic regression). ( f ) Spearman r between GT and imputed PROGENy pathway scores (14 pathways, n=167,584 spots). See figs. S2 , S3 , S5 , S4 and S5 for extended results.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Imputed AHSP expression on a held-out embryo section at flow steps 1, 3 and 5: HoloTea (top), STFlow (bottom), and ground truth (right).
Parameter
Default
Backbone layers L
4
Hidden dimension d
128
Pairwise / edge dimension
128
Attention heads
4
k nearest neighbors
8
Dropout / Attention dropout
0.2 / 0.2
Appendix
Table S1: Core model hyperparameters (defaults).
Parameter
Default / Setting
Batch size (slide-level)
2
Epochs
100
Optimizer / Learning rate
Adam / 5×10−4
Gradient clipping (L2)
1.0
Time conditioning
Sinusoidal → MLP
Inference steps ( S )
5
Appendix
Table S2: Training and inference hyperparameters.
Component
Default / Setting
Prior sampler
ZINB (default)
ZINB parameters
total_count =1 , logits =0.1 , zi_logits =0.0
Learned ZINB prior ( ϕe )
optional (frozen at train time)
ControlNet conditioning
optional; channels =32 , grid =64 , scale =1.0
Appendix
Table S3: Priors and optional modules.
Encoder
Patch size
Output dim
UNI v1 (official)
224×224
1024
GigaPath
224×224
1536
CIGA
224×224
512
Appendix
Table S4: Foundation encoders and node-feature dimensions. Encoders are frozen.
Figure S2: Extended spatial gene expression grid across all three annotated sections. Each pair of rows shows GT and HoloTea imputed expression for sections 70, 84, and 91. Gene expression columns (ZIC1, PAX1, NR2E1, MYL2) are followed by four annotation columns: GT anatomical annotation (GT rows only); logistic regression, k -nearest neighbour ( k=15 ), and random forest predictions trained on GT or imputed expression.
Figure S3: Per-region anatomy classification accuracy (logistic regression, leave-one-section-out CV). Each bar pair shows accuracy for one of the 19 annotated anatomical regions, comparing classifiers trained on GT (blue) vs. imputed (red) expression of the 211-gene panel across sections 70, 84, and 91. Regions are sorted by GT accuracy.
Accuracy
Macro F1
Classifier
Section
GT
Imp
GT
Imp
Logistic reg.
70
0.829
0.824
0.696
0.608
84
0.773
0.882
0.633
0.701
91
0.820
0.834
0.659
0.628
Overall
0.807
0.847
0.664
0.657
k -NN ( k=15 )
70
0.866
0.800
0.745
0.653
Appendix
Table S5: Anatomy classification performance (leave-one-section-out CV). Accuracy and macro-averaged F1 score for classifiers trained on GT vs. HoloTea imputed expression of the 211-gene panel (sections 70, 84, 91; 19 anatomical classes). Bold indicates the higher value within each GT/Imp pair.
Figure S4: Per-region marker gene enrichment correlation (GT vs. imputed). Each panel shows the per-gene log2 fold-change between one annotated anatomical region and the rest, computed for GT (x-axis) and HoloTea imputed expression (y-axis) across sections 70, 84, and 91. Spearman r is shown per panel.
Figure S5: Per-pathway PROGENy activity score scatter (GT vs. imputed). Each panel shows per-spot GT (x-axis) vs. HoloTea imputed (y-axis) pathway activity scores for one of the 14 PROGENy signaling pathways, across all 26 held-out sections ( n=167,584 spots). Spearman r and p -value are shown per panel.
m
MSE ↓
MAE ↓
PCC SPOT (mean ± std) ↑
PCC GENE (mean ± std) ↑
0
0.255
0.270
0.557 ± 0.248
0.663 ± 0.166
4
0.207
0.237
0.571 ± 0.244
0.681 ± 0.171
8
0.212
0.241
0.570 ± 0.246
0.680 ± 0.173
16
0.210
0.240
0.572 ± 0.246
0.675 ± 0.174
32
0.206
0.236
0.567 ± 0.244
0.680 ± 0.173
64
0.204
0.236
0.568 ± 0.245
0.679 ± 0.170
Appendix
Table S6: Effect of m (powers of two) on performance for the EMBRYO dataset. This is a sensitivity (scaling) study of how much global-attention capacity helps; the m=0 row is STFlow. Lower is better for MAE/MSE; higher is better for PCC SPOT / PCC GENE.
m
MSE ↓
MAE ↓
PCC SPOT (mean ± std) ↑
PCC GENE (mean ± std) ↑
0
0.443
0.532
0.775 ± 0.080
0.540 ± 0.142
4
0.412
0.502
0.781 ± 0.014
0.561 ± 0.101
8
0.403
0.491
0.782 ± 0.012
0.563 ± 0.098
16
0.392
0.480
0.784 ± 0.012
0.571 ± 0.012
32
0.382
0.479
0.786 ± 0.012
0.572 ± 0.038
64
0.384
0.481
0.780 ± 0.045
0.569 ± 0.019
Appendix
Table S7: Effect of m on performance for the ST dataset. This is a sensitivity (scaling) study of how much global-attention capacity helps; the m=0 row is STFlow. table 1 uses a fixed m=64 . Lower is better for MAE/MSE; higher is better for PCC SPOT / PCC GENE.
m
MSE ↓
MAE ↓
PCC SPOT (mean ± std) ↑
PCC GENE (mean ± std) ↑
0
0.477
0.534
0.740 ± 0.125
0.620 ± 0.146
4
0.473
0.533
0.739 ± 0.010
0.618 ± 0.013
8
0.471
0.533
0.740 ± 0.011
0.62 ± 0.037
16
0.462
0.530
0.741 ± 0.023
0.623 ± 0.012
32
0.472
0.533
0.740 ± 0.021
0.621 ± 0.023
Appendix
Table S8: Effect of m on performance for the HER2 dataset. This is a sensitivity (scaling) study of how much global-attention capacity helps; the m=0 row is STFlow. Lower is better for MAE/MSE; higher is better for PCC SPOT / PCC GENE.
HER2
ST
EMBRYO
378
482
6299
Appendix
Table S9: Average number of spots per dataset.
Figure S6: Imputed gene panels per datasets HER2, ST-Dataset, In-house Embryo HVG and Custom
Figure S7: HoloTea leverages spatial neighborhood information from adjacent sections to improve 3D consistency in H&E to ST imputation.
Model
MSE ( ↓ )
MAE ( ↓ )
PCC ( ↑ )
Spot
Gene
UNI
0.412
0.398
0.554±0 .249)
0.663±0 .111)
STFlow
0.387
0.381
0.581±0 .103)
0.700±0 .104)
ASIGN 3D
0.689
0.71
0.362±0 .178)
0.442±0 .013)
Ours
0.330
0.343
0.598(0.105)
0.726(0.105)
Appendix
Table S10: Embryo imputation with the custom gene panel. Same split as table 1 (split (ii), even-slice hold-out); this is not a second split. Only the gene panel differs: the custom panel ( ∼ 150 genes highly variable across bins of sections spread along the stack; see Gene sets in the experimental settings) replaces HVG-250. Metrics, uncertainties (standard deviation across spots for PCC Spot and across genes for PCC Gene) and model configurations are as in table 1 ; Ours is the full HoloTea. Best results in bold.
Model
# Spots ( N )
GFLOPs
Memory [GB]
ASIGN (3D)
327532
6032.08
79.8
Our method (Train)
327532
333.21
26.9
Our method (Inference)
327532
114.9
3.7
Appendix
Table S11: Computational complexity on in-house embryo dataset: number of spots ( N ), forward-pass FLOPs, and peak GPU memory.
Regime
Genes
MSE ↓
MAE ↓
PCC(GENE) ↑
Vanilla STFlow
50
0.160
0.107
0.693
100
0.177
0.120
0.688
200
0.176
0.119
0.673
500
0.164
0.124
0.646
Learned Prior
50
0.181
–
0.691
100
0.175
–
0.690
Appendix
Table S12: Gene-level performance (median per-gene MSE, mean per-gene PCC) on the even-slice hold-out split for different ablation settings on the in-house embryo dataset. For each panel size, the best value is shown in bold . Lower is better for MSE/MAE; higher is better for PCC. Rows are cumulative: Vanilla STFlow, then the learned prior alone, then ControlNet, then the cross-slide branch. –: MAE was not recorded for this run.
Dataset
Method
PCC-gene ↑
MSE ↓
HER2
TRIPLEX
0.335±0.07
0.486
HoloTea (ours)
0.638±0.15
0.455
ST-Data
TRIPLEX
0.311±0.07
0.980
HoloTea (ours)
0.612±0.13
0.364
Appendix
Table S13: Comparison with the per-section predictor TRIPLEX [ 4 ] , re-trained under our protocol on the two public datasets (best in bold).
Method
PCC-gene ↑
PCC-spot ↑
Interpolation (nearest spot)
0.647
0.521
Interpolation (4 nearest spots)
0.688
0.582
STFlow
0.650±0.004
0.556±0.003
HoloTea (ours)
0.697±0.001
0.583±0.003
Appendix
Table S14: Interpolation baseline versus STFlow and HoloTea on the in-house embryo (even-slice hold-out, 150-gene panel, mean ± sd over 3 seeds, last checkpoint). STFlow and HoloTea are re-runs and differ slightly from the single runs in table 1 . Interpolation copies measured expression from the closest profiled sections below and above (nearest spot by (a,b) , or mean of the 4 nearest spots, averaged over the two sections). Best in bold.
Spatial transcriptomics (ST) provides spatially resolved gene expression measurements, but most 3D ST datasets are acquired as independently processed 2D sections. Computational approaches span slice registration and integration, explicit or generative interpolation, and coordinate-field modeling. Within the last family, implicit neural representations (INRs) offer a continuous, coordinate-queryable parameterization, but standard isotropic coordinate encodings do not explicitly reflect that serial-section measurements are dense in-plane and sparse along the axial direction. We present HoloGene, an anisotropic conditional INR framework for estimating continuous 3D gene-expression fields from pre-registered ST sections. Its central contribution is an acquisition-aware factorization that couples dense in-plane synthesis with sparse axial evolution using established GAE, FiLM, pseudo-label supervision, and CORAL components. HoloGene compresses high-dimensional expression profiles into a graph-autoencoder latent manifold, uses pseudo-label-derived signature conditioning to provide coarse intra-section context, and applies depth-dependent FiLM modulation to model axial variation. A signature-stratified correlation-alignment regularizer encourages cross-slice covariance alignment in latent features. Using held-out measured sections as cross-sectional references, HoloGene improves reconstruction accuracy and biological preservation metrics compared with general-purpose INRs and adapted spatial-transcriptomics baselines. These results support anisotropic conditional modeling as a practical strategy for estimating 3D expression fields from discontinuous ST sections. Code is available at https://github.com/Szym29/HoloGene.
Yumin Zheng, Qingtian Zhu, Ziyan Zhu +3
McGill University · MUHC Research Institute · The University of Tokyo +2
Three-dimensional spatial transcriptomics (3D ST) enables the analysis of gene-expression patterns across tissue depths. However, 3D ST measurements are acquired from discrete 2D tissue sections, which are often sparsely sampled due to high acquisition costs. To address this limitation, we propose a histology-informed multi-task implicit neural representation (HIM-INR) for continuous 3D gene-expression reconstruction from sparse ST observations. HIM-INR jointly reconstructs gene-expression profiles and histological representations using an INR with a shared backbone, leveraging densely sampled histological images as auxiliary spatial supervision. Once trained, HIM-INR predicts gene expression at arbitrary 3D coordinates without requiring histological images. Experiments on two 3D ST datasets demonstrate improved reconstruction over existing methods, particularly as the number of missing ST sections increases.
Spatial Transcriptomics (ST) measures gene expression while preserving spatial context, but its high cost and low throughput leave public datasets small. Inferring expression directly from widely available Hematoxylin and Eosin (H&E) stained histology offers a cost-effective alternative. However, existing approaches face several limitations: regression methods over-smooth toward the conditional mean, while generative methods are faithful but require slow multi-step inference; most methods treat genes as independent and equally important, ignoring inter-gene dependencies and heterogeneous gene informativeness; and most are tailored to a single resolution, either spot-level or cell-level. To address these issues, we propose DriftST, a unified framework for inferring spatially resolved gene expression from H&E images. DriftST builds on a Cellular Drifting generative model that learns a direct drift from a histology-conditioned source to the expression distribution, retaining generative expressiveness while enabling efficient one-step generation. To capture gene structure, we introduce the STransformer, which combines a co-expression attention module for inter-gene dependencies with a gene residual gate for differential gene importance. Operating on a generic gene-panel representation, DriftST applies directly to both spot-level and cell-level data in one framework, and extensive experiments across diverse tissues and platforms show that it achieves state-of-the-art performance at both resolutions.
Yuhang Yang, Yonggan Bu, Shengyuan Zhou +2
University of Science and Technology of China, Hefei, China · Peking University Cancer Hospital, Beijing, China