Is H&E Image-to-Spatial Transcriptomics Simpler Than It Looks?

Duc T. Nguyen; Thanh Ha Do; Phuong M. Cao; Hieu Pham

cs.CVSep 26, 2026

Is H&E Image-to-Spatial Transcriptomics Simpler Than It Looks?

Authors: Duc T. Nguyen, Thanh Ha Do, Phuong M. Cao, Hieu Pham

Organizations: VinUniversity, Hanoi, Vietnam · VinUni-Illinois Smart Health Center (VISHC), VinUniversity, Hanoi, Vietnam · Posts and Telecommunications Institute of Technology, Hanoi, Vietnam · National Center for Supercomputing Applications, University of Illinois Urbana-Champaign, Urbana, IL, USA

Abstract

Predicting spatial gene expression from routine H&E histology offers a scalable route toward spatial molecular profiling. Recent work has pursued increasingly sophisticated architectures to capture spatial context and richer expression structure. At the same time, simple estimators have shown strong performance in several studies, but what they already solve and where additional complexity is needed remain unclear. We study this behavior through the structure of prediction error under the mean-squared error (MSE) objective. Differences in average expression across genes can account for a substantial part of aggregate prediction performance, while a key unresolved error lies in recovering variation within each slide. Decomposing MSE into slide-level and within-slide components, we find that the within-slide component has lower residual-normalized parameter sensitivity in controlled neural experiments. This motivates Component-Guided Loss (CGL), which increases supervision of the within-slide component. CGL-Linear is a closed-form affine instantiation that achieves overall state-of-the-art performance across HEST-1k cohorts and gene-panel sizes. The same within-slide supervision improves existing neural models. These results suggest that substantial gains can come from aligning the training objective with prediction-error structure rather than increasing model complexity.

Figures & tables

Figure 1: Slide-wise expression structure. For one gene on one slide, observed expression can be viewed as a slide-wise mean together with variation around that mean. Each dot denotes a measured spatial location, and the H&E image is shown for spatial context.

Method	Venue	$\mathrm{PCC}_{\mathrm{flat}}$	$\mathrm{PCC}_{\mathrm{gene}}$	$R^{2}_{\mathrm{flat}}$	$R^{2}_{\mathrm{local}}$
MeanOnly	–	$0.6921$	N.A.	$0.4280$	$0.0000$
Linear	–	$0.7226$	$0.1354$	$0.4953$	$-0.0423$
HiST	ICML’26	$0.7363$	$0.1343$	$0.5067$	$0.0805$
HistoPrism	ICLR’26	$0.7372$	$0.1337$	$0.5032$	$0.0921$
HEXST	ICML’26	$0.7126$	$0.1560$	$0.4547$	$0.0497$
STFlow	ICML’25	$0.7053$	$0.1366$	$0.4236$	$-0.0565$

Table 1: Mean-only reference and learned predictors with UNI and

G=500

. Scores are equal-cohort test averages over ten cohorts. Gene PCC is undefined for MeanOnly because its prediction is constant for every gene. Bold marks the highest mean in each column.

Figure 2: Component sensitivity under standard MSE. Curves show the cohort median of

\kappa_{P}/\kappa_{C}

for DINOv2 and ResNet18 on training and validation data. Shaded bands show cohort interquartile ranges. Values above one indicate greater residual-normalized sensitivity for the slide-level component.

Figure 3: Effect of CGL on the controlled MLP. For DINOv2 (a) and ResNet18 (b), the left and middle columns show

\mathcal{L}_{j}^{\mathrm{MSE}}-\mathcal{L}_{j}^{\mathrm{CGL}}

on training and validation data. Positive values indicate lower error under CGL. Lines show equal-cohort means and shaded regions show cohort interquartile ranges. The right column reports test metric changes from MSE to CGL.

Figure 4: Benchmark results across image representations and gene panels. Each point is an equal-cohort macro test score over ten cohorts. Neural methods are averaged over three seeds, while Linear and CGL-Linear are deterministic. All methods use matched image representations, data splits, and targets. Higher is better for every metric.

Figure 5: Computational scaling. Total training time, inference time, and model size are compared as the number of spatial locations increases.

Figure 6: Adding within-slide supervision to existing models. Cells show equal-cohort macro test gains over the corresponding base method across ten cohorts using DINOv2 representations. Positive values favor the augmented objective.

Figure 7: Spatial expression predictions on held-out slides. Row (A) shows MUC2 in colon slide MISC71, and row (B) shows MYH6 in heart slide MISC138. Each row contains the H&E image, measured expression, and predictions from the displayed methods.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Cohort	Slides (train / val / test)			Spots (train / val / test)
Brain	9	2	3	23,780	4,891	4,219
Colon	33	4	4	94,871	6,359	15,138
Heart	26	7	8	70,802	14,100	14,195
Kidney	24	5	5	27,739	5,649	5,691
Liver	10	2	2	26,518	5,695	4,955
Lung	12	4	4	20,810	8,763	8,933

Appendix

Table 2: Cohort sizes in the fixed canonical split. Slide counts are recovered from the

500

-gene evaluation support; spot counts are the aligned H&E–expression pairs used by the experiments.

Method	Epochs	Readout configuration	Optimization
HiST	300	Sparse hierarchy 111–21, window size 3, feed-forward ratio 4, three global tokens.	AdamW, LR $10^{-4}$ , WD $10^{-5}$ , cosine schedule, centered-target MSE.
HistoPrism	300	Hidden size 256, two Transformer layers, eight heads, dropout 0.1.	AdamW, LR $5\times 10^{-5}$ , WD $10^{-2}$ , linear decay with 5% warm-up, centered-target MSE.
HEXST	300	Hidden size 512, HexMSA/HexRoPE, 3,072-dimensional scFoundation alignment, dropout 0.2.	Adam, LR $4\times 10^{-5}$ , WD $10^{-4}$ , warm-up and cosine schedule, composite regression/correlation/deviation/alignment objective.
STFlow	300	Hidden size 128, four layers, four heads, eight neighbors, five sampling steps.	Adam, LR $5\times 10^{-4}$ , gradient clipping at 1.0, no scheduler, conditional flow-matching objective.
BLEEP	300	Dual 256-dimensional projections, dropout 0.1, temperature 1, top-50 retrieval.	AdamW, LR $10^{-3}$ , WD $10^{-3}$ , batch size 512, symmetric contrastive cross-entropy.
SEPAL	$300$	Image readout followed by a six-neighbor graph head, relative positional encoding, two GCN layers, SAGPooling.	SGD, LR 0.0099913, momentum 0.9, batch size 32, MSE in both stages, frozen first-stage predictor during graph fitting.

Appendix

Table 3: Configurations of the compared predictors. LR denotes learning rate and WD denotes weight decay.

Method	$\lambda$
HiST	10
HistoPrism	15
HEXST	20
STFlow	10
SEPAL	5

Appendix

Table 4: Validation-selected component weights used in the neural augmentation experiments.

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$\underline{0.775}\,\pm\,0.001$	$0.772\,\pm\,0.004$	$0.764\,\pm\,0.001$	$0.754\,\pm\,0.010$	$0.702\,\pm\,0.026$	$0.762\,\pm\,0.008$	$0.735$	$\mathbf{0.776}$
Colon	$\underline{0.744}\,\pm\,0.018$	$0.731\,\pm\,0.009$	$0.719\,\pm\,0.003$	$0.710\,\pm\,0.002$	$0.622\,\pm\,0.068$	$0.713\,\pm\,0.011$	$0.713$	$\mathbf{0.758}$
Heart	$0.789\,\pm\,0.009$	$\mathbf{0.804}\,\pm\,0.007$	$0.647\,\pm\,0.002$	$\underline{0.791}\,\pm\,0.005$	$0.771\,\pm\,0.009$	$0.768\,\pm\,0.018$	$0.762$	$0.736$
Kidney	$0.734\,\pm\,0.006$	$0.714\,\pm\,0.000$	$0.719\,\pm\,0.000$	$0.676\,\pm\,0.004$	$0.702\,\pm\,0.014$	$\underline{0.745}\,\pm\,0.011$	$0.727$	$\mathbf{0.747}$
Liver	$0.864\,\pm\,0.003$	$0.871\,\pm\,0.002$	$0.858\,\pm\,0.001$	$\underline{0.872}\,\pm\,0.000$	$0.858\,\pm\,0.001$	$0.858\,\pm\,0.004$	$0.862$	$\mathbf{0.874}$

Appendix

Table 5: Cohort-level test results with UNI and

G=500

. Neural entries are mean

\pm

seed standard deviation over three runs. Linear and CGL-Linear are deterministic. Bold and underlining mark the best and second-best unrounded means in each row. Macro rows average cohort means without weighting by cohort size. Higher is better.

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.807\,\pm\,0.008$	$\underline{0.808}\,\pm\,0.004$	$0.800\,\pm\,0.001$	$0.797\,\pm\,0.029$	$0.748\,\pm\,0.003$	$0.799\,\pm\,0.007$	$0.779$	$\mathbf{0.812}$
Colon	$\underline{0.746}\,\pm\,0.011$	$0.743\,\pm\,0.004$	$0.716\,\pm\,0.011$	$0.691\,\pm\,0.044$	$0.659\,\pm\,0.002$	$0.704\,\pm\,0.003$	$0.717$	$\mathbf{0.759}$
Heart	$0.776\,\pm\,0.005$	$\mathbf{0.787}\,\pm\,0.015$	$0.674\,\pm\,0.001$	$\underline{0.776}\,\pm\,0.005$	$0.773\,\pm\,0.004$	$0.760\,\pm\,0.005$	$0.760$	$0.747$
Kidney	$\underline{0.720}\,\pm\,0.016$	$0.708\,\pm\,0.001$	$0.709\,\pm\,0.001$	$0.680\,\pm\,0.015$	$0.712\,\pm\,0.003$	$0.719\,\pm\,0.020$	$0.714$	$\mathbf{0.733}$
Liver	$0.861\,\pm\,0.000$	$\underline{0.872}\,\pm\,0.005$	$0.856\,\pm\,0.002$	$0.868\,\pm\,0.002$	$0.860\,\pm\,0.001$	$0.859\,\pm\,0.001$	$0.865$	$\mathbf{0.874}$

Appendix

Table 6: Cohort-level test results with UNI and

G=1000

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.814\,\pm\,0.005$	$\underline{0.817}\,\pm\,0.003$	$0.813\,\pm\,0.001$	$0.806\,\pm\,0.015$	$0.753\,\pm\,0.014$	$0.817\,\pm\,0.001$	$0.792$	$\mathbf{0.823}$
Colon	$\mathbf{0.768}\,\pm\,0.011$	$0.747\,\pm\,0.008$	$0.716\,\pm\,0.003$	$0.739\,\pm\,0.014$	$0.688\,\pm\,0.001$	$0.726\,\pm\,0.016$	$0.727$	$\underline{0.767}$
Heart	$0.783\,\pm\,0.003$	$\mathbf{0.789}\,\pm\,0.008$	$0.699\,\pm\,0.000$	$\underline{0.786}\,\pm\,0.002$	$0.783\,\pm\,0.000$	$0.770\,\pm\,0.005$	$0.772$	$0.764$
Kidney	$\underline{0.758}\,\pm\,0.009$	$0.745\,\pm\,0.002$	$0.747\,\pm\,0.001$	$0.713\,\pm\,0.016$	$0.750\,\pm\,0.000$	$0.755\,\pm\,0.016$	$0.753$	$\mathbf{0.766}$
Liver	$0.873\,\pm\,0.008$	$0.875\,\pm\,0.003$	$0.866\,\pm\,0.000$	$\underline{0.876}\,\pm\,0.003$	$0.872\,\pm\,0.002$	$0.869\,\pm\,0.008$	$0.870$	$\mathbf{0.878}$

Appendix

Table 7: Cohort-level test results with UNI and

G=1500

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.809\,\pm\,0.003$	$\underline{0.810}\,\pm\,0.001$	$0.807\,\pm\,0.001$	$0.802\,\pm\,0.011$	$0.767\,\pm\,0.019$	$0.809\,\pm\,0.003$	$0.788$	$\mathbf{0.818}$
Colon	$\underline{0.749}\,\pm\,0.035$	$0.740\,\pm\,0.010$	$0.705\,\pm\,0.002$	$0.718\,\pm\,0.025$	$0.678\,\pm\,0.007$	$0.715\,\pm\,0.025$	$0.722$	$\mathbf{0.763}$
Heart	$0.773\,\pm\,0.000$	$\mathbf{0.786}\,\pm\,0.008$	$0.709\,\pm\,0.000$	$\underline{0.776}\,\pm\,0.002$	$0.772\,\pm\,0.010$	$0.749\,\pm\,0.006$	$0.771$	$0.766$
Kidney	$\underline{0.753}\,\pm\,0.002$	$0.747\,\pm\,0.002$	$0.742\,\pm\,0.001$	$0.726\,\pm\,0.017$	$0.743\,\pm\,0.007$	$0.750\,\pm\,0.010$	$0.748$	$\mathbf{0.762}$
Liver	$\underline{0.876}\,\pm\,0.002$	$0.872\,\pm\,0.004$	$0.857\,\pm\,0.006$	$\mathbf{0.878}\,\pm\,0.007$	$0.870\,\pm\,0.000$	$0.843\,\pm\,0.012$	$0.868$	$0.875$

Appendix

Table 8: Cohort-level test results with UNI and

G=2000

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.752\,\pm\,0.018$	$\mathbf{0.777}\,\pm\,0.002$	$0.765\,\pm\,0.001$	$0.770\,\pm\,0.000$	$0.706\,\pm\,0.010$	$0.771\,\pm\,0.006$	$0.738$	$\underline{0.773}$
Colon	$0.677\,\pm\,0.009$	$0.699\,\pm\,0.023$	$\underline{0.710}\,\pm\,0.002$	$0.687\,\pm\,0.000$	$0.625\,\pm\,0.021$	$0.694\,\pm\,0.002$	$0.664$	$\mathbf{0.731}$
Heart	$\mathbf{0.765}\,\pm\,0.005$	$0.709\,\pm\,0.009$	$0.646\,\pm\,0.000$	$0.644\,\pm\,0.013$	$\underline{0.748}\,\pm\,0.009$	$0.738\,\pm\,0.039$	$0.711$	$0.684$
Kidney	$0.719\,\pm\,0.020$	$0.684\,\pm\,0.017$	$0.717\,\pm\,0.000$	$0.707\,\pm\,0.010$	$0.686\,\pm\,0.010$	$\mathbf{0.741}\,\pm\,0.000$	$0.720$	$\underline{0.730}$
Liver	$0.863\,\pm\,0.005$	$\mathbf{0.871}\,\pm\,0.000$	$0.862\,\pm\,0.000$	$0.859\,\pm\,0.005$	$0.861\,\pm\,0.001$	$0.857\,\pm\,0.005$	$0.861$	$\underline{0.869}$

Appendix

Table 9: Cohort-level test results with DINOv2-Large and

G=500

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.799\,\pm\,0.002$	$\underline{0.810}\,\pm\,0.001$	$0.801\,\pm\,0.001$	$\mathbf{0.812}\,\pm\,0.000$	$0.747\,\pm\,0.006$	$0.805\,\pm\,0.003$	$0.780$	$0.808$
Colon	$0.731\,\pm\,0.017$	$\mathbf{0.737}\,\pm\,0.012$	$0.703\,\pm\,0.005$	$0.683\,\pm\,0.004$	$0.540\,\pm\,0.066$	$0.697\,\pm\,0.009$	$0.675$	$\underline{0.734}$
Heart	$\mathbf{0.748}\,\pm\,0.001$	$\underline{0.733}\,\pm\,0.002$	$0.674\,\pm\,0.000$	$0.686\,\pm\,0.008$	$0.727\,\pm\,0.012$	$0.722\,\pm\,0.000$	$0.722$	$0.710$
Kidney	$0.705\,\pm\,0.013$	$0.673\,\pm\,0.005$	$0.707\,\pm\,0.001$	$0.679\,\pm\,0.008$	$0.659\,\pm\,0.006$	$\underline{0.710}\,\pm\,0.021$	$0.705$	$\mathbf{0.719}$
Liver	$0.863\,\pm\,0.001$	$\underline{0.866}\,\pm\,0.001$	$0.858\,\pm\,0.001$	$0.865\,\pm\,0.009$	$0.856\,\pm\,0.010$	$0.863\,\pm\,0.002$	$0.862$	$\mathbf{0.867}$

Appendix

Table 10: Cohort-level test results with DINOv2-Large and

G=1000

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.796\,\pm\,0.006$	$\underline{0.819}\,\pm\,0.001$	$0.814\,\pm\,0.001$	$\mathbf{0.821}\,\pm\,0.001$	$0.773\,\pm\,0.005$	$0.813\,\pm\,0.002$	$0.793$	$0.818$
Colon	$0.715\,\pm\,0.022$	$\underline{0.740}\,\pm\,0.014$	$0.713\,\pm\,0.000$	$0.693\,\pm\,0.002$	$0.619\,\pm\,0.019$	$0.701\,\pm\,0.003$	$0.690$	$\mathbf{0.744}$
Heart	$\underline{0.742}\,\pm\,0.003$	$0.739\,\pm\,0.002$	$0.658\,\pm\,0.058$	$0.709\,\pm\,0.002$	$\mathbf{0.750}\,\pm\,0.004$	$0.732\,\pm\,0.009$	$0.739$	$0.734$
Kidney	$0.738\,\pm\,0.009$	$0.730\,\pm\,0.007$	$\underline{0.745}\,\pm\,0.001$	$0.714\,\pm\,0.002$	$0.718\,\pm\,0.017$	$0.744\,\pm\,0.003$	$0.741$	$\mathbf{0.756}$
Liver	$0.863\,\pm\,0.005$	$\underline{0.873}\,\pm\,0.001$	$0.864\,\pm\,0.000$	$\mathbf{0.875}\,\pm\,0.002$	$0.827\,\pm\,0.032$	$0.857\,\pm\,0.011$	$0.867$	$0.872$

Appendix

Table 11: Cohort-level test results with DINOv2-Large and

G=1500

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.797\,\pm\,0.000$	$0.810\,\pm\,0.000$	$0.809\,\pm\,0.000$	$\mathbf{0.815}\,\pm\,0.002$	$0.771\,\pm\,0.005$	$0.809\,\pm\,0.001$	$0.788$	$\underline{0.812}$
Colon	$0.729\,\pm\,0.013$	$\underline{0.731}\,\pm\,0.012$	$0.702\,\pm\,0.003$	$0.688\,\pm\,0.003$	$0.659\,\pm\,0.013$	$0.693\,\pm\,0.005$	$0.686$	$\mathbf{0.739}$
Heart	$\mathbf{0.748}\,\pm\,0.002$	$0.738\,\pm\,0.002$	$0.668\,\pm\,0.058$	$0.719\,\pm\,0.008$	$0.739\,\pm\,0.002$	$0.731\,\pm\,0.000$	$\underline{0.742}$	$0.740$
Kidney	$0.736\,\pm\,0.014$	$0.732\,\pm\,0.002$	$\underline{0.741}\,\pm\,0.000$	$0.706\,\pm\,0.023$	$0.716\,\pm\,0.021$	$0.740\,\pm\,0.002$	$0.737$	$\mathbf{0.752}$
Liver	$0.867\,\pm\,0.004$	$0.866\,\pm\,0.002$	$0.859\,\pm\,0.000$	$\mathbf{0.868}\,\pm\,0.004$	$0.859\,\pm\,0.008$	$0.851\,\pm\,0.018$	$0.865$	$\underline{0.868}$

Appendix

Table 12: Cohort-level test results with DINOv2-Large and

G=2000

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.761\,\pm\,0.004$	$0.772\,\pm\,0.001$	$0.763\,\pm\,0.002$	$\mathbf{0.778}\,\pm\,0.002$	$0.704\,\pm\,0.045$	$0.774\,\pm\,0.002$	$0.755$	$\underline{0.774}$
Colon	$0.706\,\pm\,0.024$	$\underline{0.717}\,\pm\,0.005$	$0.709\,\pm\,0.008$	$0.688\,\pm\,0.002$	$0.511\,\pm\,0.037$	$0.686\,\pm\,0.014$	$0.693$	$\mathbf{0.720}$
Heart	$0.710\,\pm\,0.014$	$\underline{0.719}\,\pm\,0.003$	$0.647\,\pm\,0.000$	$0.654\,\pm\,0.000$	$\mathbf{0.727}\,\pm\,0.005$	$0.644\,\pm\,0.006$	$0.642$	$0.661$
Kidney	$0.671\,\pm\,0.013$	$0.679\,\pm\,0.007$	$0.711\,\pm\,0.006$	$0.708\,\pm\,0.001$	$0.687\,\pm\,0.001$	$0.710\,\pm\,0.030$	$\underline{0.716}$	$\mathbf{0.725}$
Liver	$0.869\,\pm\,0.002$	$\underline{0.875}\,\pm\,0.000$	$0.856\,\pm\,0.006$	$0.860\,\pm\,0.003$	$0.852\,\pm\,0.009$	$0.860\,\pm\,0.001$	$0.853$	$\mathbf{0.877}$

Appendix

Table 13: Cohort-level test results with ResNet18 and

G=500

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.801\,\pm\,0.006$	$0.808\,\pm\,0.002$	$0.796\,\pm\,0.003$	$\mathbf{0.818}\,\pm\,0.003$	$0.752\,\pm\,0.036$	$0.807\,\pm\,0.001$	$0.793$	$\underline{0.808}$
Colon	$\mathbf{0.736}\,\pm\,0.012$	$0.710\,\pm\,0.001$	$0.705\,\pm\,0.005$	$0.685\,\pm\,0.001$	$0.516\,\pm\,0.096$	$0.682\,\pm\,0.020$	$0.694$	$\underline{0.718}$
Heart	$\underline{0.721}\,\pm\,0.021$	$\mathbf{0.724}\,\pm\,0.031$	$0.674\,\pm\,0.001$	$0.675\,\pm\,0.011$	$0.713\,\pm\,0.008$	$0.685\,\pm\,0.007$	$0.679$	$0.695$
Kidney	$0.701\,\pm\,0.004$	$0.690\,\pm\,0.001$	$0.704\,\pm\,0.000$	$0.699\,\pm\,0.001$	$0.689\,\pm\,0.008$	$\mathbf{0.718}\,\pm\,0.014$	$0.706$	$\underline{0.716}$
Liver	$0.865\,\pm\,0.003$	$\underline{0.871}\,\pm\,0.004$	$0.852\,\pm\,0.006$	$0.862\,\pm\,0.006$	$0.841\,\pm\,0.022$	$0.863\,\pm\,0.007$	$0.854$	$\mathbf{0.874}$

Appendix

Table 14: Cohort-level test results with ResNet18 and

G=1000

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.813\,\pm\,0.001$	$\underline{0.818}\,\pm\,0.001$	$0.809\,\pm\,0.002$	$0.817\,\pm\,0.007$	$0.766\,\pm\,0.007$	$0.817\,\pm\,0.000$	$0.805$	$\mathbf{0.819}$
Colon	$0.721\,\pm\,0.001$	$\underline{0.724}\,\pm\,0.004$	$0.711\,\pm\,0.005$	$0.694\,\pm\,0.001$	$0.558\,\pm\,0.008$	$0.706\,\pm\,0.016$	$0.705$	$\mathbf{0.727}$
Heart	$\mathbf{0.748}\,\pm\,0.012$	$0.740\,\pm\,0.021$	$0.699\,\pm\,0.000$	$0.707\,\pm\,0.001$	$\underline{0.741}\,\pm\,0.001$	$0.711\,\pm\,0.001$	$0.707$	$0.722$
Kidney	$0.741\,\pm\,0.019$	$0.738\,\pm\,0.005$	$0.744\,\pm\,0.000$	$0.739\,\pm\,0.001$	$0.723\,\pm\,0.018$	$\underline{0.745}\,\pm\,0.011$	$0.745$	$\mathbf{0.755}$
Liver	$0.869\,\pm\,0.006$	$\mathbf{0.880}\,\pm\,0.001$	$0.846\,\pm\,0.025$	$0.867\,\pm\,0.007$	$0.857\,\pm\,0.013$	$0.855\,\pm\,0.008$	$0.857$	$\underline{0.880}$

Appendix

Table 15: Cohort-level test results with ResNet18 and

G=1500

. Neural entries are mean

\pm

$\mathrm{PCC}_{\mathrm{flat}}\;\uparrow$
Cohort	HiST	HistoPrism	HEXST	STFlow	BLEEP	SEPAL	Linear	CGL-Linear
Brain	$0.810\,\pm\,0.000$	$\underline{0.812}\,\pm\,0.000$	$0.806\,\pm\,0.002$	$0.812\,\pm\,0.000$	$0.763\,\pm\,0.010$	$0.809\,\pm\,0.001$	$0.801$	$\mathbf{0.813}$
Colon	$\mathbf{0.730}\,\pm\,0.004$	$0.719\,\pm\,0.003$	$0.699\,\pm\,0.001$	$0.686\,\pm\,0.000$	$0.579\,\pm\,0.029$	$0.700\,\pm\,0.012$	$0.699$	$\underline{0.721}$
Heart	$\mathbf{0.753}\,\pm\,0.006$	$\underline{0.748}\,\pm\,0.022$	$0.709\,\pm\,0.000$	$0.705\,\pm\,0.017$	$0.741\,\pm\,0.004$	$0.718\,\pm\,0.019$	$0.718$	$0.732$
Kidney	$0.740\,\pm\,0.001$	$0.731\,\pm\,0.001$	$0.739\,\pm\,0.000$	$0.731\,\pm\,0.000$	$0.723\,\pm\,0.005$	$0.739\,\pm\,0.004$	$\underline{0.741}$	$\mathbf{0.751}$
Liver	$0.866\,\pm\,0.003$	$\underline{0.873}\,\pm\,0.002$	$0.833\,\pm\,0.035$	$0.865\,\pm\,0.012$	$0.854\,\pm\,0.011$	$0.850\,\pm\,0.024$	$0.855$	$\mathbf{0.877}$

Appendix

Table 16: Cohort-level test results with ResNet18 and

G=2000

. Neural entries are mean

\pm

Gene Expression Prediction from Histopathology Images

Gene expression prediction from histopathology images links H&E tissue morphology to molecular profiles, enabling virtual spatial transcriptomics without sequencing every sample.

See all 45 papers

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology
Oct 13, 2026Koushik Howlader, Tirtho Roy, Md Tauhidul Islam +1Spatial Context Modeling Spatial Gene Expression Prediction
Iowa State University, Iowa, USA · Stanford University, CA, USA
CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction
Apr 23, 2026Changfan Wang, Xinran Wang, Donghai Liu +4Spatial Transcriptomics Post-Hoc Calibration
Beijing University of Posts and Telecommunications, Beijing 100876, China, and also with the Beijing Key Laboratory of Network System and Network Culture, Beijing 100876, China · Peking University Third Hospital, Beijing 100191, China
HistoLoop: Iterative Contextual Inference with a Looped Transformer for Histology-Based Spatial Gene Expression Prediction
Oct 20, 2026Ziang Liu, Jie Liu, Shuai Li +2Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images
Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology · National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

Oct 13, 2026J/K move · Enter open · S save

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

Koushik Howlader, Tirtho Roy, Md Tauhidul Islam, Wei Le

Iowa State University, Iowa, USA · Stanford University, CA, USA

Spatial Context Modeling Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images Spatial Transcriptomics Multi-Scale Attention Spatial Reasoning

Open paper AskPDF arXiv

Predicting spatial gene expression from standard H&E images could extend molecular analysis to tissues without spatial transcriptomic measurements. A standard Transformer applies self-attention over spots but ignores where each spot sits in the tissue, whereas a spot's molecular state depends on tissue context at multiple spatial scales. Building on this insight, we introduce MIST, a Multi-context Spatial Transformer that augments the self-attention backbone with two context pathways applied in every layer: a coordinate-invariant local pathway that attends over spatial k-nearest-neighbor spots, and a slide pathway that broadcasts a whole-slide summary to every spot; the model predicts all spots in a single forward pass. Because coordinates enter only through pairwise distances, the local pathway is invariant to translations, rotations, and reflections when image features are fixed. In intra-cohort evaluation across six human organ cohorts from HEST-1k, MIST improves mean Pearson correlation over the strongest evaluated baseline by 9.7%, 11.6%, and 11.4% for 10-, 50-, and 100-gene panels, respectively, with the lowest average MSE and MAE. MIST further improves leave-one-organ-out transfer by 6.0% over the next strongest baseline, attains the best expression-derived domain recovery (highest mean ARI and NMI at every clustering resolution, and higher NMI than the closest baseline on 84% of held-out slides), and reproduces spatial expression patterns of selected cancer-marker genes. Together, these results show that adding local and slide context to a vanilla Transformer backbone is an effective approach to spatial gene expression prediction from histology.

Abstract

Gene Expression Prediction from Histopathology Images

Explore similar work

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction

HistoLoop: Iterative Contextual Inference with a Looped Transformer for Histology-Based Spatial Gene Expression Prediction