Estimating slide- and patch-level gene expression profiles from pathology images enables rapid and low-cost molecular analysis with broad clinical impact. Despite strong results, existing approaches treat gene expression as a mere slide- or spot-level signal and do not incorporate the fact that the measured expression arises from the aggregation of underlying cell-level expression. To explicitly introduce this missing cell-resolved guidance, we propose a Cell-type Prototype-informed Neural Network (CPNN) that leverages publicly available single-cell RNA-sequencing datasets. Since single-cell measurements are noisy and not paired with histology images, we first estimate cell-type prototypes-mean expression profiles that reflect stable gene-gene co-variation patterns.CPNN then learns cell-type compositional weights directly from images and models the relationship between prototypes and observed bulk or spatial expression, providing a biologically grounded and structurally regularized prediction framework. We evaluate CPNN on three slide-level datasets and three patch-level spatial transcriptomics datasets. Across all settings, CPNN achieves the highest performance in terms of Spearman correlation. Moreover, by visualizing the inferred compositional weights, our framework provides interpretable insights into which cell types drive the predicted expression. Code is publicly available at https://github.com/naivete5656/CPNN.
Figures & tables
Figure 1: (a) Gene expression is observed together with histology image as the sum of unobserved cell-level expressions. Expression of individual cells is not directly measurable. (b) Cell-level gene expression profiles are obtained from a public single-cell RNA sequencing (scRNA-seq) database, which contains noisy measurements without corresponding image information.
Figure 2: Illustration of concept of Cell-type Prototype-informed Neural Network (CPNN). We estimate gene expression based on cell-type prototypes by modeling the relationship between observed gene expression and cell-level gene expression.
Techniques
Bulk
Single-cell
Spatial
With image
✓
✗
✓
Exp. resolution
Tissue
Single-cell
Spot
Number of samples
High
Moderate
Low
Cost per sample
Low
Medium
High
Table 1: Comparison of bulk, single-cell, and spatial transcriptomics techniques.
Figure 3: Overview of our method. Gene expression is estimated based on cell-type prototype Tˉ and weight w(xi(n)) . The prototype supports the integration of covariance of gene expression among cells.
Method
References
BRCA
KIRC
LUAD
PCC
SCC
PCC
SCC
PCC
SCC
MIL
Max [ 39 ]
Pattern Recog. 2018
0.219
0.225
0.162
0.161
0.170
0.173
Mean [ 39 ]
Pattern Recog. 2018
0.247
0.263
0.211
0.217
0.203
0.213
AbMIL [ 14 ]
ICML 2018
0.280
0.310
0.258
0.260
0.261
0.270
ILRA [ 42 ]
ICLR 2023
0.291
0.310
0.261
0.276
0.245
0.283
S4MIL [ 8 ]
MICCAI2023
0.292
0.301
0.225
0.231
0.243
0.252
Table 2: Comparison performance of slide-level gene expression estimation on TCGA-BRCA, KIRC, and LUAD. MIL: multiple instance learning, MIR: multiple instance regression, PCC: Pearson correlation coefficient, and SCC: spearman correlation coefficient
Method
Ref.
CSCC [ 15 ]
Her2st [ 3 ]
STNet [ 13 ]
PCC
SCC
PCC
SCC
PCC
SCC
BLEEP [ 43 ]
NeurIPS 2023
-0.0050
-0.0024
0.0212
0.0321
-0.0051
-0.0062
EGN [ 45 ]
WACV 2023
0.0552
0.1340
0.0941
0.0479
0.0570
0.0272
STNet [ 13 ]
Nat. Biomed.
0.0856
0.1156
0.0849
0.0772
0.0514
0.0536
w/ Ours
Eng. 2020
0.1008 (+0.0152)
0.1662 (+0.0506)
0.0825 (-0.0024)
0.1111 (+0.0339)
0.0532 (+0.0018)
0.0623 (+0.0087)
TRIPLEX [ 6 ]
CVPR 2024
0.0895
0.1239
0.0900
0.0861
0.0606
0.0546
Table 3: Comparison of methods across CSCC, Her2st, and STNet datasets. Each ( +digit or -digit ) shows the difference from the baseline
Settings
PI
MC
U
R
SCC
w/o PI, MC, R
✗
✗
✓
✗
0.305
w/o MC, U, R
✓
✗
✗
✗
0.174
w/o U, R
✓
✓
✗
✗
0.248
w/o R
✓
✓
✓
✗
0.336
Ours
✓
✓
✓
✓
0.338
Table 4: Ablation study results. PI: prototype initialization of Tˉ , MC: modality correction using α and β , U: prototype update of Tˉ , R: regularization loss.
Coarse (8)
medium (29)
fine (49)
0.317
0.336
0.338
Table 5: Effect of cell-type label granularity on model performance (SCC).
Figure 4: Average weight for each subtype of BRCA slides. We visualized the estimated weights for the cancer-related prototypes.
Method
Setting
SCC
Baseline
10 clusters
0.297 (-0.041)
Baseline
40 clusters
0.300 (-0.038)
Baseline
70 clusters
0.303 (-0.035)
Average prototype
–
0.329 (-0.009)
w/o α and β
–
0.331 (-0.007)
Ours
cell-type informed
0.338
Table 6: Comparison of prototype construction strategies. SCC values are reported, with performance differences from the proposed method shown in parentheses.
Figure 5: Example of estimated proportion.
BRCA
KIRC
LUAD
0.338
0.370
0.253
Table 7: Deconvolution result of cell2location.
Figure 6: The parameter distribution for cell-type prototype on BRCA before and after training. Each scatter plot corresponds to a specific prototype, with individual points representing genes. The horizontal axis denotes parameter values before training, while the vertical axis represents values after training.