Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity

Xiaoqing Lian; Pei Liu; Jiahui Tan; Quan Zou; Chen Lin; Xiangxiang Zeng

cs.AIOct 13, 2026

Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity

Authors: Xiaoqing Lian, Pei Liu, Jiahui Tan, Quan Zou, Chen Lin, Xiangxiang Zeng

Organizations: College of Computer Science and Electronic Engineering, Hunan University, Changsha, China · University of Electronic Science and Technology of China, Chengdu, China · Xiamen University, Xiamen, China · State Key Laboratory of Chemo and Biosensing, Institute of AI4Science, Hunan University, Changsha 410082, P. R. China ·

Abstract

Single-cell RNA sequencing (scRNA-seq) enables cell-level analysis of biological heterogeneity, but learning effective representations from high-dimensional, unordered gene expression profiles remains challenging. Gene Expression-to-Image (GE2I) offers an effective learning strategy by arranging gene values on pixel grids for convolutional neural networks, but its underlying mechanisms remain unclear. We study GE2I on full-transcriptome data while keeping the network and training procedure fixed. Our results reveal a trade-off between spatial smoothness and gene-level fidelity. Structured mappings produce smooth, low-frequency patterns that support faster early optimization, whereas assigning multiple genes to the same pixel obscures expression differences and can reduce predictive performance. This trade-off explains why random mappings can compete with structured layouts when many genes are represented. One-to-one gene-to-pixel assignment improves prediction across the tested visual backbones even when image smoothness decreases. We further find that the benefits of pretraining and augmentation depend on the backbone and mapping. These findings provide practical guidance for pixel allocation and the transfer of vision priors to gene expression data.

Figures & tables

Figure 1: Gene expression-to-image (GE2I). A coordinate mapping $\mathcal{M}$ and aggregation operator $\mathcal{A}$ convert gene expression vectors into 2D images for a CNN. Colored gene labels illustrate local neighborhoods in the layout.

	DKD		GTEx v9		HypoMap		Immune Cell Atlas		Tabula Sapiens
Method	ACC	F1-M	ACC	F1-M	ACC	F1-M	ACC	F1-M	ACC	F1-M
XGBoost	$0.8646_{\pm 0.01}$	$0.6860_{\pm 0.01}$	$0.4509_{\pm 0.04}$	$0.1249_{\pm 0.04}$	$0.9661_{\pm 0.00}$	$0.3933_{\pm 0.19}$	$0.6040_{\pm 0.03}$	$0.4350_{\pm 0.02}$	$0.2387_{\pm 0.08}$	$0.0925_{\pm 0.03}$
scANVI	$0.9107_{\pm 0.01}$	$0.8082_{\pm 0.03}$	$0.5608_{\pm 0.05}$	$0.1694_{\pm 0.06}$	$0.9863_{\pm 0.00}$	$0.5061_{\pm 0.23}$	$0.7529_{\pm 0.02}$	$0.6207_{\pm 0.02}$	$0.3856_{\pm 0.13}$	$0.1484_{\pm 0.06}$
scANVI (scArches)	$0.9242_{\pm 0.00}$	$\mathbf{0.8424}_{\pm 0.02}$	$0.5691_{\pm 0.05}$	$0.1539_{\pm 0.03}$	$0.9866_{\pm 0.00}$	$0.5105_{\pm 0.23}$	$0.7606_{\pm 0.02}$	$0.6468_{\pm 0.01}$	$0.3236_{\pm 0.02}$	$0.1594_{\pm 0.07}$
Seurat	$0.8648_{\pm 0.00}$	$0.5908_{\pm 0.03}$	$0.5308_{\pm 0.07}$	$0.1305_{\pm 0.05}$	$0.9843_{\pm 0.00}$	$0.4782_{\pm 0.23}$	$0.5458_{\pm 0.02}$	$0.4208_{\pm 0.02}$	$0.3154_{\pm 0.10}$	$0.1287_{\pm 0.05}$
scGPT	$\mathbf{0.9421}_{\pm 0.01}$	$0.8286_{\pm 0.02}$	$\mathbf{0.7588}_{\pm 0.01}$	$\mathbf{0.2896}_{\pm 0.07}$	-	-	$\mathbf{0.7713}_{\pm 0.02}$	$\mathbf{0.7050}_{\pm 0.01}$	$\mathbf{0.4192}_{\pm 0.16}$	$0.2123_{\pm 0.03}$

Table 1: Cell-type prediction in the HVG setting. GE2I methods use a shared ResNet-18 and training configuration. External methods follow their recommended implementations and settings. Entries report means and standard deviations over three seeds.

Figure 2: Spectral properties of GE2I under HVG and full-transcriptome inputs. LFER and SE characterize the native images produced by each mapping. Several structured mappings concentrate more power at low spatial frequencies than Random Mapping.

	DKD		GTEx v9		HypoMap		Immune Cell Atlas		Tabula Sapiens
Method	Acc	F1-M	Acc	F1-M	Acc	F1-M	Acc	F1-M	Acc	F1-M
scGPT	$0.9462_{\pm 0.00}$	$0.8553_{\pm 0.02}$	$0.8091_{\pm 0.02}$	$0.3415_{\pm 0.09}$	-	-	$0.8061_{\pm 0.03}$	$0.7462_{\pm 0.00}$	$0.3930_{\pm 0.15}$	$0.1977_{\pm 0.06}$
Random	$0.9421_{\pm 0.00}$	$0.8053_{\pm 0.01}$	$0.8489_{\pm 0.03}$	$0.4601_{\pm 0.03}$	$0.9901_{\pm 0.00}$	$0.6553_{\pm 0.23}$	$0.8785_{\pm 0.00}$	$0.7305_{\pm 0.01}$	$0.4569_{\pm 0.10}$	$\mathbf{0.3453}_{\pm 0.11}$
DeepInsight (t-SNE)	$0.9537_{\pm 0.00}$	$0.8868_{\pm 0.02}$	$0.8548_{\pm 0.04}$	$\mathbf{0.5064}_{\pm 0.04}$	$0.9918_{\pm 0.00}$	$0.7487_{\pm 0.17}$	$0.8766_{\pm 0.01}$	$0.7480_{\pm 0.01}$	$0.4270_{\pm 0.13}$	$0.2467_{\pm 0.02}$
DeepInsight (UMAP)	$0.9572_{\pm 0.00}$	$0.8924_{\pm 0.00}$	$0.8462_{\pm 0.04}$	$0.4885_{\pm 0.03}$	$0.9935_{\pm 0.00}$	$0.7713_{\pm 0.15}$	$0.8820_{\pm 0.00}$	$0.7625_{\pm 0.02}$	$0.4275_{\pm 0.21}$	$0.2672_{\pm 0.06}$
DeepInsight (PCA)	$0.9575_{\pm 0.00}$	$0.8458_{\pm 0.00}$	$0.8416_{\pm 0.03}$	$0.4415_{\pm 0.01}$	$0.9926_{\pm 0.00}$	$0.7276_{\pm 0.19}$	$0.8716_{\pm 0.01}$	$0.7483_{\pm 0.03}$	$0.4423_{\pm 0.09}$	$0.3054_{\pm 0.11}$

Table 2: Full-transcriptome prediction. Absolute accuracy and F1-Macro are reported under the shared GE2I learning configuration. scGPT is an external reference; the raw-expression MLP is reported in Appendix Table 9 . T/O denotes a fit exceeding the six-hour construction budget; dashes denote unreported results.

Target	Descriptor	Coef.	Cluster SE	$z^{*}$	$p$	$q$
Accuracy	APPG	-1.044	0.358	-2.92	0.006	0.011
	CR	-3.251	0.617	-5.27	$<0.001$	$<0.001$
	LFER	4.954	1.367	3.62	$<0.001$	0.003
	SE	0.933	0.265	3.52	0.001	0.003
F1-Macro	APPG	-0.966	0.414	-2.33	0.026	0.041
	CR	-3.349	0.791	-4.23	$<0.001$	0.001

Table 3: Beta regression on full-transcriptome performance. Descriptors enter jointly with dataset-by-seed fixed effects, and SE is clustered by dataset–method. The

p

-values use a

t

distribution with 34 degrees of freedom;

q

denotes the Benjamini–Hochberg-adjusted

p

-value across the 16 descriptor tests. Coefficients are on the logit-mean scale.

Table 4: Image-size and pixel-assignment experiments on DKD. (a) Prediction and allocation descriptors across the reported image sizes. (b) LSA results, with superscripts indicating relative changes from the original assignment.

Dataset	Mapping	APPG	CR	LFER	SE	Acc	F1-M
DKD	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+104.8\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8021_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-15.0\%}}$	$2.3435_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+19.7\%}}$	$0.9679_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+1.5\%}}$	$0.9047_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+2.0\%}}$
DKD	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+84.8\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7811_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-13.5\%}}$	$2.4673_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-10.0\%}}$	$0.9667_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+1.0\%}}$	$0.9006_{\pm 0.02}^{{\color[rgb]{0.8867,0.543,0.543}+0.9\%}}$
GTEx v9	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+142.0\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7560_{\pm 0.04}^{{\color[rgb]{0.418,0.6211,0.8594}-12.8\%}}$	$2.7173_{\pm 0.30}^{{\color[rgb]{0.8867,0.543,0.543}+2.8\%}}$	$0.8630_{\pm 0.03}^{{\color[rgb]{0.8867,0.543,0.543}+1.0\%}}$	$0.5557_{\pm 0.02}^{{\color[rgb]{0.8867,0.543,0.543}+9.7\%}}$
GTEx v9	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+271.3\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7374_{\pm 0.05}^{{\color[rgb]{0.418,0.6211,0.8594}-8.0\%}}$	$2.8200_{\pm 0.32}^{{\color[rgb]{0.418,0.6211,0.8594}-12.0\%}}$	$0.8495_{\pm 0.04}^{{\color[rgb]{0.8867,0.543,0.543}+0.4\%}}$	$0.5357_{\pm 0.05}^{{\color[rgb]{0.8867,0.543,0.543}+9.7\%}}$
HypoMap	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+229.2\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8602_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-9.4\%}}$	$2.0356_{\pm 0.07}^{{\color[rgb]{0.418,0.6211,0.8594}-7.7\%}}$	$0.9940_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+0.2\%}}$	$0.7945_{\pm 0.13}^{{\color[rgb]{0.8867,0.543,0.543}+6.1\%}}$
HypoMap	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+359.5\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8085_{\pm 0.05}^{{\color[rgb]{0.418,0.6211,0.8594}-11.7\%}}$	$2.2516_{\pm 0.34}^{{\color[rgb]{0.418,0.6211,0.8594}-21.8\%}}$	$0.9938_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+0.0\%}}$	$0.8137_{\pm 0.13}^{{\color[rgb]{0.8867,0.543,0.543}+5.5\%}}$

Table 5: Greedy unique assignment across datasets. Results use ResNet-18. Superscripts report relative changes from conventional binning for the corresponding dataset and mapping. The algorithm is described in Appendix C.2 .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Samples	Genes	Classes/Targets	Batches	Imbalance
Task 1: Cell Type Classification
Diabetic Kidney Disease	39,176	27,980	13	11	54.7
GTEx v9	209,126	28,094	53	16	956.0
Immune Cell Atlas	329,762	29,335	35	12	964.4
HypoMap	384,925	41,642	13	24	5,350.2
Tabula Sapiens	483,152	55,904	160	29	11,734.7

Appendix

Table 7: Overview of single-cell datasets. Imbalance denotes the ratio between the majority and minority class sizes. The NeurIPS 2021 dataset is used for cross-modality prediction (regression), where the target dimension corresponds to the number of surface proteins.

Panel of Figure 6 — Figure 6: Visual comparison of gene expression-to-image mappings on DKD highly variable genes. Each row represents a specific cell type, while each column corresponds to a mapping strategy.

Method	RMSE $\downarrow$	Overall PCC $\uparrow$	Overall Spearman $\uparrow$
Random Mapping	$0.3688\pm 0.00$	$0.7996\pm 0.00$	$0.7443\pm 0.01$
DeepInsight (t-SNE)	$0.3667\pm 0.00$	$0.8009\pm 0.00$	$0.7451\pm 0.01$
DeepInsight (UMAP)	$0.4397\pm 0.05$	$0.7497\pm 0.02$	$0.7266\pm 0.02$
DeepInsight (PCA)	$0.3741\pm 0.02$	$0.7995\pm 0.01$	$0.7647\pm 0.01$
MR-DeepInsight	$0.3479\pm 0.00$	$0.8237\pm 0.01$	$0.7797\pm 0.00$
Fotomics	$0.4125\pm 0.01$	$0.7404\pm 0.01$	$0.7076\pm 0.00$

Appendix

Table 8: Gene-to-protein prediction on CITE-Seq using full-transcriptome inputs. Results are reported as mean

\pm

standard deviation. Lower RMSE and higher correlations indicate better prediction.

Metric	DKD	GTEx v9	HypoMap	ICA	TS
Acc	$0.9660_{\pm 0.0011}$	$0.9078_{\pm 0.0025}$	$0.9927_{\pm 0.0027}$	$0.9002_{\pm 0.0017}$	$0.4941_{\pm 0.1406}$
F1-M	$0.9312_{\pm 0.0161}$	$0.3624_{\pm 0.0239}$	$0.6903_{\pm 0.2036}$	$0.8285_{\pm 0.0165}$	$0.3044_{\pm 0.0524}$

Appendix

Table 9: Tuned full-genome Simple MLP baseline on raw expression vectors. Results use the same splits as the GE2I classification benchmarks.

Table 14

Method	Augmentation	ACC	F1-Macro
DeepInsight (t-SNE)	None	$0.9537_{\pm 0.00}$	$0.8868_{\pm 0.02}$
	Geometric	$0.9532_{\pm 0.01}$	$0.8819_{\pm 0.02}$
	Bio. Cutout	$0.9513_{\pm 0.01}$	$0.8540_{\pm 0.04}$
	Bio. Noise	$0.9482_{\pm 0.01}$	$0.8771_{\pm 0.01}$
	Bio. Intensity	$0.9566_{\pm 0.00}$	$0.8938_{\pm 0.01}$
	All	$0.4689_{\pm 0.31}$	$0.3227_{\pm 0.22}$

Appendix

Table 12: Image augmentation on DKD. Each augmentation is evaluated relative to the same mapping without augmentation. All tested configurations, including their combination, are reported.

Setting	Random	DeepInsight (t-SNE)	Key evidence
CAM entropy $\downarrow$	0.950	0.907	more compact saliency
Center mass $\uparrow$	0.427	0.521	stronger spatial focus
Marker hit ratio $\uparrow$	0.25	0.50	better biological alignment
DCT marker coverage $\uparrow$	2/5	5/5	SLC12A3, PVALB, TRPM6, CALB1, FXYD2
TAL marker coverage $\uparrow$	2/5	4/5	UMOD, SLC12A1, KCNJ1, CLDN16
Podocyte marker coverage $\uparrow$	2/5	5/5	NPHS1, NPHS2, PODXL, WT1, SYNPO

Appendix

Table 13: Grad-CAM interpretability comparison on DKD. Canonical-marker coverage is measured as the number of established markers falling inside the salient region. Higher center mass and marker hit ratio are better; lower entropy and radius are better.

Figure 8: Representative Grad-CAM examples for three cell types where DeepInsight-tSNE shows stronger biological alignment than the random mapping. In distal convoluted tubule, thick ascending limb, and podocytes, the structured mapping yields tighter saliency regions and substantially higher canonical-marker recovery within the hotspot.

Method	Dataset	APPG	CR	LFER	SE	Acc	F1-M
Highly Variable Genes (HVG)
Random	DKD	1.000	0.000	0.177	3.029	0.900	0.773
Random	GTEx	1.000	0.000	0.198	3.033	0.653	0.211
Random	HypoMAP	1.000	0.000	0.191	3.030	0.980	0.557
Random	ICA	1.000	0.000	0.207	3.034	0.683	0.484
Random	Tabula Sapiens	1.000	0.000	0.221	3.026	0.384	0.215

Appendix

Table 14: Detailed metrics for spectral properties, information density, and performance across all datasets.

Single-Cell Transcriptomics

Single-cell transcriptomics measures gene activity cell by cell to map cell types, states, interactions, and responses across tissues and conditions.

See all 65 papers

Aug 1, 2026

Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction from Histopathology Images

Predicting spatial gene expression from histopathology images enables large-scale transcriptomic profiling without the cost of direct measurement. Existing methods decode the target gene set as a flat, unstructured vector, ignoring the inter-gene dependencies arising from shared biological pathways and regulatory programs. Without explicit structural guidance, models must infer these dependencies entirely from limited paired data, constraining prediction quality. We propose MSGR (Multi-Scale Gene Refiner), which bridges this gap by incorporating the Gene Ontology (GO), a curated functional hierarchy of genes, as an explicit structural prior. MSGR organizes target genes into a four-level GO tree. Its GO-guided decoder then progressively refines predictions from coarse functional domains to fine individual genes via residual corrections under scale-weighted supervision. Operating solely on the gene side, the GO-guided decoder serves as a seamless plug-in replacement that consistently improves existing architectures without requiring any image-side modifications. Extensive experiments on nine datasets from the HEST-1k benchmark provide empirical evidence for two central claims: GO-structured decoding consistently outperforms flat decoding, even against a state-of-the-art generative baseline, and the gain is attributable to biological ontology structure rather than hierarchical decomposition per se, as confirmed by a +0.027 margin over a structurally equivalent random hierarchy.

Zhiwen Xu, Xiaoming Yan, Chengkun Wu +3

National University of Defense Technology Changsha, China

Computational Pathology Spatial Transcriptomics Hierarchical Representation Learning Spatial Gene Expression Prediction Genomics Gene Expression Prediction from Histopathology Images

Ask PDF

Sep 30, 2026

GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics

Spatial transcriptomics enables spatially resolved gene expression analysis from slide-level images while preserving morphological features, providing valuable information for studying disease mechanisms and developing treatments. However, spatial gene expression profiling typically requires expensive and time-consuming tests. While existing image-based prediction optimizations mostly revolve around including positional embeddings and further image-based changes, text-based optimizations remain relatively unexplored. We present GATE-ST, which incorporates text-based inputs into image-based spatial gene expression predictions. With this approach, generated text descriptions of genes are utilized to better spatial transcriptomics prediction results. Gene summaries are put through a text encoder, generating embeddings that integrate with image embeddings through cross-attention layers to align with morphological features. We demonstrate the effectiveness of such text inputs by benchmarking performance against random gene embeddings and multiple other image-text fusion architectures, and show that GATE-ST outperforms these alternatives. Our results demonstrate the effectiveness of GATE-ST in pathology imaging, which may greatly reduce the time and cost of accurate spatial transcriptomic predictions, proving the potential of text-guided spatial gene expression prediction.

Lucas Ni, Jian Luo, Wentao Huang +1

Ridge High School Basking Ridge, United States · Stony Brook University Stony Brook, United States

Spatial Transcriptomics Multimodal Embedding Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images Whole-Slide Image Analysis

Ask PDF

Oct 13, 2026

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

Predicting spatial gene expression from standard H&E images could extend molecular analysis to tissues without spatial transcriptomic measurements. A standard Transformer applies self-attention over spots but ignores where each spot sits in the tissue, whereas a spot's molecular state depends on tissue context at multiple spatial scales. Building on this insight, we introduce MIST, a Multi-context Spatial Transformer that augments the self-attention backbone with two context pathways applied in every layer: a coordinate-invariant local pathway that attends over spatial k-nearest-neighbor spots, and a slide pathway that broadcasts a whole-slide summary to every spot; the model predicts all spots in a single forward pass. Because coordinates enter only through pairwise distances, the local pathway is invariant to translations, rotations, and reflections when image features are fixed. In intra-cohort evaluation across six human organ cohorts from HEST-1k, MIST improves mean Pearson correlation over the strongest evaluated baseline by 9.7%, 11.6%, and 11.4% for 10-, 50-, and 100-gene panels, respectively, with the lowest average MSE and MAE. MIST further improves leave-one-organ-out transfer by 6.0% over the next strongest baseline, attains the best expression-derived domain recovery (highest mean ARI and NMI at every clustering resolution, and higher NMI than the closest baseline on 84% of held-out slides), and reproduces spatial expression patterns of selected cancer-marker genes. Together, these results show that adding local and slide context to a vanilla Transformer backbone is an effective approach to spatial gene expression prediction from histology.

Koushik Howlader, Tirtho Roy, Md Tauhidul Islam +1

Iowa State University, Iowa, USA · Stanford University, CA, USA

Spatial Context Modeling Spatial Gene Expression Prediction Gene Expression Prediction from Histopathology Images Spatial Transcriptomics Multi-Scale Attention Spatial Reasoning

Ask PDF

Abstract

Single-Cell Transcriptomics

Explore similar work

Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction from Histopathology Images

GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology