Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity

Xiaoqing Lian; Pei Liu; Jiahui Tan; Quan Zou; Chen Lin; Xiangxiang Zeng

cs.AIOct 13, 2026

Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity

Authors: Xiaoqing Lian, Pei Liu, Jiahui Tan, Quan Zou, Chen Lin, Xiangxiang Zeng

Organizations: College of Computer Science and Electronic Engineering, Hunan University, Changsha, China · University of Electronic Science and Technology of China, Chengdu, China · Xiamen University, Xiamen, China · State Key Laboratory of Chemo and Biosensing, Institute of AI4Science, Hunan University, Changsha 410082, P. R. China ·

Abstract

Single-cell RNA sequencing (scRNA-seq) enables cell-level analysis of biological heterogeneity, but learning effective representations from high-dimensional, unordered gene expression profiles remains challenging. Gene Expression-to-Image (GE2I) offers an effective learning strategy by arranging gene values on pixel grids for convolutional neural networks, but its underlying mechanisms remain unclear. We study GE2I on full-transcriptome data while keeping the network and training procedure fixed. Our results reveal a trade-off between spatial smoothness and gene-level fidelity. Structured mappings produce smooth, low-frequency patterns that support faster early optimization, whereas assigning multiple genes to the same pixel obscures expression differences and can reduce predictive performance. This trade-off explains why random mappings can compete with structured layouts when many genes are represented. One-to-one gene-to-pixel assignment improves prediction across the tested visual backbones even when image smoothness decreases. We further find that the benefits of pretraining and augmentation depend on the backbone and mapping. These findings provide practical guidance for pixel allocation and the transfer of vision priors to gene expression data.

Figures & tables

Figure 1: Gene expression-to-image (GE2I). A coordinate mapping $\mathcal{M}$ and aggregation operator $\mathcal{A}$ convert gene expression vectors into 2D images for a CNN. Colored gene labels illustrate local neighborhoods in the layout.

	DKD		GTEx v9		HypoMap		Immune Cell Atlas		Tabula Sapiens
Method	ACC	F1-M	ACC	F1-M	ACC	F1-M	ACC	F1-M	ACC	F1-M
XGBoost	$0.8646_{\pm 0.01}$	$0.6860_{\pm 0.01}$	$0.4509_{\pm 0.04}$	$0.1249_{\pm 0.04}$	$0.9661_{\pm 0.00}$	$0.3933_{\pm 0.19}$	$0.6040_{\pm 0.03}$	$0.4350_{\pm 0.02}$	$0.2387_{\pm 0.08}$	$0.0925_{\pm 0.03}$
scANVI	$0.9107_{\pm 0.01}$	$0.8082_{\pm 0.03}$	$0.5608_{\pm 0.05}$	$0.1694_{\pm 0.06}$	$0.9863_{\pm 0.00}$	$0.5061_{\pm 0.23}$	$0.7529_{\pm 0.02}$	$0.6207_{\pm 0.02}$	$0.3856_{\pm 0.13}$	$0.1484_{\pm 0.06}$
scANVI (scArches)	$0.9242_{\pm 0.00}$	$\mathbf{0.8424}_{\pm 0.02}$	$0.5691_{\pm 0.05}$	$0.1539_{\pm 0.03}$	$0.9866_{\pm 0.00}$	$0.5105_{\pm 0.23}$	$0.7606_{\pm 0.02}$	$0.6468_{\pm 0.01}$	$0.3236_{\pm 0.02}$	$0.1594_{\pm 0.07}$
Seurat	$0.8648_{\pm 0.00}$	$0.5908_{\pm 0.03}$	$0.5308_{\pm 0.07}$	$0.1305_{\pm 0.05}$	$0.9843_{\pm 0.00}$	$0.4782_{\pm 0.23}$	$0.5458_{\pm 0.02}$	$0.4208_{\pm 0.02}$	$0.3154_{\pm 0.10}$	$0.1287_{\pm 0.05}$
scGPT	$\mathbf{0.9421}_{\pm 0.01}$	$0.8286_{\pm 0.02}$	$\mathbf{0.7588}_{\pm 0.01}$	$\mathbf{0.2896}_{\pm 0.07}$	-	-	$\mathbf{0.7713}_{\pm 0.02}$	$\mathbf{0.7050}_{\pm 0.01}$	$\mathbf{0.4192}_{\pm 0.16}$	$0.2123_{\pm 0.03}$

Table 1: Cell-type prediction in the HVG setting. GE2I methods use a shared ResNet-18 and training configuration. External methods follow their recommended implementations and settings. Entries report means and standard deviations over three seeds.

Figure 2: Spectral properties of GE2I under HVG and full-transcriptome inputs. LFER and SE characterize the native images produced by each mapping. Several structured mappings concentrate more power at low spatial frequencies than Random Mapping.

	DKD		GTEx v9		HypoMap		Immune Cell Atlas		Tabula Sapiens
Method	Acc	F1-M	Acc	F1-M	Acc	F1-M	Acc	F1-M	Acc	F1-M
scGPT	$0.9462_{\pm 0.00}$	$0.8553_{\pm 0.02}$	$0.8091_{\pm 0.02}$	$0.3415_{\pm 0.09}$	-	-	$0.8061_{\pm 0.03}$	$0.7462_{\pm 0.00}$	$0.3930_{\pm 0.15}$	$0.1977_{\pm 0.06}$
Random	$0.9421_{\pm 0.00}$	$0.8053_{\pm 0.01}$	$0.8489_{\pm 0.03}$	$0.4601_{\pm 0.03}$	$0.9901_{\pm 0.00}$	$0.6553_{\pm 0.23}$	$0.8785_{\pm 0.00}$	$0.7305_{\pm 0.01}$	$0.4569_{\pm 0.10}$	$\mathbf{0.3453}_{\pm 0.11}$
DeepInsight (t-SNE)	$0.9537_{\pm 0.00}$	$0.8868_{\pm 0.02}$	$0.8548_{\pm 0.04}$	$\mathbf{0.5064}_{\pm 0.04}$	$0.9918_{\pm 0.00}$	$0.7487_{\pm 0.17}$	$0.8766_{\pm 0.01}$	$0.7480_{\pm 0.01}$	$0.4270_{\pm 0.13}$	$0.2467_{\pm 0.02}$
DeepInsight (UMAP)	$0.9572_{\pm 0.00}$	$0.8924_{\pm 0.00}$	$0.8462_{\pm 0.04}$	$0.4885_{\pm 0.03}$	$0.9935_{\pm 0.00}$	$0.7713_{\pm 0.15}$	$0.8820_{\pm 0.00}$	$0.7625_{\pm 0.02}$	$0.4275_{\pm 0.21}$	$0.2672_{\pm 0.06}$
DeepInsight (PCA)	$0.9575_{\pm 0.00}$	$0.8458_{\pm 0.00}$	$0.8416_{\pm 0.03}$	$0.4415_{\pm 0.01}$	$0.9926_{\pm 0.00}$	$0.7276_{\pm 0.19}$	$0.8716_{\pm 0.01}$	$0.7483_{\pm 0.03}$	$0.4423_{\pm 0.09}$	$0.3054_{\pm 0.11}$

Table 2: Full-transcriptome prediction. Absolute accuracy and F1-Macro are reported under the shared GE2I learning configuration. scGPT is an external reference; the raw-expression MLP is reported in Appendix Table 9 . T/O denotes a fit exceeding the six-hour construction budget; dashes denote unreported results.

Target	Descriptor	Coef.	Cluster SE	$z^{*}$	$p$	$q$
Accuracy	APPG	-1.044	0.358	-2.92	0.006	0.011
	CR	-3.251	0.617	-5.27	$<0.001$	$<0.001$
	LFER	4.954	1.367	3.62	$<0.001$	0.003
	SE	0.933	0.265	3.52	0.001	0.003
F1-Macro	APPG	-0.966	0.414	-2.33	0.026	0.041
	CR	-3.349	0.791	-4.23	$<0.001$	0.001

Table 3: Beta regression on full-transcriptome performance. Descriptors enter jointly with dataset-by-seed fixed effects, and SE is clustered by dataset–method. The

p

-values use a

t

distribution with 34 degrees of freedom;

q

denotes the Benjamini–Hochberg-adjusted

p

-value across the 16 descriptor tests. Coefficients are on the logit-mean scale.

Table 4: Image-size and pixel-assignment experiments on DKD. (a) Prediction and allocation descriptors across the reported image sizes. (b) LSA results, with superscripts indicating relative changes from the original assignment.

Dataset	Mapping	APPG	CR	LFER	SE	Acc	F1-M
DKD	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+104.8\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8021_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-15.0\%}}$	$2.3435_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+19.7\%}}$	$0.9679_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+1.5\%}}$	$0.9047_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+2.0\%}}$
DKD	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+84.8\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7811_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-13.5\%}}$	$2.4673_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-10.0\%}}$	$0.9667_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+1.0\%}}$	$0.9006_{\pm 0.02}^{{\color[rgb]{0.8867,0.543,0.543}+0.9\%}}$
GTEx v9	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+142.0\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7560_{\pm 0.04}^{{\color[rgb]{0.418,0.6211,0.8594}-12.8\%}}$	$2.7173_{\pm 0.30}^{{\color[rgb]{0.8867,0.543,0.543}+2.8\%}}$	$0.8630_{\pm 0.03}^{{\color[rgb]{0.8867,0.543,0.543}+1.0\%}}$	$0.5557_{\pm 0.02}^{{\color[rgb]{0.8867,0.543,0.543}+9.7\%}}$
GTEx v9	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+271.3\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.7374_{\pm 0.05}^{{\color[rgb]{0.418,0.6211,0.8594}-8.0\%}}$	$2.8200_{\pm 0.32}^{{\color[rgb]{0.418,0.6211,0.8594}-12.0\%}}$	$0.8495_{\pm 0.04}^{{\color[rgb]{0.8867,0.543,0.543}+0.4\%}}$	$0.5357_{\pm 0.05}^{{\color[rgb]{0.8867,0.543,0.543}+9.7\%}}$
HypoMap	DeepInsight (t-SNE)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+229.2\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8602_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-9.4\%}}$	$2.0356_{\pm 0.07}^{{\color[rgb]{0.418,0.6211,0.8594}-7.7\%}}$	$0.9940_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+0.2\%}}$	$0.7945_{\pm 0.13}^{{\color[rgb]{0.8867,0.543,0.543}+6.1\%}}$
HypoMap	DeepInsight (UMAP)	$1.0000_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+359.5\%}}$	$0.0000_{\pm 0.00}^{{\color[rgb]{0.418,0.6211,0.8594}-100.0\%}}$	$0.8085_{\pm 0.05}^{{\color[rgb]{0.418,0.6211,0.8594}-11.7\%}}$	$2.2516_{\pm 0.34}^{{\color[rgb]{0.418,0.6211,0.8594}-21.8\%}}$	$0.9938_{\pm 0.00}^{{\color[rgb]{0.8867,0.543,0.543}+0.0\%}}$	$0.8137_{\pm 0.13}^{{\color[rgb]{0.8867,0.543,0.543}+5.5\%}}$

Table 5: Greedy unique assignment across datasets. Results use ResNet-18. Superscripts report relative changes from conventional binning for the corresponding dataset and mapping. The algorithm is described in Appendix C.2 .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Samples	Genes	Classes/Targets	Batches	Imbalance
Task 1: Cell Type Classification
Diabetic Kidney Disease	39,176	27,980	13	11	54.7
GTEx v9	209,126	28,094	53	16	956.0
Immune Cell Atlas	329,762	29,335	35	12	964.4
HypoMap	384,925	41,642	13	24	5,350.2
Tabula Sapiens	483,152	55,904	160	29	11,734.7

Appendix

Table 7: Overview of single-cell datasets. Imbalance denotes the ratio between the majority and minority class sizes. The NeurIPS 2021 dataset is used for cross-modality prediction (regression), where the target dimension corresponds to the number of surface proteins.

Panel of Figure 6 — Figure 6: Visual comparison of gene expression-to-image mappings on DKD highly variable genes. Each row represents a specific cell type, while each column corresponds to a mapping strategy.

Method	RMSE $\downarrow$	Overall PCC $\uparrow$	Overall Spearman $\uparrow$
Random Mapping	$0.3688\pm 0.00$	$0.7996\pm 0.00$	$0.7443\pm 0.01$
DeepInsight (t-SNE)	$0.3667\pm 0.00$	$0.8009\pm 0.00$	$0.7451\pm 0.01$
DeepInsight (UMAP)	$0.4397\pm 0.05$	$0.7497\pm 0.02$	$0.7266\pm 0.02$
DeepInsight (PCA)	$0.3741\pm 0.02$	$0.7995\pm 0.01$	$0.7647\pm 0.01$
MR-DeepInsight	$0.3479\pm 0.00$	$0.8237\pm 0.01$	$0.7797\pm 0.00$
Fotomics	$0.4125\pm 0.01$	$0.7404\pm 0.01$	$0.7076\pm 0.00$

Appendix

Table 8: Gene-to-protein prediction on CITE-Seq using full-transcriptome inputs. Results are reported as mean

\pm

standard deviation. Lower RMSE and higher correlations indicate better prediction.

Metric	DKD	GTEx v9	HypoMap	ICA	TS
Acc	$0.9660_{\pm 0.0011}$	$0.9078_{\pm 0.0025}$	$0.9927_{\pm 0.0027}$	$0.9002_{\pm 0.0017}$	$0.4941_{\pm 0.1406}$
F1-M	$0.9312_{\pm 0.0161}$	$0.3624_{\pm 0.0239}$	$0.6903_{\pm 0.2036}$	$0.8285_{\pm 0.0165}$	$0.3044_{\pm 0.0524}$

Appendix

Table 9: Tuned full-genome Simple MLP baseline on raw expression vectors. Results use the same splits as the GE2I classification benchmarks.

Table 14

Method	Augmentation	ACC	F1-Macro
DeepInsight (t-SNE)	None	$0.9537_{\pm 0.00}$	$0.8868_{\pm 0.02}$
	Geometric	$0.9532_{\pm 0.01}$	$0.8819_{\pm 0.02}$
	Bio. Cutout	$0.9513_{\pm 0.01}$	$0.8540_{\pm 0.04}$
	Bio. Noise	$0.9482_{\pm 0.01}$	$0.8771_{\pm 0.01}$
	Bio. Intensity	$0.9566_{\pm 0.00}$	$0.8938_{\pm 0.01}$
	All	$0.4689_{\pm 0.31}$	$0.3227_{\pm 0.22}$

Appendix

Table 12: Image augmentation on DKD. Each augmentation is evaluated relative to the same mapping without augmentation. All tested configurations, including their combination, are reported.

Setting	Random	DeepInsight (t-SNE)	Key evidence
CAM entropy $\downarrow$	0.950	0.907	more compact saliency
Center mass $\uparrow$	0.427	0.521	stronger spatial focus
Marker hit ratio $\uparrow$	0.25	0.50	better biological alignment
DCT marker coverage $\uparrow$	2/5	5/5	SLC12A3, PVALB, TRPM6, CALB1, FXYD2
TAL marker coverage $\uparrow$	2/5	4/5	UMOD, SLC12A1, KCNJ1, CLDN16
Podocyte marker coverage $\uparrow$	2/5	5/5	NPHS1, NPHS2, PODXL, WT1, SYNPO

Appendix

Table 13: Grad-CAM interpretability comparison on DKD. Canonical-marker coverage is measured as the number of established markers falling inside the salient region. Higher center mass and marker hit ratio are better; lower entropy and radius are better.

Figure 8: Representative Grad-CAM examples for three cell types where DeepInsight-tSNE shows stronger biological alignment than the random mapping. In distal convoluted tubule, thick ascending limb, and podocytes, the structured mapping yields tighter saliency regions and substantially higher canonical-marker recovery within the hotspot.

Method	Dataset	APPG	CR	LFER	SE	Acc	F1-M
Highly Variable Genes (HVG)
Random	DKD	1.000	0.000	0.177	3.029	0.900	0.773
Random	GTEx	1.000	0.000	0.198	3.033	0.653	0.211
Random	HypoMAP	1.000	0.000	0.191	3.030	0.980	0.557
Random	ICA	1.000	0.000	0.207	3.034	0.683	0.484
Random	Tabula Sapiens	1.000	0.000	0.221	3.026	0.384	0.215

Appendix

Table 14: Detailed metrics for spectral properties, information density, and performance across all datasets.

Single-Cell Transcriptomics

Single-cell transcriptomics measures gene activity cell by cell to map cell types, states, interactions, and responses across tissues and conditions.

See all 65 papers

Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction from Histopathology Images
Aug 1, 2026Zhiwen Xu, Xiaoming Yan, Chengkun Wu +3Computational Pathology Spatial Transcriptomics
National University of Defense Technology Changsha, China
GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics
Sep 30, 2026Lucas Ni, Jian Luo, Wentao Huang +1Spatial Transcriptomics Multimodal Embedding
Ridge High School Basking Ridge, United States · Stony Brook University Stony Brook, United States
MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology
Oct 13, 2026Koushik Howlader, Tirtho Roy, Md Tauhidul Islam +1Spatial Context Modeling Spatial Gene Expression Prediction
Iowa State University, Iowa, USA · Stanford University, CA, USA

Aug 1, 2026J/K move · Enter open · S save

Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction from Histopathology Images

Zhiwen Xu, Xiaoming Yan, Chengkun Wu, Juan Chen+2

National University of Defense Technology Changsha, China

Computational Pathology Spatial Transcriptomics Hierarchical Representation Learning Spatial Gene Expression Prediction Genomics Gene Expression Prediction from Histopathology Images

Open paper AskPDF arXiv

Predicting spatial gene expression from histopathology images enables large-scale transcriptomic profiling without the cost of direct measurement. Existing methods decode the target gene set as a flat, unstructured vector, ignoring the inter-gene dependencies arising from shared biological pathways and regulatory programs. Without explicit structural guidance, models must infer these dependencies entirely from limited paired data, constraining prediction quality. We propose MSGR (Multi-Scale Gene Refiner), which bridges this gap by incorporating the Gene Ontology (GO), a curated functional hierarchy of genes, as an explicit structural prior. MSGR organizes target genes into a four-level GO tree. Its GO-guided decoder then progressively refines predictions from coarse functional domains to fine individual genes via residual corrections under scale-weighted supervision. Operating solely on the gene side, the GO-guided decoder serves as a seamless plug-in replacement that consistently improves existing architectures without requiring any image-side modifications. Extensive experiments on nine datasets from the HEST-1k benchmark provide empirical evidence for two central claims: GO-structured decoding consistently outperforms flat decoding, even against a state-of-the-art generative baseline, and the gain is attributable to biological ontology structure rather than hierarchical decomposition per se, as confirmed by a +0.027 margin over a structurally equivalent random hierarchy.

Abstract

Single-Cell Transcriptomics

Explore similar work

Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction from Histopathology Images

GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology