cs.AIApr 4, 2026

A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

Authors: Jinxi Xiang, Siyu Hou, Yuchen Li, Ryan Quinton, Xiaoming Zhang, Feyisope Eweje, Xiangde Luo, Yijiang Chen, +12 more

Organizations: Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA · Department of Statistics and Data Science, Yale University, New Haven, CT, USA · Department of Medicine (Oncology), Stanford University School of Medicine, Stanford, CA, USA · Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA · Perelman School of Medicine at the University of Pennsylvania , Philadelphia, PA, USA · Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, USA · Stanford Institute for Human-Centered Artificial Intelligence, Stanford, CA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H&E) staining offers rich morphology yet lacks molecular resolution. We present \ours (Spatial Transcriptomics and histOlogy Representation Model), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine.

Figures & tables

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Explore similar work

CardsList

  1. Path2ST: Hierarchical Cell-Tissue Grounded Cross-Modal Translation for Spatial Transcriptomics

    Aug 11, 2026Ruochen Liu, Wei LouSpatial Gene Expression PredictionGene Expression Prediction from Histopathology Images

    Faculty of Science and Engineering University of Liverpool Liverpool, United Kingdom · College of Mathematical Medicine Zhejiang Normal University Jinhua, China

  2. HiST: A Hierarchical Sparse Transformer for Cross-Modal Spatial Transcriptomics Modeling

    Jun 12, 2026Weiyi Wu, Xinwen Xu, Xingjian Diao +4Computational PathologySpatial Transcriptomics

    Dartmouth College · Mass General Hospital · New Jersey Institute of Technology +1

  3. Concept-based explanation of gene expression prediction from H&E images

    Aug 17, 2026Amos Muench, Jonathan Thielmann, Reduan Achtibat +9Gene Expression Prediction from Histopathology ImagesComputational Pathology

    Institute of Pathology – Charité Universitätsmedizin, Berlin, Germany · Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Center for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France +5