spora: A Unified Multimodal Dataset and Platform for Spatial Proteomics

Benedikt von Querfurth; Johann Wenckstern; Eeshaan Jain; Lukas Klein; Phil Cheng; Yexiang Cheng; Cedric Vincent-Cuaz; Martina Haberecker

cs.CVOct 15, 2026

spora: A Unified Multimodal Dataset and Platform for Spatial Proteomics

Authors: Benedikt von Querfurth, Johann Wenckstern, Eeshaan Jain, Lukas Klein, Phil Cheng, Yexiang Cheng, Cedric Vincent-Cuaz, Martina Haberecker,

Organizations: EPFL · University Hospital Zurich · University of Zurich · University Hospital Geneva ·

Abstract

Spatial proteomics (SP) captures the molecular composition and spatial organization of tissues at single-cell resolution, opening a direct view of the cellular ecosystems that drive disease. Protein panels, acquisition platforms, and tissue contexts differ from one cohort to the next, and a coherent picture of tissue biology can only emerge from models trained to bridge this heterogeneity. Foundation models offer the natural path forward, but their training depends on large, harmonized corpora that span platforms, panels, and cohorts. No public resource currently approaches that scale; available data sit in small, narrowly scoped releases with incompatible formats and conventions. We present $\texttt{spora}$ , a large-scale multimodal SP dataset of 13,546 standardized multiplex images from 5,477 patients across 35 cohorts and five major SP technologies. Curated with biomedical experts, $\texttt{spora}$ provides model training-ready formats together with cell and tissue segmentations, structured clinical metadata, and, where available, paired H&E enabling the training of virtual staining models. It is complemented by $\texttt{spora [io]}$ , a unified interface for tiling, sampling, and standardization across all cohorts and modalities, and $\texttt{spora [bench]}$ , a benchmark suite spanning cell phenotyping, diagnostic and therapeutic prediction as well as virtual staining tasks across twelve cohorts that places current SP foundation models on common ground against task specific baselines.

Figures & tables

Figure 1: Overview of spora . a. Unified dataset composition and per-sample file structure of spora [data] . b. Library pipeline of spora [io] from dataset browsing and tiling to standardized model input. c. spora [bench] benchmark tasks for foundation model evaluation.

Figure 2: Exemplary tissues and annotations. Multiplex images of a. an IMC tumor core ( Rigamonti et al., 2024 ) and b. a CODEX whole-slide image ( Ruf et al., 2023 ) , with cell instance segmentations from VirTues and tissue-aware tile distributions from spora [io] . The tiling adapts to tissue geometry, achieving tissue-to-background ratios well above standard grid tiling.

Model	KRONOS	VirTues
Dataset
Cords et al. (2023)	0.601 [.597, .605]	0.782 [.778, .785]
Cords et al. (2024)	-	0.757 [.755, .758]
Danenberg et al. (2022)	0.355 [.352, .358]	0.611 [.607, .615]
Hoch et al. (2022)	0.574 [.566, .582]	0.836 [.830, .841]
Lin et al. (2023a)	0.400 [.397, .402]	0.642 [.638, .645]

Table 1: Linear-probe cell phenotyping results. Macro average F1-score per dataset and model. Best performing model per dataset is shown in bold. The 95% confidence intervals calculated via bootstrapping (

n=1000

repeats) are shown in brackets.

Figure 3: Tissue-level diagnostic and therapeutic results. Performance across diagnostic and therapeutic tasks on three cohorts spanning non-small cell lung cancer (NSCLC), triple-negative breast cancer (TNBC), and a heterogeneous breast cancer cohort. Each task contains two classes.

Figure 4: Multiplex virtual staining results. a. Average Pearson correlation between predicted and ground-truth markers across tissues; the baseline (light-gray cross) substitutes the closest available marker for the target. b. Representative multiplex images and their virtual staining predictions obtained from VirTues.

		GigaTIME		HistoPlexer		ROSIE
Dataset	# markers	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$
Hickey et al. (2023)	10/5/23	0.056	$0.384$	$0.004$	0.603	$0.043$	$0.082$
Lin et al. (2023a)	10/3/15	$0.046$	$0.266$	$0.004$	$0.056$	0.066	0.301
Phillips et al. (2021)	15/6/29	0.106	$0.294$	$0.003$	0.576	$0.060$	$0.076$
Schürch et al. (2020)	14/7/33	0.072	$0.245$	$0.004$	0.408	$0.038$	$0.098$

Table 2: H&E virtual staining results. Pearson correlation and SSIM averaged over the markers predicted by all three models in a given cohort. Predicted marker set differs between models, indicated by # markers in the order GigaTIME/HistoPlexer/ROSIE. Best performing model per dataset and metric is shown in bold.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Circle plot indicating, for each protein (HUGO gene nomenclature) and non-protein nuclear marker in the dataset, the cohorts in which it is represented.

Figure A2: Overview of the dataset. a. Distribution of multiplexed tiles across specimen types, split between tumor cores (TC) and whole-slide images. b. Number of mx tiles per organ (log scale), spanning 13 tissue types. c. Number of datasets containing diagnostic and therapeutic annotations that can be used ofr downstream tasks. d. Per-cohort breakdown of mx tiles (blue, left axis) and number of patients (green, right axis), both on a log scale, sorted by decreasing tile count.

Figure A3: Example mx images for four IMC datasets with instance cell masks.

Figure A4: Example mx images for four CODEX datasets with VirTues predicted instance cell masks.

Figure A5: Example mx images for two MIBI datasets with VirTues predicted instance cell masks.

Figure A6: Example mx image for the Orion dataset with VirTues predicted instance cell masks.

Figure A7: Comparison of cell instance segmentations obtained from VirTues and provided with the original study by Cords et al. (2024) .

Figure A8: Comparison of cell instance segmentations obtained from VirTues and provided with the original study by Liu et al. (2022) .

Figure A9: Tile distributions under classical grid tiling ( $\geq$ 10% tissue inclusion) compared with our coverage-optimized tiling ( $\Delta=c/2$ , $\tau=0.95$ , $\gamma=0.95$ , $\alpha=0.1$ ). Tissue-to-background ratios show improved coverage under our method, particularly near image borders where grid tiling discards partial tiles or includes large out-of-image portions.

Figure A10: a. Tile visualizer tool to compare different tiling hyperparameter visually. b. Dataset explorer command line tool that is part of spora [io] to explore and combine different cohorts based on meta data.

Model	VirTues	KRONOS	Eva
Dataset
Cords et al. (2023)	✓	✗	?
Cords et al. (2024)	✓	✗	?
Danenberg et al. (2022)	✓	✗	?
Eling et al. (2025)	✓	✗	?
Hickey et al. (2023)	✓	✗	?

Appendix

Table A3: Disclosure of pretraining exposure. We indicate pretraining occurrence for each foundation model and benchmark cohort of spora [bench] . We use the model instance virtues-sp32 of VirTues. Pretraining data of Eva is not clearly disclosed.

Target benchmark cohort	Selected source cohort
Cords et al. (2023)	Meyer et al. (2025)
Cords et al. (2024)	Rigamonti et al. (2024)
Danenberg et al. (2022)	Meyer et al. (2025)
Lin et al. (2023a)	Rigamonti et al. (2024)
Meyer et al. (2025)	Cords et al. (2023)
Moldoveanu et al. (2022)	Danenberg et al. (2022)

Appendix

Table A4: Source cohort selected for training the MAPS label-transfer model for each target benchmark cohort. Source cohorts were chosen to match the tissue origin of the target where possible and to maximize the number of shared markers.

Model	VirTues	astir	MAPS
Dataset
Cords et al. (2023)	0.717 [.713, .721]	0.532 [.530, .535]	0.614 [.611, .617]
Cords et al. (2024)	0.652 [.650, .653]	0.592 [.591, .594]	0.502 [.500, .503]
Danenberg et al. (2022)	0.553 [.551, .556]	0.407 [.404, .410]	0.390 [.387, .392]
Lin et al. (2023a)	0.478 [.475, .480]	0.520 [.518, .521]	0.637 [.634, .640]
Meyer et al. (2025)	0.605 [.602, .608]	0.596 [.594, .599]	0.693 [.691, .696]

Appendix

Table A5: Cross-cohort cell annotation results. Macro average F1-score per dataset and model. Best performing model per dataset is highlighted in bold, the second-best is underlined. The 95% confidence intervals calculated via bootstrapping (

n=1000

repeats) are shown in brackets.

Model	KRONOS	VirTues
Class
CAF	0.627 [.622, .632]	0.791 [.788, .794]
Endothelial	0.420 [.408, .433]	0.585 [.573, .596]
Immune	0.591 [.587, .596]	0.819 [.816, .822]
Tumour	0.765 [.762, .768]	0.932 [.930, .933]

Appendix

Table A6: Detailed cell phenotyping results for cords2023cancer ( Cords et al., 2023 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
Fibroblast	-	0.780 [.778, .781]
Immune cell	-	0.739 [.738, .741]
T cell	-	0.772 [.770, .775]
Tumor cell	-	0.943 [.942, .943]
Vascular cell	-	0.549 [.543, .555]

Appendix

Table A7: Detailed cell phenotyping results for cords2024cancer ( Cords et al., 2024 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown. Results for KRONOS are omitted because the high dimensionality of its cell embeddings, combined with the large number of cells, made logistic regression computationally infeasible.

Model	KRONOS	VirTues
Class
Antigen-presenting cell	0.099 [.081, .117]	0.251 [.230, .272]
B/plasma cell	0.427 [.417, .437]	0.798 [.792, .804]
ER- epithelial cell	0.512 [.510, .515]	0.674 [.671, .677]
ER+ epithelial cell	0.342 [.337, .347]	0.582 [.578, .586]
Myeloid cell	0.348 [.341, .354]	0.596 [.590, .602]

Appendix

Table A8: Detailed cell phenotyping results for danenberg2022breast ( Danenberg et al., 2022 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
Lymphocyte	0.500 [.467, .530]	0.838 [.820, .856]
Macrophage	0.643 [.630, .656]	0.829 [.819, .837]
Stromal cell	0.325 [.309, .341]	0.626 [.613, .639]
T cell	0.618 [.609, .627]	0.949 [.945, .952]
Tumor cell	0.785 [.781, .788]	0.937 [.935, .939]

Appendix

Table A9: Detailed cell phenotyping results for hoch2022multiplexed ( Hoch et al., 2022 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.557 [.545, .568]	0.762 [.753, .770]
CD8 T cell	0.495 [.485, .504]	0.672 [.664, .679]
Endothelial cell	0.396 [.387, .405]	0.815 [.809, .820]
Exhausted CD8 T cell	0.067 [.058, .076]	0.309 [.275, .343]
M2-like macrophage	0.483 [.476, .490]	0.704 [.698, .710]

Appendix

Table A10: Detailed cell phenotyping results for lin2023highsubset ( Lin et al., 2023a ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.541 [.530, .553]	0.851 [.845, .858]
CD4 T cell	0.557 [.548, .566]	0.834 [.828, .839]
CD8 T cell	0.596 [.587, .605]	0.831 [.825, .836]
Endothelial cell	0.435 [.427, .444]	0.758 [.752, .763]
HLA-DR+ cell	0.107 [.090, .125]	0.312 [.289, .333]

Appendix

Table A11: Detailed cell phenotyping results for meyer2025stratification ( Meyer et al., 2025 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.218 [.154, .282]	0.626 [.556, .691]
Endothelial cell	0.750 [.727, .772]	0.940 [.931, .950]
Activated CD8 T cell	0.625 [.596, .654]	0.798 [.778, .818]
Naive CD8 T cell	0.347 [.296, .394]	0.525 [.474, .576]
Activated CD4 T cell	0.519 [.475, .565]	0.715 [.677, .750]

Appendix

Table A12: Detailed cell phenotyping results for moldoveanu2022spatially ( Moldoveanu et al., 2022 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.330 [.284, .382]	0.720 [.685, .754]
Epithelial cell	0.602 [.590, .614]	0.945 [.941, .949]
Langerhans cell	0.210 [.152, .266]	0.328 [.262, .395]
Mast cell	0.553 [.456, .639]	0.754 [.692, .813]
Myeloid cell	0.464 [.448, .480]	0.757 [.747, .768]

Appendix

Table A13: Detailed cell phenotyping results for phillips2021immune ( Phillips et al., 2021 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.666 [.648, .683]	0.795 [.782, .807]
CD4 T cell	0.572 [.556, .588]	0.712 [.700, .725]
CD8 T cell	0.673 [.662, .684]	0.807 [.798, .816]
Ki67+ cell	0.114 [.057, .180]	0.224 [.150, .295]
Macrophage	0.613 [.602, .622]	0.728 [.719, .736]

Appendix

Table A14: Detailed cell phenotyping results for rigamonti2024integrating ( Rigamonti et al., 2024 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	KRONOS	VirTues
Class
B cell	0.546 [.527, .563]	0.835 [.824, .846]
CD4 T cell	0.492 [.470, .514]	0.766 [.751, .781]
CD8 T cell	0.570 [.554, .585]	0.784 [.775, .795]
Ki67+ cell	0.630 [.619, .641]	0.873 [.866, .879]
Dendritic cell	0.346 [.329, .366]	0.519 [.501, .537]

Appendix

Table A15: Detailed cell phenotyping results for schulz2024immucan ( Eling et al., 2025 ) . F1-scores and bootstrapped 95% confidence intervals per cell phenotype are shown.

Model	VirTues	Cellpose	InstanSeg	StarDist
Dataset
Cords et al. (2023)	0.801	0.312	0.251	0.120
Cords et al. (2024)	0.742	0.326	0.369	0.128
Danenberg et al. (2022)	0.537	0.299	0.340	0.210
Lin et al. (2023a)	0.340	0.495	0.454	0.300
Meyer et al. (2025)	0.597	0.694	0.157	0.408

Appendix

Table A16: Cross-cohort cell segmentation results. Instance F1-score at

\tau=0.5

IoU threshold per dataset and model. Best performing model per dataset is highlighted in bold, the second-best is underlined.

Figure A11: Virtual staining results. a. Average Pearson correlation between predicted and ground-truth markers across multiplex images; the baseline (light-gray cross) substitutes the closest available marker for the target. b. Representative multiplex images and their virtual staining predictions obtained from VirTues.

	GigaTIME		HistoPlexer		ROSIE
Marker	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$
BCL2	–	–	–	–	0.060	0.081
CD11C	0.034	0.379	–	–	$0.002$	$0.135$
CD138	0.022	0.364	–	–	–	–
CD16	0.050	$0.439$	$0.011$	0.505	–	–
CD163	–	–	–	–	0.057	0.027

Appendix

Table A17: Per-marker virtual staining results on Hickey et al. (2023) . Pearson correlation and SSIM for every marker predicted by at least one model. Dashes mark markers absent from a model’s output panel (GigaTIME: 10, HistoPlexer: 5, ROSIE: 23). Best performing model per marker and metric is shown in bold.

	GigaTIME		HistoPlexer		ROSIE
Marker	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$
CD163	–	–	–	–	0.030	0.228
CD20	$0.039$	$0.164$	$-0.004$	$0.053$	0.069	0.254
CD3	$0.062$	$0.240$	$0.009$	$0.017$	0.069	0.399
CD4	0.079	$0.304$	–	–	$0.075$	0.316
CD45	–	–	–	–	0.119	0.265

Appendix

Table A18: Per-marker virtual staining results on Lin et al. (2023a) . Pearson correlation and SSIM for every marker predicted by at least one model. Dashes mark markers absent from a model’s output panel (GigaTIME: 10, HistoPlexer: 3, ROSIE: 15). Best performing model per marker and metric is shown in bold.

	GigaTIME		HistoPlexer		ROSIE
Marker	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$
BCL2	–	–	–	–	0.138	0.096
CD11C	0.069	0.312	–	–	$0.028$	$0.106$
CD138	0.085	0.223	–	–	–	–
CD16	0.125	0.417	$0.014$	$0.342$	–	–
CD163	–	–	–	–	0.043	0.038

Appendix

Table A19: Per-marker virtual staining results: Phillips et al. (2021) . Pearson correlation and SSIM for every marker predicted by at least one model. Dashes mark markers absent from a model’s output panel (GigaTIME: 15, HistoPlexer: 6, ROSIE: 29). Best performing model per marker and metric is shown in bold.

	GigaTIME		HistoPlexer		ROSIE
Marker	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$	Pearson $\uparrow$	SSIM $\uparrow$
BCL2	–	–	–	–	0.037	0.115
CD11C	0.011	0.248	–	–	$-0.020$	$0.142$
CD138	-0.026	0.155	–	–	–	–
CD16	0.036	$0.212$	$0.020$	0.246	–	–
CD163	–	–	–	–	0.024	0.035

Appendix

Table A20: Per-marker virtual staining results on Schürch et al. (2020) . Pearson correlation and SSIM for every marker predicted by at least one model. Dashes mark markers absent from a model’s output panel (GigaTIME: 14, HistoPlexer: 7, ROSIE: 33). Best performing model per marker and metric is shown in bold.

Multimodal Dataset

See all 47 papers

Jun 23, 2026

SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysis

Spatial proteomics enables single-cell-resolution characterization of protein expression within tissue architecture, playing a critical role in understanding tumor microenvironments and guiding precision medicine. However, current analysis workflows remain fragmented, requiring expert manual orchestration of heterogeneous tools and limiting research scalability and reproducibility. We present SP-Mind, the first autonomous AI agent designed to unify the spatial proteomics analysis pipeline, from raw multiplexed tissue imaging to downstream phenotype discovery. Equipped with expert-curated biological analysis skills and specialized computational tools, SP-Mind converts natural-language queries into end-to-end analytical workflows without task-specific fine-tuning. To rigorously evaluate its capabilities, we introduce SP-Bench, a comprehensive benchmark spanning diverse tissue types, comprising 102 tasks across 18 distinct categories. Through extensive evaluation on SP-Bench and established downstream tasks, SP-Mind achieves state-of-the-art performance compared to existing open-source biomedical agent baselines.

Yucheng Yuan, Yuanfeng Ji, Zhongxiao Li +1

Department of Computer Science, Stanford University, Stanford, USA · Department of Radiation Oncology, Stanford University, Stanford, USA

AI Agent Benchmarks Tool-Using Agents Scientific Workflow Automation Autonomous Research Agents

Ask PDF

Apr 4, 2026

A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H&E) staining offers rich morphology yet lacks molecular resolution. We present \ours (Spatial Transcriptomics and histOlogy Representation Model), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine.

Jinxi Xiang, Siyu Hou, Yuchen Li +17

Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA · Department of Statistics and Data Science, Yale University, New Haven, CT, USA · Department of Medicine (Oncology), Stanford University School of Medicine, Stanford, CA, USA +4

Spatial Transcriptomics Gene Expression Prediction from Histopathology Images Spatial Gene Expression Prediction Clinical Outcome Prediction

Ask PDF

Apr 21, 2026

scpFormer: A Foundation Model for Unified Representation and Integration of the Single-Cell Proteomics

The integration of single-cell proteomic data is often hindered by the fragmented nature of targeted antibody panels. To address this limitation, we introduce scpFormer, a transformer-based foundation model designed for single-cell proteomics. Pre-trained on over 390 million cells, scpFormer replaces standard index-based tokenization with a continuous, sequence-anchored approach. By combining Evolutionary Scale Modeling (ESM) with value-aware expression embeddings, it dynamically maps variable panels into a shared semantic space without artificial discretization. We demonstrate that scpFormer generates global cell representations that perform competitively in large-scale batch integration and unsupervised clustering. Moreover, its open-vocabulary architecture facilitates in silico panel expansion, assisting in the reconstruction of biological manifolds in sparse clinical datasets. Finally, this learned protein co-expression logic is transferable to bulk-omics tasks, supporting applications like cancer drug response prediction. scpFormer provides a versatile, panel-agnostic framework to facilitate scalable biomarker discovery and precision oncology.

Qifeng Zhou, Lei Yu, Yuzhi Guo +5

Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, 76010, TX, USA. · Quantitative Biomedical Research Center, Department of Health Data Science and Biostatistics, Peter O’Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, 75390, TX, USA.

Topic pending

Ask PDF

Abstract

Multimodal Dataset

Explore similar work

SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysis

A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

scpFormer: A Foundation Model for Unified Representation and Integration of the Single-Cell Proteomics