Sampling-Aware 3D Spatial Analysis in Multiplexed Imaging

Ido Harlev; Tamar Oukhanov; Raz Ben-Uri; Leeat Keren; Shai Bagon

cs.CVApr 9, 2026

Sampling-Aware 3D Spatial Analysis in Multiplexed Imaging

Authors: Ido Harlev, Tamar Oukhanov, Raz Ben-Uri, Leeat Keren, Shai Bagon

Organizations: Weizmann Institute of Science, Israel

Abstract

Highly multiplexed microscopy enables rich spatial characterization of tissues at single-cell resolution, yet most analyses rely on two-dimensional sections despite inherently three-dimensional tissue organization. Acquiring dense volumetric data in spatial proteomics remains costly and technically challenging, leaving practitioners to choose between 2D sections or 3D serial sections under limited imaging budgets. In this work, we study how sampling geometry impacts the stability of commonly used spatial statistics, and we introduce a geometry-aware reconstruction module that enables sparse yet consistent 3D analysis from serial sections. Using controlled simulations, we show that planar sampling reliably recovers global cell-type abundance but exhibits high variance for local statistics such as cell clustering and cell-cell interactions, particularly for rare or spatially localized populations. We observe consistent behavior in real multiplexed datasets, where interaction metrics and neighborhood relationships fluctuate substantially across individual sections. To support sparse 3D analysis in practice, we present a reconstruction approach that links cell projections across adjacent sections using phenotype and proximity constraints and recovers single-cell 3D centroids using cell-type-specific shape priors. We further analyze the trade-off between section spacing, coverage, and redundancy, identifying acquisition regimes that maximize reconstruction utility under fixed imaging budgets. We validate the reconstruction module on a public imaging mass cytometry dataset with dense axial sampling and demonstrate its downstream utility on an in-house CODEX dataset by enabling structure-level 3D analyses that are unreliable in 2D. Together, our results provide diagnostic tools and practical guidance for deciding when 2D sampling suffices and when sparse 3D reconstruction is warranted.

Figures & tables

Figure 2: Simulation (MRF): sampling geometry affects recovery of global prevalence versus local interactions. We simulate 3D tissues from a pairwise MRF with unary parameters $\boldsymbol{\alpha}$ (global prevalence) and pairwise affinities $\mathbf{B}$ (local interactions). ( a ) With the same $\boldsymbol{\alpha}$ , changing $\mathbf{B}$ yields markedly different spatial organizations, from near-i.i.d. mixtures to clustered domains. ( b ) Changing the $\boldsymbol{\alpha}$ , for a fixed $\mathbf{B}$ yields different scales of structures. ( c ) Under matched voxel budgets, MPLE recovers $\boldsymbol{\alpha}$ reliably from both independent 2D sections and sparse serial sampling, whereas recovery of $\mathbf{B}$ (and interaction-derived quantities) exhibits substantially higher error and variance under independent 2D sampling, especially in regimes with spatial localization.

Figure 3: Empirical stability in real IMC under dense axial sampling (2 $\mu$ m; [ 13 ] ). We treat a densely sampled IMC stack as a volumetric reference and repeatedly evaluate section-based statistics across depth. ( a ) Total cell counts per type in the full volume (global abundance). ( b ) Detectability under planar sampling: for each type, the probability that a random set of $M{=}20$ sections contains at least $k{=}100$ cells of that type (a section-level proxy for 3D dispersion vs clustering). B cells have low detectability, due to their low abundance and tendency to cluster, while similarly low-abundance but spatially dispersed types (e.g., endothelial) are detected consistently; conversely, abundant yet clustered types (e.g., myCAF_SMA fibroblasts) can still be missed in many section samples. ( c ) Variability of neighborhood enrichment for Fibroblasts_apCAF: distribution (across sampled sections) of enrichment z-scores against each partner type (dashed lines indicate $|z|{=}2$ ). Even within a single tissue volume, section choice can change whether an interaction appears strongly enriched or negligible, illustrating high sampling-induced variance.

Figure 4: Inter-slice matching as constrained assignment with cell-type-specific geometric priors (PDAC CODEX, $\Delta z=4\,\mu$ m). ( a ) Two consecutive sections illustrate correspondence ambiguity: multiple nearby candidates of the same phenotype may exist for a given cell projection (highlighted regions). ( b )(i) Candidate correspondences are constructed under two constraints: phenotype consistency and a cell-type-specific proximity gate. (ii) We form the in-plane distance matrix $D_{ij}$ [ $\mu m$ ] between cells in adjacent sections; entries violating phenotype or distance constraints are set to $\infty$ . Green dashed entries indicate selected matches after solving a one-to-one assignment via the Hungarian algorithm. (iii) The assignment partitions observations into shared cells (SC), matched across adjacent sections; and lone cells (LC), observed only in one section. ( c ) Cell-type-specific proximity tolerances and weak ellipsoidal priors are derived from empirical distributions of 2D cross-sectional areas (examples shown).

Figure 5: Spacing trade-offs and localization accuracy on densely sampled IMC data [ 13 ] . ( a ) Effect of axial spacing $\Delta z$ relative to a dense 2 $\mu$ m reference. For each spacing, the left stacked bar shows the composition of cells observed in the sampled stack , decomposed into shared cells (SC; observed in consecutive sections) and lonely cells (LC; observed once), illustrating redundancy and overlap. The right stacked bar shows coverage of true unique cell IDs relative to the dense reference, decomposed into captured (green) and missed (red) cells. As spacing increases, redundancy (SC) decreases and the fraction of missed unique cells increases, reducing overall coverage. ( b ) In-plane 3D centroid localization error at $\Delta z=4\,\mu$ m relative to the 2 $\mu$ m reference. The histogram shows the distribution of centroid errors. The dashed black line marks the mean error (2.99 $\mu$ m), and the dotted black line marks the standard deviation (3.86 $\mu$ m). The purple dashed line indicates a representative neutrophil diameter for biological context. Localization errors remain small relative to typical cell size, indicating that sparse reconstruction preserves neighborhood-scale geometry under moderate spacing.

Biological goal	Target statistic	Risk under 2D	Preferred acquisition (fixed budget)
Abundance / composition	global frequencies, proportions	low (averages out)	2D sections (maximize coverage)
Rare population detection	presence / prevalence	medium (cluster-dependent)	mixed: 2D for discovery; add sparse serial if localized
Cell–cell interactions	neighborhood enrichment, contact proxies	high (depth collapse confounds neighborhoods)	sparse serial sections + reconstruction (preserve continuity)
Spatial clustering / niches	local micro-environments, patches	high (section-to-section variance)	sparse serial sections + reconstruction
Structure-centric analysis	ducts/vessels as contiguous objects	very high (fragmentation)	sparse serial sections + reconstruction
Along-structure gradients	marker profiles along objects	very high (orientation + depth collapse)	sparse serial sections + reconstruction

Table 1: Decision support under a fixed imaging budget. Independent 2D sections are efficient for global composition, while analyses that rely on local neighborhoods, interactions, or extended structures degrade under depth collapse and fragmentation, favoring sparse serial acquisition and reconstruction.

Figure 6: Structure-centric analysis enabled by sparse 3D reconstruction (PDAC CODEX, $\Delta z=4\,\mu$ m). ( a ) Consecutive 2D sections show fragmented cross-sections of a ductal structure (left), which become a coherent connected object after 3D reconstruction (right). This restoration of continuity enables structure-centric coordinate systems and object-level queries that are not observable in single sections. ( b ) Effect of planar measurement on proximity estimates. For representative cell-type and structure-to-structure queries, distances computed within a single 2D section are systematically larger than distances computed in the reconstructed 3D volume. Mean distances (dashed lines) shift substantially (e.g., duct–vessel and epithelial–neutrophil pairs), indicating that planar measurements can bias proximity- and interaction-based metrics.

Figure 7: Along-structure gradients enabled by sparse 3D. Using structure-centric coordinates derived from reconstructed 3D objects, we compute marker profiles along ducts/vessels. Sparse 3D reconstruction reduces confounds from arbitrary 2D intersection geometry, enabling depth-aware aggregation of along-structure trends.

Microscopy Image Analysis

Microscopy image analysis uses computational methods to detect, segment, track, reconstruct, and quantify structures in scientific images.

See all 198 papers

Sep 18, 2026

Learning 3D biophysical cell properties from 2D images and cell-population statistics

Inferring 3D cellular properties from 2D microscopy is difficult when a reference instrument reports only population statistics rather than labels for individual cells. Here we develop a population-supervised framework that maps single 2D red-cell images to latent biophysical quantities and aggregates them to mean corpuscular volume, red-cell distribution width and mean corpuscular haemoglobin. The model combines shared local inference, a biophysically structured decoder for volume and haemoglobin, learned instance weighting and device-specific calibration. We formalise conditions under which aggregate observations identify restricted instance predictors, show why population agreement does not by itself identify single-cell properties or 3D geometry, and derive the dispersion penalty induced by subset mean matching. The development dataset comprises 390 specimens and 1,105 acquisitions across six devices, with reported Pearson correlations of 0.86--0.98 against a Sysmex analyser. The framework provides a testable route from 2D images and population supervision to 3D cellular biophysics without claiming explicit 3D reconstruction.

Santiago Hernández-Orozco, Hector Zenil

Algocyte | Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, UK · Algorithmic Dynamics Lab, Department of Biomedical Computing, School of Biomedical Engineering and Imaging Sciences, King’s Institute for AI, King’s College London, UK

Microscopy Image Analysis

Ask PDF

Jun 22, 2026

3D Masked Autoencoders are Robust Learners of Volumetric and Multimodal Cellular Representations for Microscopy

Self-supervised learning in fluorescence microscopy often relies on 2D projections, despite the inherently three-dimensional nature of cells. We present a systematic comparison of 2D and 3D masked autoencoders (MAE-2D vs. MAE-3D) on volumetric microscopy data. Under matched architectures and training protocols, MAE-3D consistently outperforms 2D max-projection and slice-based variants on downstream single-cell tasks. We further align visual representations with a pretrained protein language model (ESM2) and show that cross-modal supervision yields larger gains for volumetric models. Channel cross-attention and frequency-domain regularization are critical for leveraging 3D spatial context. On protein--protein interaction prediction, our best model achieves a ROC--AUC of 0.86, while on protein localization it reaches an AUC

_{\text{micro}}

of 0.95 and an F1

_{\text{micro}}

of 0.74, demonstrating competitive performance on both tasks. Overall, our findings highlight the potential of volumetric modeling and multimodal alignment for representation learning in single-cell microscopy.

Amirhossein Kardoost, Lion Gleiter, Tingying Peng +1

Institute of AI for Health & Helmholtz AI, Computational Health Center, Helmholtz Munich – German Research Center for Environmental Health, Neuherberg, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Department of Medicine III, Ludwig-Maximilian-University Hospital, Munich, Germany +2

Cross-Modal Representation Learning Masked Autoencoders Fluorescence Microscopy 3D Representation Learning Self-Supervised Visual Representation Learning Microscopy Image Analysis

Ask PDF

May 27, 2026

Geometry-First Generative Spatial Single-Cell Reconstruction

Single-cell RNA sequencing (scRNA-seq) profiles large numbers of cells but loses spatial context, whereas spatial transcriptomics (ST) preserves partial spatial structure at lower resolution. Most existing integration methods either deconvolve spot mixtures or map cells onto a measured spot lattice, which ties reconstructions to a fixed grid and slide-specific coordinate systems, a limitation that is especially problematic in unpaired settings. We propose GEARS, a geometry-first framework that reconstructs an intrinsic single-cell spatial geometry guided by ST, without relying on cell-type labels, histological images, or cell-to-spot assignment. GEARS first learns a domain-invariant expression encoder that aligns ST spots and dissociated cells, and then trains a permutation-equivariant generator with a diffusion-based refiner with EDM-style preconditioning to generate local spatial geometries under pose-invariant supervision derived from ST coordinates. At inference, GEARS reconstructs geometry on many overlapping subsets of scRNA-seq cells, aggregates predicted pairwise distances across subsets, and solves a global distance-geometry problem to obtain canonical two-dimensional coordinates and a dense distance matrix. Extensive quantitative and qualitative experiments, including cross-section generalization, show that GEARS consistently improves global distance preservation, local neighborhood fidelity, and spatial distribution alignment compared to strong spatial mapping and deconvolution baselines.

Ehtesamul Azim, Muhtasim Noor Alif, Tae Hyun Hwang +2

University of Central Florida Orlando, Florida, USA · Vanderbilt University Medical Center Nashville, Tennessee, USA · Arizona State University Tempe, Arizona, USA

Spatial Transcriptomics Genomics Single-Cell RNA Sequencing

Ask PDF

Abstract

Microscopy Image Analysis

Explore similar work

Learning 3D biophysical cell properties from 2D images and cell-population statistics

3D Masked Autoencoders are Robust Learners of Volumetric and Multimodal Cellular Representations for Microscopy

Geometry-First Generative Spatial Single-Cell Reconstruction