Cellular states are shaped by both intrinsic gene expression programs and the spatial microenvironment in which cells reside. Spatial transcriptomics makes this context measurable, but effective representation learning requires capturing both spatially informative molecular signals and intercellular organization. Existing approaches address these challenges only partially: conventional gene selection can overlook spatially structured signals, while methods for encoding cell-cell relationships often either reduce a cell's microenvironment to proximity alone, discarding direction, or rely on absolute coordinates tied to an arbitrary tissue frame. Consequently, cells with similar neighbor distances but distinct spatial organization may receive overly similar representations, despite experiencing different biological contexts that shape their cellular states. We introduce Tessera, a spatial representation learning framework that addresses both molecular selection and cellular context. Tessera combines spatially informed gene selection with a geometric inductive bias that contextualizes each cell within its local tissue environment. By modeling relative cell-cell relationships, Tessera is translation-invariant while preserving direction, allowing neighboring cells to contribute differently according to their spatial context and cellular state. Across held-out tissues and technologies, Tessera outperforms existing spatial foundation models in representation quality and generalizes zero-shot to contexts up to 16× larger than those seen during training.
Figures & tables
Figure 1: Tessera learns spatially contextualized cell representations in two stages. Spatially informed ranking (SIR) selects spatially informative genes to represent cells, encoded by a pretrained scRNA-seq foundation model. The resulting cell embeddings are then contextualized by a geometry-aware aggregator with 2D-RoPE, letting neighbors contribute according to their relative distance and direction while remaining translation-invariant. Learning proceeds by masking cells and reconstructing masked cell expression from their spatial neighborhood.
Predictor
Pearson r
Δ vs. Tessera
t
Tessera without geometry
0.26565
−0.18370
−10.49
H VG-PCA
0.29072
−0.15863
−8.22
s cGPT (no aggregator)
0.29985
−0.14950
−7.97
S moothed PCA
0.30282
−0.14653
−7.81
N eighbour average
0.30806
−0.14129
−7.41
Tessera
0.44935
—
—
Table 1: Impact of spatial geometry. Held-out expression-imputation Pearson correlation across nine sections. Δ and t are paired section-wise comparisons with Tessera . The geometry-free ablation differs only by removing positional information.
Figure 2: Direction sensitivity of learned spatial attention. Attention patterns for four positional encodings on a held-out MERSCOPE breast tissue section. Top: Mean attention weights as a function of relative cell displacement, aggregated across eight attention heads using a shared logarithmic scale. Middle: Radial attention profiles showing spatial locality. Bottom: Angular attention profiles within a fixed 25–75 μ m distance range, normalized by their mean. The coefficient of variation (CV) quantifies directional sensitivity, with higher values indicating stronger anisotropy. RoPE exhibits the strongest directional variation (CV = 0.43), while the distance-only bias remains approximately isotropic (CV = 0.05).
Figure 3: Zero-shot spatial extrapolation beyond the training window. Comparison of four positional encodings as the spatial context expands from 256 ( 1× ) to 4096 ( 16× ) cells, without retraining or positional rescaling. Top left: Held-out cross-cell masked reconstruction loss across increasing window sizes. Top right: Illustration of the training and extrapolation windows, showing the corresponding spatial radii and scaled attention cutoffs. Bottom: Attention entropy relative to logN (left), attention mass within 25 μ m (middle), and self-attention weight (right), characterizing attention diffusion, locality, and collapse. RoPE and PoPE exhibit limited performance degradation (1.6% and 2.0%, respectively), compared with 23% for the radial distance bias, demonstrating stronger zero-shot extrapolation to larger spatial contexts.
Neighbourhood density
Transfer ( r )
Model
r
95% CI
X → X
M → M
M → X
X → M
HVG → PCA
0.1452
[0.0543,0.2414]
0.187
0.110
−0.032
0.034
Nicheformer
0.2130
[0.1037,0.3459]
0.349
0.173
0.237
0.004
scGPT-spatial
0.2526
[0.1150,0.4112]
0.389
0.209
0.125
0.037
Tessera
0.6246
[0.4897,0.7517]
0.694
0.555
0.540
0.409
Table 2: Comparison with spatial foundation models. Linear probes are trained on identical training-cell budgets and evaluated on the same nine held-out sections. Left: neighborhood-density prediction from individual cell embeddings (Pearson r , 95% CI). Right: within- and cross-platform transfer between Xenium (X) and MERSCOPE (M).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Method
kNN Purity ↑
ARI ↑
Silhouette ↑
HVG
0.6236
0.2957
-0.0323
SVG
0.7666
0.2928
-0.0280
SIR (Ours)
0.8149
0.5107
-0.0007
Appendix
Table 3: Effect of gene selection on spatial representation quality. Clustering of the HER2 cohort using the same downstream pipeline while varying only the input gene-selection strategy. Higher is better for all metrics.
Encoding
Pearson r
Δr
t
Mass <25μ m
Rose CV
No positional information
0.2575
—
—
0.068
0.008
Absolute sinusoidal
0.2580
+0.0005
1.18
0.067
0.007
PoPE [ 9 ]
0.4235
+0.1660
9.04
0.759
0.207
Additive radial distance bias
0.4176
+0.1602
9.07
0.468
0.050
RoPE (Ours)
0.4311
+0.1736
9.44
0.826
0.490
Appendix
Table 4: Effect of positional encoding on spatial gene reconstruction. Four spatial encoding strategies are compared against a geometry-free baseline under a matched training protocol. Pearson correlation is averaged across nine held-out sections; Δ and t report the improvement and paired test statistic relative to the baseline ( df=8 ). Mass denotes the fraction of attention within 25μ m, and Rose CV measures directional anisotropy (higher is more directional). Relative encodings substantially improve reconstruction, with RoPE achieving the highest correlation, spatial locality, and directional anisotropy.
Encoding
translation
rotation
reflection
uses direction
no positional information
invariant
invariant
invariant
no
absolute sinusoidal
no
no
no
yes, and absolute position
PoPE
invariant, exactly
equivariant
no
yes
radial bias brad(d)
invariant
invariant
invariant
no
angular bias in a local frame
invariant
invariant
invariant
yes, co-rotating frame
RoPE (this work)
invariant, exactly
equivariant
no
yes, in the lab frame
Appendix
Table 5: What each encoding can see. “Uses direction” means the attention logit depends on the bearing of Δij and not only on its magnitude. “Equivariant” means a rotation of the tissue is exactly equivalent to a rotation of the frequency field, so the representation transforms predictably rather than staying fixed.
Figure 4: Attention under spatial transformations. Attention patterns for a fixed query cell under translation, rotation by 37∘ , and both combined. RoPE and PoPE are translation-invariant but rotation-sensitive; the radial bias is invariant to both, while absolute sinusoidal encoding is sensitive to both. Tables report maximum absolute changes in attention logits and weights, evaluated in float64.
Encoding
translation
rotation 37∘
reflection
live ∣A∣
no positional information
0
0
0
26.56
absolute sinusoidal
1.256
0.460
0.395
20.94
PoPE
0
27.63
33.79
30.72
radial distance bias
0
0
0
21.76
RoPE (this work)
0
35.05
30.90
26.38
Appendix
Table 6: Sensitivity to rigid motion. Largest absolute change in the attention logit when a held-out window is translated ( 12.3×8.9 mm), rotated ( 37∘ about its centroid) or reflected, over eight windows. Live ∣A∣ is the largest untransformed logit, so no row can appear invariant by producing no signal. Zeros are exact in float64 and asserted: for a relative encoding the translation entry is an identity, so a non-zero raises an error rather than being reported.
Figure 5: Neighbour-count decodability against target radius; bands are 95% bootstrap intervals over nine sections and the dashed line marks the 200μ m attention cutoff. Had the model learned its own cutoff the curve would peak there; it does not, decreasing monotonically and reaching its maximum at the finest radius tested. The registered criterion holds at 50/100/200μ m and fails at 400/800μ m, and the verdict is unanimous across six training seeds.
Radius
Tessera ( 50 k)
best baseline
six-seed mean ± SD
seeds above
verdict
50μ m
0.7651
0.3332
0.7234±0.0241
6/6
holds 6/6
100μ m
0.7113
0.2858
0.6698±0.0229
6/6
holds 6/6
200μ m
0.6246
0.2526
0.5894±0.0197
6/6
holds 6/6
400μ m
0.4938
0.2312
0.4762±0.0158
6/6
fails 0/6
800μ m
0.3630
0.1941
0.3708±0.0222
6/6
fails 0/6
Appendix
Table 7: Neighbour-count decodability by target radius. Left: a single checkpoint at 50,000 steps with the best baseline for reference. Right: six seeds at 18,000 steps, mean ± standard deviation, with the per-seed verdict under the registered criterion.
Section
none
absolute
PoPE
radial
RoPE
Xenium breast, invasive lobular
0.2366
0.2379
0.3592
0.3481
0.3636
Xenium colon
0.3372
0.3370
0.5210
0.5194
0.5232
Xenium lung, non-diseased
0.1237
0.1231
0.2348
0.2332
0.2440
Xenium melanoma
0.3338
0.3371
0.5362
0.5324
0.5453
MERSCOPE breast, patient 1
0.2273
0.2276
0.4765
0.4713
0.4868
MERSCOPE lung, patient 2
0.3265
0.3264
0.5031
0.4774
0.5086
Appendix
Table 8: The encoding comparison, all nine held-out sections , no selection. Values are held-out imputation Pearson at 18,000 steps, seed 0 .
Single-cell transcriptomics enables profiling of cellular states at unprecedented resolution, but its high dimensionality, sparsity, and technical batch effects pose significant challenges for representation learning. Existing single-cell foundation models typically encode each cell independently or only model cells from the same batch for denoising, thereby underutilizing the rich relational information across batches and cell types to model gene expression patterns. We argue that single-cell models can benefit from more informative cell-context modeling. By comparing consistency and variation across cells, models can capture fine-grained gene-gene dependencies associated with cell states, which are essential for learning high-quality representations. Inspired by the use of multiple sequence alignment (MSA) context in protein modeling, we propose CellMSA, a single-cell representation learning framework that introduces an MSA-inspired inductive bias into transcriptomic modeling. For each target cell, CellMSA retrieves relevant cells from different batches and biologically related cell types as context, and summarizes cross-cell patterns into a context-dependent gene-pair representation. This representation is then injected into a pair-aware target-cell encoder for fine-grained representation learning. We pretrain CellMSA on a large-scale human single-cell corpus of approximately 109 million cell observations, including 65.6 million primary observations. Experiments show that our framework consistently outperforms existing methods across multiple benchmarks. Code is available at the following repository: https://github.com/PharMolix/CellMSA.
Suyuan Zhao, Minghao Liu, Yizhen Luo +1
Institute for AI Industry Research (AIR), Tsinghua University · Department of Computer Science and Technology, Tsinghua University · Tsinghua Institute of Multidisciplinary Biomedical Research (TIMBR), Tsinghua University +2
This paper shows that latent-space predictive pretraining can provide a scalable route to foundation models for spatial transcriptomics. Existing spatial transcriptomics foundation models primarily reconstruct masked gene identities or expression values, potentially encouraging the reproduction of assay-specific technical variation and limiting representation transferability. To avoid directly reconstructing such variation, we shift the prediction target from observed gene measurements to latent cell representations and introduce CellWorld, which predicts the latent representations of masked cells from visible spatial context and a limited partial-expression hint. We pretrain four CellWorld variants, spanning 5.74M to 94.56M trainable parameters, on a corpus of 46 million human cells. Our controlled scaling experiments show that performance improves with model capacity, particularly on spatial tasks, while spatial transfer depends more on sufficient optimization and broad biological source diversity than on cell count alone. Across four held-out datasets, even CellWorld-Small, with 5.74M trainable parameters, outperforms every baseline on all 11 linear-probe benchmarks and all seven fine-tuned spatial benchmarks. Most notably, a frozen CellWorld-Large pretrained on only 5% of the corpus with broad biological source coverage outperforms every fully fine-tuned baseline across all seven spatial benchmarks. Code is available at https://github.com/UoM-HealthAI/CellWorld.
Spatial transcriptomics (ST) links tissue morphology with molecular programs, motivating multimodal pretraining methods that align histology images with gene expression. However, existing approaches suffer from two key limitations: spatially informative gene selection is often dominated by ubiquitous housekeeping genes, leading to weakly discriminative representations, and independent spot-patch alignment fails to capture spatial dependencies that are critical for tissue organization. To address these challenges, we introduce PaSTel, a hierarchical multimodal pretraining framework that integrates biological priors at three levels. At the spot level, TF-IDF reweighting is used to identify spatially informative genes; at the functional level, curated KEGG pathways serve as anchors for encoding global biological semantics; and at the regional level, spatial clustering aggregates neighboring spots to model meso-scale tissue structure. Across multiple downstream tasks, PaSTel consistently outperforms existing vision and vision-omics encoders, demonstrating that incorporating multiscale biological priors yields more informative and transferable representations for spatial transcriptomics.
Azim Dehghani Amirabad, Junchao Zhu, Pushpak Pati +3