cs.AIAug 14, 2026

Disentangled Shared Representations Improve Morpho-Transcriptomic Integration

Authors: Julian Ostermaier, Swann Ruyter, Reuben Dorent, Daniel Racoceanu

Organizations: ESPCI Paris, PSL University, Paris, France · Sorbonne Université, CNRS, Inserm, AP-HP, Inria, Paris Brain Institute – ICM, Paris, France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Spatial transcriptomics (ST) enables the simultaneous profiling of gene expression and tissue morphology, creating an opportunity to learn multimodal representations capturing shared morpho-transcriptomic structure. However, standard multimodal models often compress modalities into a common latent space without explicitly separating shared and modality-specific sources of variation, which may limit downstream utility. We investigate whether explicit disentanglement of shared and private latent components improves multimodal representation learning for paired Hematoxylin & Eosin (H&E) and ST data. We compare VAE-based and contrastive approaches, each in standard and disentangled variants, across two cancer cohorts under matched experimental conditions. Representations are evaluated using cross-modal reconstruction, downstream probing and cross-modal probe transfer. The experiments suggest two main trends. First, contrastive objectives yield higher downstream probing performance than VAE-based models. Second, disentangled variants improve the selected reconstruction and probing metrics, although the gains depend on the model family, task, direction, and disentanglement strength. Overall, our results suggest that explicitly factorizing shared and modality-specific information can improve multimodal representation learning for spatial transcriptomics and provides a useful evaluation framework for future foundation models.

Figures & tables

Figure 1: Multimodal representation learning pipeline. (A) Pre-processing of the paired gene expression and H&E image patch data. (B) Representations are learned using different model families including contrastive and VAE methods as well as their disentangled versions. Representations are evaluated using (C) cross-modal reconstruction and (D) prediction of downstream labels.
 FamilyModeltop-50 PCC (HE →\to G)cosine similarity (G →\to HE)
   KK 2005001000KK 2005001000
ColonUni- modalPCA47.9 ±\pm 1.551.4 ±\pm 1.052.7 ±\pm 0.859.2 ±\pm 1.060.7 ±\pm 0.560.7 ±\pm 0.8
VAE43.7 ±\pm 0.646.7 ±\pm 0.847.2 ±\pm 1.057.0 ±\pm 0.558.5 ±\pm 0.458.9 ±\pm 0.5
VAEMMVAE42.3 ±\pm 0.448.6 ±\pm 0.249.7 ±\pm 0.560.0 ±\pm 0.359.9 ±\pm 0.361.4 ±\pm 0.2
MMVAE+47.5 ±\pm 0.650.8 ±\pm 0.852.5 ±\pm 0.861.5 ±\pm 0.460.8 ±\pm 0.362.5 ±\pm 0.5
MMVAE+ sg\mathrm{sg}47.6 ±\pm 0.551.8 ±\pm 0.453.2 ±\pm 0.662.3 ±\pm 0.361.5 ±\pm 0.562.7 ±\pm 0.4
Table 1: Cross-modal reconstruction results on Colon and GBM datasets for different gene panel sizes KK . Metrics are reported as percentages and presented as mean ±\pm standard deviation over five repeated runs with different random seeds. For disSSL, 10−310^{-3} , 10−210^{-2} , and 10−110^{-1} denote the value of β\beta . Bold and underline indicate the best and second-best values per column within each model family.
Figure 2: Downstream task performance for tissue niche classification ( HE→GHE\rightarrow G ) and tissue type classification ( G→HEG\rightarrow HE ). AUC prediction performance (five repeated runs with random seeds) with gene panel sizes KK for (A) tissue niche classification derived from scRNA-seq reference Belgian cohort and (B) pathologist tissue types classification. Stars indicate non-disentangled and squares disentangled versions. The gray arrow connects contrastive models from CLIP ( β=0\beta=0 ) to increasing disentanglement strength.
ModelContrastiveGenerativeShared latentPrivate latent
MMVAE [ 16 ]–✓✓–
MMVAE+ [ 13 ]–✓✓✓
MMVAE+ sg\mathrm{sg} [ 11 ]–✓✓✓
CLIP [ 14 ]✓–✓–
disSSL [ 19 ]✓–✓✓
Table 2: Summary of the evaluated representation learning models.
ComparisonMetricpp (BH)Sig.
Colon
——— 
GBM
MMVAE vs MMVAE+sgPCC (HE →\to G) 5000.035∗*
MMVAE vs MMVAE+Cosine (G →\to HE) 2000.015∗*
MMVAE vs MMVAE+Cosine (G →\to HE) 5000.019∗*
Table 3: Statistically significant Wilcoxon signed-rank test results after Benjamini–Hochberg FDR correction ( p<0.05p<0.05 ) across Colon and GBM dataset (HVG, log1p preprocessing). For each slide performance metrics were averaged across repeated runs, and paired slide-level metric values were used as observations in the Wilcoxon signed-rank test. Significance levels refer to the BH-adjusted pp -value: ∗ p<0.05*\ p<0.05 ; ∗∗ p<0.01**\ p<0.01 ; ∗∗∗ p<0.001***\ p<0.001 .
HyperparameterVAE-basedContrastive
Optimizer
Learning rate1×10−41\times 10^{-4}1×10−41\times 10^{-4}
Weight decay1×10−51\times 10^{-5}—
OptimizerAdam (AMSGrad)Adam (AMSGrad)
Training schedule
Batch size64256
Table 4: Training hyperparameters shared across all model configurations.
CohortHEST IDs
Colorectal CancerZEN37, ZEN40, ZEN42, ZEN43, ZEN44, ZEN47, ZEN49
Table 5: Overview of the retained samples used for model training after filtering of low quality samples and enforcing one sample per patient. HEST ID corresponds to the unique identifier of the sample in the HEST database [ 7 ] .
Figure 3: Downstream task performance for tissue niche classification ( HE→GHE\rightarrow G ). AUC prediction performance (mean over five repeated runs with different random seeds) from different gene panel sizes KK for tissue niche classification derived from scRNA-seq reference Korean cohort. Stars indicate the non-disentangled model and circles the disentangled versions. The gray arrow connects contrastive models from CLIP ( β=0\beta=0 ) to increasing disentanglement strength.
More in this topic

Cross-Modal Representation Learning

Cross-modal representation learning connects information across data types such as images, text, audio, biosignals, and sensor measurements.

See all 210 papers

Explore similar work

CardsList

  1. A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

    Apr 4, 2026Jinxi Xiang, Siyu Hou, Yuchen Li +17Spatial TranscriptomicsGene Expression Prediction from Histopathology Images

    Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA · Department of Statistics and Data Science, Yale University, New Haven, CT, USA · Department of Medicine (Oncology), Stanford University School of Medicine, Stanford, CA, USA +4

  2. Cross-Modal Knowledge Distillation from Spatial Transcriptomics to Histology

    Apr 10, 2026Arbel Hizmi, Artemii Bakulin, Shai Bagon +1Computational PathologyCross-Modal Knowledge Distillation

    Weizmann Institute of Science · Reichman University

  3. LATTICE: Graph Self-Supervised Learning for Multimodal Spatial Omics Integration

    Jul 15, 2026Jagan Mohan Reddy Dwarampudi, Veena Kochat, Suresh Satpati +2Spatial TranscriptomicsGraph SSL

    University of Houston · MD Anderson Cancer Center