q-bio.QMJan 10, 2026
SaveImaging-anchored multiomics in cardiovascular disease: integrating cardiac imaging, bulk, single-cell, and spatial transcriptomics
Organizations: International Ph.D. Program in Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan · AIBioMed Research Group, Taipei Medical University, Taipei, Taiwan · Computational Biology Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science, Emory University, Atlanta, GA, USA ·
Abstract
Cardiovascular disease arises from interactions between inherited risk, molecular programmes, and tissue-scale remodelling that are observed clinically through imaging. Cardiac MRI (CMR), computed tomography (CT), and echocardiography are integral to routine cardiovascular care, while bulk RNA sequencing, single-cell RNA sequencing, and spatial transcriptomics are providing increasingly detailed molecular characterization of cardiac tissue. Yet, these imaging and molecular data are still analysed in largely separate pipelines. This review examines joint representations that link cardiac imaging phenotypes to transcriptomic and spatially resolved molecular states. An imaging-anchored perspective is adopted in which echocardiography, CMR, and CT define a spatial phenotype of the heart, and bulk, single-cell and spatial transcriptomics provide cell-type- and location-specific molecular context. We define the representation requirements of each modality, compare multimodal fusion strategies, and synthesize integrative pipelines for radiogenomics, spatial alignment, and image-based gene-expression prediction, together with their validation requirements, limitations, and failure modes. Spatial multiomic maps of human myocardium and atherosclerotic plaque, together with single-cell, spatial, and multimodal medical foundation models, are advancing imaging-anchored multiomics; however, cost, scalability, and tissue availability remain substantial barriers to large-scale cardiovascular translation.
Figures & tables
| Dataset/study | Type | Cardiovascular focus | Modalities included | Approximate scale and access | Link |
|---|---|---|---|---|---|
| UK Biobank imaging extension [ 17 ] | Population imaging cohort | Cardiac structure, function, and risk in mid-life adults | CMR, vascular ultrasound and other MRI sequences linked to genome-wide genotyping, exome sequencing, plasma proteomics, metabolomics, EHR, and questionnaires | 100,000 participants with CMR and ¿500,000 with genomics; application-based access | UK Biobank |
| Multi-Ethnic Study of Atherosclerosis (MESA) [ 31 ] | Population cohort with multimodal imaging | Subclinical atherosclerosis and incident CVD in a diverse community sample | CMR, coronary artery calcium CT, carotid ultrasound, and other imaging linked to genotype, labs, and risk-factor profiling | 6,800 participants; controlled access via NHLBI repositories | BioLINCC |
| Hypertrophic Cardiomyopathy Registry (HCMR) [ 32 ] | Disease registry and biobank | Risk stratification in hypertrophic cardiomyopathy | CMR, echocardiography and electrocardiography (ECG) combined with targeted gene sequencing and circulating biomarkers | 2,700 patients; collaborative access via data-use agreements | HCMR |
| MIMIC-IV and related PhysioNet modules [ 33 ] | Critical-care waveform and EHR resource | Acute CVD and haemodynamic instability in ICU/ED settings | Twelve-lead ECG waveforms linked to high-resolution EHR; additional echo and note modules for subsets | 800,000 ECGs in 160,000 subjects; credentialed access via PhysioNet | PhysioNet |
| EchoNet family (EchoNet-Dynamic, EchoNet-LVH) [ 5 , 16 ] | Curated echocardiography video datasets | LV systolic function, wall motion, and hypertrophy phenotypes | Apical four-chamber echo videos with contour tracings, volumetric measurements, and labels (EF, LVH) | On the order of videos; de-identified datasets downloadable for research | EchoNet |
| Cardiac Atlas Project CMR cohorts [ 34 ] | Aggregated CMR repositories | Cardiac structure and function in health and early disease | 3D cine CMR studies with segmentations, finite-element meshes and clinical covariates in selected cohorts | Thousands of CMR exams; access under data-use agreements | Cardiac Atlas |
Table 1: Public multimodal cardiology datasets relevant to imaging-anchored multiomics, including resources that directly pair cardiovascular imaging with molecular data and adjacent multimodal cardiovascular resources useful for encoder pretraining and transfer learning.
| Biomarker (modality) | Clinical interpretation | Typical computational extraction | Plausible or reported molecular correlates |
|---|---|---|---|
| LGE extent and pattern (CMR) [ 20 ] | Focal myocardial scar/fibrosis; arrhythmic and HF risk | Segmentation of LGE-positive regions; regional burden by AHA segments; texture/radiomics | Fibroblast activation, collagen deposition (ECM programmes), immune infiltration in border zones; stress and remodelling programmes in cardiomyocytes |
| Global longitudinal strain (echo) [ 21 ] | Subclinical systolic dysfunction; prognostic across many CVD states | Speckle-tracking strain curves; regional strain maps; video encoders for deformation fields | Energetic stress, sarcomeric dysfunction, inflammatory signalling; chamber-specific cardiomyocyte stress responses |
| Low-attenuation noncalcified plaque (coronary CTA) [ 22 ] | Lipid-rich plaque; higher MI risk | Plaque segmentation and attenuation profiling; radiomics; phenotype classifiers | Macrophage-driven inflammation, necrotic core formation, matrix remodelling; immune niches at plaque shoulders |
| Napkin-ring sign/high-risk plaque morphology (CTA) [ 23 ] | High-risk plaque architecture; potential plaque vulnerability | CNN/ViT-based plaque morphology classification; handcrafted geometry features | Protease and cytokine signalling, macrophage activation, smooth muscle cell phenotypic switching; local hypoxia |
| Perivascular fat attenuation index (FAI) (CTA) [ 24 ] | Non-invasive readout of coronary inflammation | Perivascular fat segmentation; attenuation gradients; radiotranscriptomic mapping | Inflammatory adipose and vascular wall transcripts; cytokine signalling and fibrosis-related programmes |
Table 2: Representative cardiovascular imaging biomarkers that often serve as anchors in imaging-anchored multiomics, with typical computational extraction strategies and plausible molecular correlates.
Figure 1: The resolution scale illustrates the gap between in vivo cardiac imaging at centimetre-to-millimetre resolution and molecular spatial profiling, from Visium 55 µm spots and Visium HD/Stereo-seq 2-8 µm bins to single-cell platforms such as Xenium and MERSCOPE. Cardiac imaging modalities, including echocardiography, cardiac magnetic resonance imaging (CMR), and cardiac CT/CTA, define spatial phenotypes of the heart using measures such as ejection fraction, global longitudinal strain, wall motion, late gadolinium enhancement, T1 mapping, cine imaging, plaque features, fat attenuation index, and coronary artery calcium. These phenotypes can be indexed at the voxel, American Heart Association segment, or joint latent-space level and linked to molecular omics layers, including spatial transcriptomics, bulk or single-nucleus RNA sequencing, and proteomics/spatial omics. The resulting joint representations support downstream tasks including clinical prediction, virtual transcriptomics, spatial alignment, radiogenomics, and multimodal fusion. Foundation models spanning single-cell/spatial omics, vision–omics, imaging, and medical multimodal data provide computational support across the framework. Applications include disease subtyping, mechanism discovery, biomarker prioritisation, and risk prediction. Abbreviations: AHA, American Heart Association; CMR, cardiac magnetic resonance imaging; CT, computed tomography; CTA, computed tomography angiography; EF, ejection fraction; GLS, global longitudinal strain; LGE, late gadolinium enhancement; FAI, fat attenuation index; CAC, coronary artery calcium; scVI, single-cell variational inference; DCA, deep count autoencoder; DCCA, deep canonical correlation analysis; MOFA+, multi-omics factor analysis; VLM, vision–language model.
Figure 2: Multimodal fusion architectures for radiomics–omics integration. Panel (A): early fusion concatenates radiomics and omics feature vectors followed by a single predictive model. Panel (B): intermediate fusion uses modality-specific encoders and a shared latent space, often with reconstruction heads; the shared latent captures biology concordant across modalities while private components retain modality-specific information. Panel (C): late fusion combines predictions from separate radiomics and omics models via an ensemble or calibration layer; it is robust to missing modalities but cannot capture cross-modal feature interactions. Panel (D): graph-based and contrastive approaches construct cross-modal latent spaces using biological or spatial graphs and contrastive objectives.
| Approach | Strengths | Common failure modes | When it is a good default |
|---|---|---|---|
| Classical regression/survival models with hand-crafted radiomics summaries | High interpretability; strong calibration; stable in small samples | Limited ability to capture spatial heterogeneity and cross-modal interactions; features may miss subtle biology | Early translational work; limited paired data; when interpretability and calibration dominate |
| Early fusion (feature concatenation) | Simple; uses standard ML toolkits; often competitive baselines | Sensitive to missing modalities; scaling/normalization issues; collinearity, and overfitting with high-dimensional omics | Moderate sample sizes with mostly complete modalities; when a strong baseline is needed quickly |
| Intermediate fusion (joint latent space: DCCA, multimodal VAE, and HEALNet) | Captures non-linear cross-modal interactions; supports “virtual omics” and mechanistic exploration; handles missing modalities via product-of-experts | Latent space collapse; modality imbalance; requires careful regularization and validation | When paired subsets exist but are not large; when cross-modal generation/interpretation is desired |
| Late fusion (ensembles/stacking) | Robust to missing modalities; reuses validated unimodal models; simplifies regulatory adoption | Cannot learn feature-level synergy; may underperform when cross-modal interactions are important; forfeits opportunity to learn from costly paired data | When missingness is high; when unimodal models are already mature and well-validated; as deployment fallback |
| Graph/contrastive and transformer-based fusion | Naturally incorporates spatial and biological priors; can scale with pretraining; captures neighbourhood relationships lost by flat vectors | Data-hungry; computationally expensive; prone to spurious shortcuts without strong evaluation | When large weakly labelled data exist for pretraining, or strong spatial/biological graphs are available |
Table 3: Practical comparison of traditional unimodal/low-dimensional modelling versus multimodal fusion strategies for imaging-anchored multi-omics.
| Task | Example outputs | Useful metrics | Validation emphasis |
|---|---|---|---|
| Clinical prediction/risk modelling | Binary outcome, time-to-event risk, treatment response | AUC/time-dependent AUC, C-index, Brier score; calibration slope/curve; decision-curve net benefit [ 89 ] | External validation; leave-one-site-out; calibration and subgroup performance; reporting per TRIPOD+AI/PROBAST+AI [ 91 ] |
| Radiogenomic association | Imaging feature gene/pathway score | Effect sizes, false discovery rate control; pathway enrichment; replication rate | Confounder adjustment; permutation tests; replication in independent cohorts |
| Cross-modal alignment/retrieval | Paired embeddings; matched region–spot correspondences | Cross-modal retrieval (Recall@K), embedding correlation/CCA correlation, alignment error | Robustness to missing modalities; negative controls (shuffled pairing); sensitivity to batch |
| Spatial domain discovery | Spot clusters/spatial domains | Adjusted Rand index (if labels exist), silhouette score; spatial autocorrelation statistics | Stability under downsampling; cross-section reproducibility; biological plausibility via marker genes |
| Virtual transcriptomics | Predicted gene/module expression maps | Pearson/Spearman correlation, , RMSE; spatial pattern similarity | Hold-out donors/centres; robustness across staining/scanner protocols; uncertainty estimates |
Table 4: Evaluation metrics and validation designs commonly used (or recommended) in imaging-anchored multiomics.
| Framework/study | Modalities integrated | Integration strategy | Cardiovascular relevance - example use | Key contribution |
|---|---|---|---|---|
| General-purpose multimodal frameworks applicable to imaging–omics integration | ||||
| MOFA+, scVI/totalVI, and scvi-tools [ 50 , 51 , 92 ] | Bulk and single-cell transcriptomics, epigenomics, proteomics; imaging-derived features can be included as an additional view | Probabilistic factor models (MOFA+) and hierarchical VAEs (scVI) that learn joint low-dimensional representations while explicitly modelling batch effects and missing data | Provide molecular latent spaces—shared factors or scVI latent variables—that can be aligned with cardiac imaging embeddings for radiogenomic and spatial integration | Established ecosystem with documented APIs; standard entry point for cardiovascular multiomics before fusion with imaging |
| Multimodal VAEs [ 80 , 81 ] | Heterogeneous combinations of imaging, omics, and clinical data | Product-of-experts or mixture-of-experts aggregation of modality-specific encoder posteriors into a shared latent space; joint posterior remains well-defined when modalities are missing | Prototype framework for learning shared imaging–omics latent spaces; supports “virtual omics” generation from imaging inputs alone | Handle missing modalities by design; probabilistic formulation enables uncertainty quantification and cross-modal simulation |
| PORPOISE [ 87 ] | Whole-slide H&E histopathology and multi-omic profiles (RNA-seq, CNV, mutations) across 14 TCGA cancer types | Attention-based multiple instance learning (MIL) for whole-slide images (WSIs) fused with a self-normalizing network for omics via Kronecker-product gated fusion; weakly supervised with survival labels | Architecturally transferable to cardiac histology–omics; interpretable morpho-molecular prognostic markers at pan-cancer scale | First large-scale interpretable histology–genomic fusion; open-source code and interactive atlas; highlighted batch-effect risks in site-unaware splits [ 88 ] |
| HEALNet [ 82 ] | Whole-slide histopathology and multi-omic data (four TCGA cancer datasets) | Iterative cross-attention between a shared latent bottleneck and modality-specific token streams; processes raw inputs rather than precomputed embeddings | Directly applicable to cardiac tissue sections with matched omics; handles missing modalities at both training and inference | State-of-the-art survival prediction among end-to-end fusion models; open-source; modular design facilitates addition of imaging modalities |
| Cardiovascular imaging–omics studies | ||||
Table 5: Representative frameworks and studies integrating cardiovascular imaging with molecular or other high-dimensional omic data, including general-purpose frameworks that have been applied or are directly applicable to imaging-anchored multiomics and cardiovascular studies that exemplify concrete imaging–omics applications in cardiology.
Figure 3: Archetypal pipelines for imaging–omics integration. Radiogenomics workflows link imaging-derived features to bulk or single-cell molecular profiles via feature extraction, joint latent modelling, and association or causal inference; spatial molecular alignment pipelines register in vivo imaging, ex vivo imaging, histology and spatial transcriptomics into a common coordinate system; virtual transcriptomics approaches use deep models to predict molecular modules or spatial expression maps directly from imaging inputs.
Figure 4: Translational roadmap from imaging-anchored multiomics to digital twins. Top lane: model stack comprising modality-specific encoders, multimodal fusion models and transformer or foundation model architectures. Middle lane: an agentic layer in which large language models orchestrate tool calls to encoders, fusion models and knowledge bases to generate structured reports and explanations. Bottom lane: translational path from data curation and model development through external validation and deployment to decision support and digital twin–enabled in silico trials.
More in this topic
Multimodal Fusion
Multimodal fusion combines information from different data types to improve prediction, perception, and decision-making.
See all 772 papers