cs.LGMar 27, 2026

Foundation Model for Cardiac Time Series via Masked Latent Attention

Authors: Moritz Vandenhirtz, Samuel Ruipérez-Campillo, Simon Böhi, Sonia Laguna, Irene Cannistraci, Andrea Agostini, Ece Ozkan, Thomas M. Sutter, +1 more

Organizations: Department of Computer Science, ETH Zurich, Switzerland · Department of Biomedical Engineering, University of Basel, Switzerland

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy. We introduce the latent attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining. Our approach models higher-order interactions across leads through latent attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations. We provide empirical evidence on the Mimic-IV-ECG database that leveraging the cross-lead connection constitutes an effective form of structural supervision, improving representation quality and transferability. Our method shows strong performance in predicting ICD-10 codes, outperforming independent-lead masked modeling and alignment-based baselines.

Figures & tables

Figure 1: Framework overview. (left) Each ECG is separated into 1212 leads, which are encoded separately and subsequently processed jointly through a latent attention transformer. The training objective is masked reconstruction. (right) The predictions are based on the latent attention’s CLS token.
Table 1: ICD-10 code prediction performance by hierarchical group under linear probing of the corresponding backbones from the studied models. Best results in bold and second best in italics .
Figure 2: Label efficiency under finetuning. Performance curves of the macro-averaged AUROC over all 228228 Chapter IX codes, as a function of the number of training studies used for finetuning.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Table 2: Clinical meaning of the ICD-10 Chapter IX codes reported in tables 3 and 1 , shown with the same hierarchy.
Table 3: Extended ICD-10 code prediction performance by hierarchical group in the models studied in table 1 . AUROC is reported for linear probing on each corresponding backbone.
Table 4: Extended ICD-10 code prediction performance by hierarchical group. Comparison of proposed LAMAE and LAMAE E under linear probing and fine-tuning of the corresponding pretrained backbone.

Explore similar work

CardsList

  1. Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning

    Sep 10, 2026Andrea Agostini, Simon Böhi, Moritz Vandenhirtz +7Masked AutoencodersHealthcare

    Department of Computer Science, ETH Zurich, Switzerland · Department of Biomedical Engineering, University of Basel, Switzerland

  2. A Hybrid CNN--State-Space--Attention Backbone with Joint-Embedding Predictive Pretraining for 12-Lead ECG Classification

    Sep 24, 2026Yakoub Bazi, Sarah Aljuhani, Mohamad M. Al Rahhal +2ElectrocardiographySelf-Supervised Time Series Representation Learning

    Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh, 11543, Saudi Arabia · Applied Computer Science Department, College of Applied Computer Science, King Saud University, Riyadh, 11543, Saudi Arabia

  3. Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study

    May 12, 2026M A Al-Masud, Nils StrodthoffElectrocardiographySelf-Supervised Time Series Representation Learning

    AI4Health Division Carl von Ossietzky Universität Oldenburg Oldenburg, Lower Saxony, Germany