Millimeter-wave (mmWave) radar and RGB-D cameras can record cardiac and respiratory waveforms continuously and without contact, but labeled recordings remain scarce because every label requires a supervised acquisition session. Self-supervised pretraining can exploit the unlabeled signals, yet contrastive methods depend on signal transformations and negative pairs whose validity is uncertain for cardiorespiratory data, where time warping changes breathing rate and distant windows can share the same physiological state. We present Phenomenon-Graph JEPA, a joint-embedding predictive architecture that learns from four processed one-dimensional streams without negative pairs or synthetic augmentation in its base configuration. Each stream is encoded by a temporal convolutional branch and a band-limited spectral branch. During pretraining, the model predicts stopped target embeddings along typed edges, which connect streams assigned to the same physiological phenomenon, and forward in time within a state episode. We treat this physiological typing as a testable hypothesis and compare it with wrong-edge and all-pairs prediction graphs. In the OMuSense-23 dataset, pretraining improves label-efficiency area over matched supervised training by 3.91 percentage points (95% interval 2.08 to 5.80, Holm-adjusted p = 0.006), and by 3.74 points under a second configuration evaluated on the same test participants. However, the wrong-edge and all-pairs controls do not establish a benefit from physiological typing. Optional Takens-inspired delay coordinates improve a validation comparison with learned history, whereas two wrist-only WESAD protocols do not establish a pretraining advantage. The study therefore separates the measured benefit of predictive representations from the physiological prior used to organize their training.
Figures & tables
Figure 1: Candidate prediction graphs. Cardiac streams are orange, respiratory teal. Typed edges pair measurements of the same phenomenon, the wrong graph crosses phenomena with the same number of directed predictions, and all pairs has six links and more predictor parameters. Every link trains in both directions.
Figure 2: Phenomenon-Graph JEPA. Delay coordinates affect only the temporal input and are optional ( K=1 in main tests). During pretraining, cross-edge and within-stream future predictors match stopped targets from EMA encoders and projectors. Recognition fine-tunes online encoders and a classifier, with no updates at inference. WESAD discards chest encoders after pretraining.
Figure 3: Evaluation protocol. Participants are separated before pretraining, which uses unlabeled windows of training participants only. Nested label budgets are identical for the JEPA-initialized and randomly initialized arms, early stopping uses validation participants, and each configuration is fixed before its test run. Inference operates on participant-level areas.
Setting
Value
Temporal CNN
widths 64/128/128/128, kernels 7/5/5/5
Branch and stream widths
128 temporal, 64 spectral, 192
Loss weights
1, 0.3, 1, 0.1, 0.004
Pretraining
3,000 steps, batch 64, AdamW
Pretraining rate
10−3 , cosine, weight decay 10−4
EMA momentum
0.996
Table 1: Implementation settings shared by all main comparisons. Loss weights refer to the cross-edge, spectral, future, variance and covariance terms of Equation ( 16 ).
Comparison
Δ
95% interval
Raw p
Holm p
OMuSense-23, fold-V test, macro-F1 area, n=10
JEPA − supervised, C1
+3.91
[+2.08,+5.80]
0.0020
0.0059
JEPA − supervised, C2
+3.74
[+1.68,+5.78]
0.0117
0.0234
Typed − wrong edges, C1
−0.74
[−1.44,+0.03]
0.1016
0.2031
Typed − all pairs, C1
−0.43
[−1.65,+0.87]
0.5352
0.5352
JEPA − sup., C2, no radar
+1.12
[−0.25,+2.65]
0.1992
0.1992
Table 2: Participant-level differences in normalized log-budget area (pp). Positive values favour the first method, Holm correction applies within each declared family as described in Section 4.4 , and intervals are 95% participant bootstrap intervals.
Figure 4: (a) C1 activity macro-F1 with wrong-edge, all-pairs and matched supervised controls. Shading is the participant standard error after three-seed averaging. Prior SSL is a five-seed historical frozen-probe reimplementation [ 11 ] , not a matched control. (b) The seven declared contrasts, with participant points, mean diamonds and 95% bootstrap intervals.
Achieved labels (%)
1.11
4.44
10
50
100
C1, base configuration
Supervised
28.68
39.38
54.09
68.53
73.70
JEPA, typed graph
36.25
45.44
55.91
71.14
75.07
JEPA, wrong edges
38.52
48.72
55.36
70.11
75.15
JEPA, all pairs
36.63
47.23
54.90
72.07
74.64
C2, shift + sensor masking
Table 3: OMuSense-23 activity macro-F1 (%) on the fold-V test participants at each achieved label fraction, as means over ten participants after averaging seeds. The historical reimplementation uses five seeds and a frozen linear probe.
Method
Activity
Pose
Published, fixed split, four streams
Contrastive SSL [ 11 ]
74.00
78.00
Handcrafted + XGBoost [ 10 ]
81.00
86.00
Handcrafted + deep fusion [ 10 ]
85.00
97.00
Published, five-fold average
Handcrafted + deep fusion [ 10 ]
87.20
92.60
Table 4: Full-label OMuSense accuracy (%), four streams. Published fixed-split and five-fold results are separate aggregates, and the reimplementation differs in adaptation and seed count. Current rows are matched within C1 or C2 on fold V.
Labels
Supervised
JEPA
OMuSense-23, C2, fold-V test, three seeds
Pose, 3 classes
84.15±10.29
84.78±7.29
Activity, 4 classes
78.93±3.65
76.94±3.35
WESAD meditation vs baseline a , LOSO, two seeds
25%
62.14±13.14
58.36±13.15
100%
70.45±17.72
71.91±19.10
Table 5: Balanced accuracy (%), mean ± SD across participant seed means. Groups separate test from exploratory validation scores. Published 30 s LOSO references [ 1 ] , which add handcrafted wrist-feature families, are a 82.80 ± 6.30, b 92.43 ± 11.04 and c 73.95 ± 16.78, contextual rather than matched. Stress versus rest is stress against baseline plus amusement, three states are baseline/stress/amusement, and four states add meditation.
Within-study comparison
Budgets (%)
Δ area
Spectral weight, 1.0 vs 0.3
10/100
−1.63
Additive noise, pretraining
1/10/100
−1.60
Additive noise, downstream
1/10/100
−0.69
Bounded shift, downstream
10/100
+1.27
Cross-attention vs token MLP
1/10/100
−1.70
Attention vs MLP classifier
1/10/100
−1.43
Table 6: OMuSense validation, one seed. Area differences (pp), each against its own reference condition on the listed grid, so grids are not interchangeable. Attention controls match capacity but differ in source-time inputs. Validation estimates, not significance tests.
Input organisation
10%
100%
Area
No delay
59.45
71.91
65.68
Short delay coordinates
58.56
74.83
66.70
Short learned-history control
56.93
73.58
65.26
Long delay coordinates
58.85
72.47
65.66
Long learned-history control
58.67
72.46
65.57
Table 7: Delay-coordinate validation study on fold IV, one seed, fine-tuned activity macro-F1 (%) at 10% and 100% labels and their normalized log-budget area. Short and long lags are 2/10 and 4/20 samples for cardiac/respiratory nodes at 20 Hz.
Foundation models have recently emerged as a powerful paradigm for learning transferable representations from large scale biomedical data, yet existing approaches for physiological waveforms primarily optimize reconstruction or forecasting objectives that do not explicitly preserve clinically meaningful waveform morphology. Electrocardiograms (ECGs) and pulse oximetry (SpO2) waveforms encode rich cardiovascular and hemodynamic information through their morphological structure. In this work, we introduce MorphologyFM, a multimodal foundation model pretrained on paired ECG and SpO2 waveforms from the MIMIC critical care database using a morphology aware self supervised learning objective. MorphologyFM combines morphology guided masking, cross modal representation learning, and contrastive latent alignment to learn representations that capture clinically relevant physiological structure without requiring manual annotations. We evaluate MorphologyFM across multiple downstream prediction tasks, including arrhythmia classification, hypoxemia prediction, mortality prediction, and length of stay estimation, demonstrating consistent improvements over representative self supervised learning methods, including Masked Autoencoders (MAE), contrastive learning, Barlow Twins, and Joint Embedding Predictive Architectures (JEPA). Furthermore, we show that jointly modeling ECG and SpO2 waveforms produces more transferable representations than single modality pretraining. Our results establish waveform morphology as a powerful inductive bias for self supervised physiological representation learning and introduce MorphologyFM as a general purpose foundation model for continuous physiological monitoring.
Saiyang Feng, Yuanyun Zhang, Shi Li
University of the Chinese Academy of Sciences · Columbia University
The cyclic structure of physiological processes offers a natural prior for self-supervised representation learning, and the cardiac cycle provides a particularly well-defined setting in which to exploit it. We derive a phase-equivariant self-supervised objective and introduce Winder, a joint-embedding architecture that organises representations into phase-invariant coordinates and phase-rotating harmonic subspaces. Its transport operator is fixed and closed-form, derived from the cycle's geometry rather than learned, and adds no parameters. Evaluated on PTB-XL under a frozen linear-probe protocol, Winder attains diagnostic accuracy within the range reported by state-of-the-art self-supervised methods at a ~1 M parameter footprint, while exhibiting phase-equivariant latent geometry. These findings demonstrate that explicitly encoding cardiac-phase symmetry can preserve diagnostically useful information while yielding a latent geometry that is legible, parameter-efficient, and directly tied to a measurable physiological quantity.
Blaise Delaney, Dominic Dootson, Juan Jose Juan Castella +5
TimeTrace Labs · University of Cambridge · University of Oxford +1
Electrocardiography (ECG), photoplethysmography (PPG), and phonocardiography (PCG) provide complementary views of the same cardiac cycle, yet existing cardiac foundation models are trained for a single sensing modality, leaving the shared physiology across sensors unexploited. We introduce CardioState-JEPA, a cardiac foundation model to learn a single shared representation jointly across ECG, PPG, and PCG, built on a physiology-aware joint-embedding predictive architecture. The model maps heterogeneous waveforms into a common token space, processes them with a single shared Transformer encoder, and learns by predicting masked latent cardiac states, placing the pretraining target on shared physiology rather than sensor-specific waveform appearance. To handle the temporal offsets between electrical, mechanical, and hemodynamic events, cross-modal prediction uses a learned delay aligner that matches signals at the corresponding cardiac time. Because synchronized multi-sensor recordings are scarce, CardioState-JEPA first learns within-modality structure from abundant unimodal data and then uses paired data to align modalities in latent cardiac time. Evaluated as a frozen encoder across 25 downstream tasks spanning ECG, PPG, and PCG, our encoder improves average PPG classification by 8.2 AUROC points, PCG murmur detection by 18.8 AUROC points, and ECG classification by 15.5 AUROC points over the best self-supervised signal baseline and matches or exceeds cardiac models trained with privileged clinical text or supervised labels on several ECG benchmarks. These results establish that heterogeneous cardiac signals can mutually supervise a single foundation model of cardiac physiology.
Hamza Shafiq, Hung Manh Pham, Bin Zhu +3
Eindhoven University of Technology, Netherlands · Singapore Management University, Singapore