Training and Scaling Compute-Optimal Physiological Waveform Foundation Models

Pingzhi Li; Jie Peng; Shuqing Luo; Zachary Plotkin; Tianlong Chen

cs.LGOct 5, 2026

Training and Scaling Compute-Optimal Physiological Waveform Foundation Models

Authors: Pingzhi Li, Jie Peng, Shuqing Luo, Zachary Plotkin, Tianlong Chen

Organizations: University of North Carolina at Chapel Hill · Foci Labs

Abstract

We investigate the scaling laws and compute-optimal training of physiological waveform foundation models (FMs). We train Aether, a family of over one hundred FMs ranging from 20M to 2.1B parameters, on up to 36.3M hours of physiological waveforms. We construct eight clinical prediction tasks from MIMIC-III and evaluate the FMs through linear probing. The 720M FM outperforms all existing baseline FMs across all eight tasks. A scaling law of model size, pretraining hours, and labeled patients predicts downstream ranking error, i.e. $1-\mathrm{AUROC}$ , effectively with $0.5\%$ prediction MAE at held-out resource scales and $0.9\%$ MAE when extrapolating to 2.1B parameters. We present three findings: (1) Compute-optimal training scales both FM size and pretraining hours. Under the fitted law, a $10.0\times$ increase in compute FLOPs scales model size by $1.2\times$ and pretraining hours by $8.2\times$ . (2) Larger FMs use waveform data more efficiently, and greater pretraining exposure increases the benefit of model scaling. Starting from 25M parameters and 4.8M pretraining hours, doubling FM size reduces the predicted hours needed for the same performance by $51.8\%$ . (3) Pretraining and clinical supervision reinforce each other: more labeled patients increase the return to pretraining, while larger FMs and longer pretraining reduce labeling requirements. For the example of the 720M FM, extending pretraining from 120K to 36.3M hours reduces the predicted patient requirement by $61\%$ at a target ranking error. These findings provide a quantitative training recipe and a promising and durable scaling path for physiological waveform modeling and downstream clinical prediction.

Figures & tables

Figure 1: Scaling clinical prediction and label efficiency. (a) Measured eight-task mean ranking error against

S=P^{g}/(AN^{-a}+BD^{-b})

. (b) Predicted labeled patient requirements at

\mathcal{R}=0.32

. Extending 720M pretraining from 120K to 36.3M hours reduces required patients by

61\%

Figure 2: Overview of Aether architecture, pretraining, and downstream adaptation. (a) Bidirectional Gated DeltaNet (Bi-GDN) and grouped-query attention (GQA) blocks form a repeating $3{:}1$ hybrid pattern. (b) Multichannel waveforms are divided into 200 ms patches, and the FM learns to reconstruct randomly masked patches. (c) The pretrained FM is frozen, and its mean-pooled representations support linear probing for downstream clinical prediction.

Figure 3: Comparison with existing waveform FMs across eight clinical prediction tasks. Each panel shows macro AUROC against FM size using 100% of labeled patients. To ensure a fair comparison, we report each model’s best linear-probe AUROC across the same regularization sweep,

\lambda\in\{0.001,0.01,0.1,1,10\}

. Higher AUROC corresponds to lower ranking error.

Figure 4: Scaling of downstream average ranking error

\mathcal{R}

across eight tasks. (a) Fitted response to the resource score

S=P^{g}/(AN^{-a}+BD^{-b})

. (b) Pretraining scaling at

P=1{,}222

. (c) Patient scaling at 4.8M pretraining hours. (d) Predictions at held-out resource levels: we hold out all configurations at one value of

N

D

, or

P

, fit on the rest, and predict the held-out configurations. MAE averages absolute prediction errors equally across the three resource axes.

Figure 5: Scaling and prediction at larger resources. (a) Pretraining scaling at 2.1B for all four values of

P

. (b) Patient scaling at 2.1B for all five values of

D

. (c) Predictions for all 20 configurations at 2.1B. (d) Predictions for 20M, 60M, and 180M after 36.3M, 17.0M, and 5.9M pretraining waveform hours, respectively. Ranking errors and absolute patient counts are averaged across the eight tasks. Solid lines connect measurements in (a,b); dashed diagonals in (c,d) mark

\widehat{\mathcal{R}}=\mathcal{R}

Figure 6: Iso-error contours under a FLOPs constraint, from Equation 5.1 . (a–d) $P=408$ , $1{,}222$ , $2{,}443$ , and $4{,}071$ , respectively. Dashed lines mark iso-FLOPs slices; red curves show the efficient frontier over the observed FM-size range. Each panel shows all 57 configurations. Ranking errors and patient counts are eight-task means.

Figure 7: Iso-error contours for FM size and pretraining waveform hours. Each curve holds the predicted eight-task average ranking error fixed under Equation 4.1 . (a–d) $P=408$ , $1{,}222$ , $2{,}443$ , and $4{,}071$ , respectively. Points show measurements, with color indicating observed ranking error. Dashed lines mark iso-parameters slices. Patient counts are averaged across the eight tasks. Moving right along a curve reduces waveform hours at fixed ranking error $\mathcal{R}$ .

Figure 8: Iso-ranking-error curves showing predicted labeling requirements averaged across eight tasks. Target ranking error

\mathcal{R}_{\mathrm{t}}

is 0.32 in (a), 0.37 in (b).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Task	Prediction time	$W$	$G$	$H$	Targets	Eligibility at $T$
Acute deterioration	Sliding	6 h	0	6 h	3	Eligible for at least one component
Circulatory deterioration	Sliding	6 h	0	6 h	1	No active vasopressors
Dynamic mortality	Sliding	6 h	6 h	24 h	1	All monitored patients
Respiratory escalation	Sliding	6 h	0	6 h	1	No invasive ventilation
Sepsis onset	Sliding	6 h	1 h	5 h	1	No prior Sepsis-3 onset
Fluid response	Infusion start	6 h	30 min	30 min	2	Documented rapid crystalloid course

Appendix

Table 4: Prediction windows and eligibility for the eight downstream tasks.

W

denotes clinical history,

G

the forecast gap, and

H

the outcome horizon. Each FM input contains one hour of waveforms ending at

T

. “Targets” gives the number of binary outcomes evaluated per task.

Explore similar work

A Scaling Study for fMRI Foundation Models
Sep 23, 2026Wenhao Ye, Xuanye Pan, Junfeng Xia +3Model Size Functional Magnetic Resonance Imaging
Southern University of Science and Technology · Shenzhen University · SLAI +1
Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study
May 12, 2026M A Al-Masud, Nils StrodthoffElectrocardiogram Foundation Models Electrocardiography
AI4Health Division Carl von Ossietzky Universität Oldenburg Oldenburg, Lower Saxony, Germany
A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency
Apr 24, 2026Nanae Aratake, Taisei Tosaki, Yuji Okamoto +5Model Size Medical World Model
1Graduate School of Medicine, Kyoto University, Kyoto, Japan. · 2Medical Data Vision Co., Ltd., Tokyo, Japan. · 3IQVIA Solutions Japan G.K., Tokyo, Japan. +1

Sep 23, 2026, cs.LGJ/K move · Enter open · S save

A Scaling Study for fMRI Foundation Models

Wenhao Ye, Xuanye Pan, Junfeng Xia, Junxiang Zhang+2

Southern University of Science and Technology · Shenzhen University · SLAI+1

Model Size Functional Magnetic Resonance Imaging Pretraining Foundation Model Scaling Computer Vision

Open paper AskPDF arXiv

Scaling laws have guided large-model development in computer vision and natural language processing, but the relationships among data, model size, and compute remain unclear for functional magnetic resonance imaging (fMRI) foundation models. Here, we conduct a controlled empirical study using pretraining data from more than 200 source datasets and over 10,000 GPU-hours of experiments. Holding the pretraining framework and downstream protocol fixed, we vary pretraining data size, model size, and training duration. Downstream performance generally improves with compute, yet models using similar compute can perform substantially differently. Additional pretraining data bring larger gains at larger model sizes, suggesting that data and model size should be scaled together. At matched compute, increasing pretraining data benefits more tasks than increasing model size, although the pattern varies across tasks. We then use in-distribution (ID) downstream performance to select the combination of pretraining data size, model size, and training duration at two fixed compute budgets. The resulting models are locked before out-of-distribution (OOD) evaluation. They achieve the highest average performance across the evaluated OOD tasks among the compared fMRI foundation models while using less pretraining compute. Overall, our results show that compute alone does not characterize fMRI scaling: performance depends on how pretraining data, model size, and training duration are combined.

Abstract

Explore similar work

A Scaling Study for fMRI Foundation Models

Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study

A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency