What masking geometry works best for EEG foundation models?

Pierre Guetschel; Bruno Aristimunha; Yassine El Ouahidi; Arnaud Delorme; Thomas Moreau; Michael Tangermann

cs.LGSep 27, 2026

What masking geometry works best for EEG foundation models?

Authors: Pierre Guetschel, Bruno Aristimunha, Yassine El Ouahidi, Arnaud Delorme, Thomas Moreau, Michael Tangermann

Organizations: Donders Institute for Brain, Cognition and Behaviour Radboud University Nijmegen, The Netherlands · Yneuro, University of California San Diego, Paris, France · Lab-STICC, IMT Atlantique Brest, France · SCCN, INC, SDSC University of California San Diego, USA CNRS, France · Université Paris-Saclay, Inria, CEA Palaiseau, France

Abstract

EEG foundation models hold promise for scalable brain-signal decoding across clinical and cognitive neuroscience applications, yet their pre-training pipelines remain poorly understood. Among design choices, the masking strategy is particularly critical: it determines what the network must predict and from which context. Yet it has never been ablated in isolation, as each new model bundles a new masking strategy with a new backbone and objective. In this paper, we formalize the design choices for spatio-temporal masking strategies and train various models with a single pipeline under varying masking configurations across two SSL frameworks (MAE and JEPA). We then systematically evaluate the resulting 58 pre-trained models on the 12 datasets of OpenEEGBench under a linear probe. Both frameworks agree on an optimal masking configuration and on shared failure modes. Outside these, performance is robust: 11 MAE and 9 JEPA configurations are statistically indistinguishable from the best. We further identify a novel JEPA-specific failure mode, tagged bias-inflation collapse, invisible to standard detectors. With a well-chosen mask, our pipeline reaches REVE-level downstream performance at a fraction of REVE's pre-training compute.

Figures & tables

Figure 1: Controlled cross of two design axes for EEG masked-prediction pre-training. (A) Framework axis. The MAE and JEPA pretexts are superimposed on a shared pipeline; components outside the coloured dashed boxes (linear tokeniser, masking, encoder, decoder) are shared between the two, while each pretext adds its own branch ( green: MAE input-space reconstruction ; orange: JEPA latent prediction against an EMA teacher ). (B) Masking axis. Block-masks parameterised by spatial radius $r$ , temporal length $L$ , and target ratio $\rho$ . This formalism unifies the masking strategies used in the EEG self-supervised literature, recovering four canonical regimes at extreme $(L,r)$ values: random patches, pure temporal blocks, pure spatial blocks, and spatio-temporal blocks. We sweep $29$ $(L,r)$ configurations at fixed $\rho=0.55$ . Holding every other component fixed (backbone, optimiser, corpus, tokenisation, downstream protocol) yields the $58=2\times 29$ pre-trained foundation models that we evaluate on OpenEEGBench under a linear probe on frozen features.

Figure 2: Mean normalised downstream performance on OpenEEGBench across the

12

datasets and

5

probe seeds, as a function of the pre-training epoch, for both frameworks at the optimal mask

(L,r)=(2\,\text{s},9\,\text{cm})

. Shaded bands are the

95\%

confidence interval across seeds and datasets. The dashed black line locates the REVE baseline; horizontal dashed grey lines locate the three supervised baselines (EEGNet, EEGConformer, ShallowFBCSPNet) trained end-to-end per task. Both frameworks reach the REVE baseline level by epoch

\sim\!4

and remain at or above it.

Figure 3: Effect of the masking geometry $(L,r)$ on downstream performances. (A) Mean normalised score above the REVE baseline ( $0=$ the public REVE-Base checkpoint) across the $12$ OpenEEGBench datasets $\times$ $5$ probe seeds. (B) Mean rank ( $1=$ best) per cell under the hierarchical bootstrap ( $T=10{,}000$ replicates over datasets and seeds). The bold cell in each framework is the bootstrap winner; white circles mark cells not statistically distinguishable from it at $\mathbb{P}(\text{winner}\!\succ\!c)\leq 0.975$ (ex-aequo cluster). (C) Training trajectories: score of (A) as a function of the pre-training epoch, faceted by spatial radius $r$ , for both frameworks and all six values of $L$ . Dashed lines locate the REVE baseline (black) and the three supervised baselines (grey). All foundation-model curves use a frozen-feature ridge probe; supervised baselines are trained end-to-end per task.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Component	# params
Linear patch embedding ( $200\!\to\!512$ , with bias)	102,912
Encoder layer (single, $\times 4$ ):	3,146,240
Self-attention $W_{QKV}$ , $W_{O}$	1,048,576
GEGLU FFN (gate + value, output projections)	2,096,640
$2\times$ RMSNorm scales	1,024
Encoder transformer total (4 layers)	12,584,960

Appendix

Table 1: Per-component parameter count of the REVE-Small backbone, derived from the configuration files. “

\dagger

” marks components that exist only for one of the two frameworks; “

\ddagger

” marks the JEPA-only EMA teacher, which is held in memory but not optimised. Numbers assume bias=False on every linear layer and the GEGLU expansion described in the prose.

Hyperparameter	Value
Optimizer	AdamW
$(\beta_{1},\beta_{2})$	$(0.9,\,0.999)$ (PyTorch default)
weight decay	$0.01$
Peak learning rate	$2.4\!\times\!10^{-4}$
Final learning rate $\eta_{\text{min}}$	$10^{-6}$
Warmup schedule	linear, factor $10^{-6}\!\to\!1$

Appendix

Table 2: Training hyperparameters shared by all

58

pre-training configurations of the core grid.

Framework	$n$	Mean	Median	SD	Min–Max
JEPA	$29$	$14.91$	$15.28$	$0.78$	$14.00$ – $16.29$
MAE	$29$	$12.70$	$12.65$	$0.25$	$12.62$ – $13.98$
Overall	$58$	$13.81$	$13.99$	$1.26$	$12.62$ – $16.29$

Appendix

Table 3: Pre-training cost of the

58

grid runs, in H100-hours (two GPUs per run;

n

runs per group).

	Temporal block length $L$ (patches)
Spatial radius $r$	$1$	$2$	$4$	$8$	$16$	$33$
$0.0$ ( $r=$ "one" )	843	422	211	106	53	26
$0.06\,\text{m}$	220	110	55	28	14	7
$0.09\,\text{m}$	98	49	24	12	6	3
$0.12\,\text{m}$	55	28	14	7	4	2
$\infty$ ( $r=$ "all" )	26	13	7	3	2	$\boldsymbol{-}$

Appendix

Table 4: Number of blocks

K

sampled for an input

x

, for each combination radius-length

(r,L)

, derived from the inclusion-exclusion formula ( 2 ) with the mask ratio

\rho^{\star}=0.55

		Temporal block length $L$ (patches)
Spatial radius $r$	metric	$1$	$2$	$4$	$8$	$16$	$33$
$0.0$ ( $r=$ "one" )	ch / sphere	$1.00\pm 0.00$	$1.00\pm 0.00$	$1.00\pm 0.00$	$1.00\pm 0.00$	$1.00\pm 0.00$	$1.00\pm 0.00$
$0.0$ ( $r=$ "one" )	tot %	$55.0\pm 1.1$	$54.8\pm 1.3$	$54.2\pm 1.8$	$53.1\pm 2.5$	$50.9\pm 3.4$	$55.8\pm 6.3$
$0.06\,\text{m}$	ch / sphere	$3.08\pm 1.60$	$3.08\pm 1.60$	$3.09\pm 1.61$	$3.08\pm 1.60$	$3.09\pm 1.60$	$3.09\pm 1.60$
$0.06\,\text{m}$	tot %	$56.2\pm 4.1$	$56.0\pm 4.2$	$55.5\pm 4.5$	$54.7\pm 5.1$	$53.0\pm 5.8$	$60.3\pm 10.0$
$0.09\,\text{m}$	ch / sphere	$6.09\pm 2.54$	$6.08\pm 2.52$	$6.10\pm 2.54$	$6.10\pm 2.54$	$6.09\pm 2.53$	$6.10\pm 2.54$

Appendix

Table 5: Empirical per-block channel coverage ( ch/sphere ) and realised total mask fraction ( total mask ) for every

(L,r)

cell, with

K

as in Table 4 ; mean

\pm

std over

1{,}000\times 64=64{,}000

samples. Target mask fraction is

\rho^{\star}=55\,\%

Figure 4: Distance from each masked token to its nearest unmasked token, empirical (blue) vs. single-block infinite-density theoretical (orange). Top row: temporal distance (in patches) on the same channel, sweeping

L

r=\texttt{"one"}

. Bottom row: spatial distance (in centimeters) at the same time, sweeping

r

L=1

. Empirical distributions are aggregated over

1{,}000

batches of

64

samples drawn from the REVE channel layouts.

Dataset	$n_{\mathrm{windows}}$	classes	Distribution (%)
arithmetic_zyma2019	$1{,}707$	$2$	$25.3\,/\,74.7$
bcic2020-3	$6{,}000$	$5$	$20\,/\,20\,/\,20\,/\,20\,/\,20$
bcic2a	$5{,}184$	$4$	$25\,/\,25\,/\,25\,/\,25$
chbmit	$37{,}046$	$2$	$\mathbf{2.5\,/\,97.5}$
faced	$19{,}217$	$8$	$13\,/\,14\,/\,13\,/\,12\,/\,\text{---}\,/\,12\,/\,17\,/\,7.7\,/\,12$
isruc-sleep	$98{,}480$	$5$	$21\,/\,14\,/\,33\,/\,19\,/\,14$

Appendix

Table 6: Per-class window counts and balance for each of the 12 OpenEEGBench downstream datasets, computed from the per-recording metadata of the Hugging Face Zarr mirror.

Dataset	$n_{\mathrm{chans}}$	$n_{\mathrm{times}}$	$n_{\mathrm{patches}}$	$D_{\mathrm{orig}}$	projected?
arithmetic_zyma2019	$19$	$1{,}000$	$5$	$48{,}640$	yes
bcic2a	$22$	$800$	$4$	$45{,}056$	yes
bcic2020-3	$64$	$600$	$3$	$98{,}304$	yes
physionet	$64$	$600$	$3$	$98{,}304$	yes
chbmit	$17$	$2{,}000$	$11$	$95{,}744$	yes
faced	$26$	$2{,}000$	$11$	$146{,}432$	yes

Appendix

Table 7: Per-dataset feature dimension

D_{\mathrm{orig}}

before the Gaussian random projection, computed from Eq. ( 3 ). Columns

n_{\mathrm{chans}}

and

n_{\mathrm{times}}

are read from the Hugging Face Zarr metadata of each dataset;

n_{\mathrm{patches}}

follows from the shared tokenisation. The last column flags whether the projection step activates (always true here, since

D_{\mathrm{orig}}>5{,}000

on every cell).

	min	max
Dataset
arithmetic_zyma2019	0.595	0.778
bcic2020-3	0.231	0.294
bcic2a	0.263	0.475
chbmit	0.771	0.922
faced	0.158	0.332

Appendix

Table 8: Per-dataset normalisation constants

s^{d}_{\min}

and

s^{d}_{\max}

used in Equation 4 . They are the minimum and maximum, across the

58

v9 sweep configurations, of the seed-averaged score on each dataset (balanced accuracy or

R^{2}

). The negative values for seed-vig are an instance of the negative-

R^{2}

phenomenon discussed in subsection C.5 .

Figure 5: Per-window PERCLOS distribution on the OpenEEGBench seed-vig split (subject-based: train

{=}\{1,2,3,10,\ldots,21\}

, val

{=}\{4,5\}

, test

{=}\{6,7,8,9\}

). Top: Kernel-density estimate per split; the dashed verticals mark the per-split means. Bottom: matching horizontal box plots (medians, IQR, whiskers at

1.5\times

IQR; outliers hidden). The three splits differ both in location (means drift monotonically from train to test) and in shape: train and val are broad and have a heavy right tail near

\mathrm{PERCLOS}\!\approx\!1

, whereas test is unimodal, sharply concentrated around the central range, and lacks the high-PERCLOS tail entirely. This is what the closed-form ridge probe cannot recover from.

Split	$n_{\mathrm{windows}}$	mean	std	range
train	$13{,}275$	$0.498$	$0.283$	$[0.022,\,1.000]$
val	$3{,}540$	$0.433$	$0.269$	$[0.044,\,1.000]$
test	$3{,}540$	$0.385$	$0.175$	$[0.050,\,1.000]$

Appendix

Table 9: Per-window PERCLOS statistics for the OpenEEGBench seed-vig split. The four test subjects happen to be both less drowsy on average and more uniform than the training subjects, which creates the negative-

R^{2}

floor described in the text.

Predictor	test $R^{2}$
DummyRegressor (predicts $\bar{y}_{\mathrm{train}}=0.498$ )	$-0.413$
REVE baseline, ridge probe (mean over 5 seeds)	$-0.355\,\pm\,0.016$
Gap above the dummy floor	$+0.058$

Appendix

Table 10: Test

R^{2}

on seed-vig for the train-mean dummy predictor and for the frozen REVE baseline under the OpenEEGBench ridge probe (5 random-projection seeds). The dummy score is the structural floor implied by the subject split visible in Figure 5 ; the ridge probe sits above it, indicating that the frozen features carry information about PERCLOS even though the metric stays negative.

Figure 6: Squared Pearson correlation $R^{2}$ of the broadband EEG on a random subsample of REVE windows. Left: within-channel autocorrelation as a function of the time-lag (grey lines: per-window curves; coloured line: average across windows). Right: between-channel correlation as a function of the Euclidean inter-electrode distance (grey points: per-pair $R^{2}$ ; coloured line: distance-binned average). Temporal redundancy decays much faster than spatial redundancy.

Figure 7: Bias-inflation collapse signature on OEB probe data, cross-window metrics computed on a single subject of physionet (top) and bcic2a (bottom) processed through each frozen checkpoint at every pre-training epoch. Only JEPA at

r=\texttt{"all"}

shows the simultaneous bias inflation (

\|\mu\|\uparrow

), residual collapse (

\sigma_{\text{token}}\downarrow

), ratio drop and direction collapse (

\overline{\cos_{\text{intra}}}\!\to\!1

) predicted by the lookup-table mechanism ( subsection E.4 ). Healthy JEPA (

r=6\,

cm) and the MAE control (

r=\texttt{"all"}

) remain stable on all four metrics.

Figure 8: Phase transition in the spatial radius

r

L=2

and the final epoch. The ratio

\sigma_{\text{token}}/\sigma_{\text{chan}}

stays clustered around

0.4

across the four finite-radius cells and drops by an order of magnitude at

r=\texttt{"all"}

on both physionet and bcic2a — a sharp transition, not a smooth trend.

Figure 9: Predictivity of

\log_{10}(\sigma_{\text{token}}/\sigma_{\text{chan}})

for the downstream OEB score at the final epoch across the

10

(framework,

r

L

) configurations. Each point is one frozen checkpoint at epoch

10

. Markers: framework (

\circ

JEPA,

\square

MAE). Colours: spatial radius (light

\to

dark for small

\to

large

r

). Pearson

r=+0.95

on physionet and

+0.91

on bcic2a.

$\delta$ band
Band / dataset	MAE $(9\,\text{cm},L{=}2)$	JEPA $(9\,\text{cm},L{=}2)$	JEPA $(\texttt{"all"},L{=}2)$ , collapsed	Untrained
arithmetic_zyma2019	$0.72$	$0.50$	$-0.02$	$0.49$
bcic2020-3	$0.62$	$0.40$	$0.11$	$0.30$
bcic2a	$0.70$	$0.47$	$-0.08$	$0.29$
chbmit	$0.84$	$0.75$	$0.04$	$0.67$
faced	$0.67$	$0.51$	$0.03$	$0.42$

Appendix

Table 11: Band-power decodability from frozen final-epoch features (

R^{2}

, mean over channels), per frequency band and per OpenEEGBench dataset, for the two optima of the core grid, the collapsed JEPA cell and an untrained encoder.

Figure 10: Candidate within-window proxies

\sigma_{\text{time}}/\sigma_{\text{chan}}

(left) and

\operatorname{eff\_rank}

(right) on real REVE pre-training windows, at epochs

1,4,7,10

for the three core runs. Lines: mean across the

32

windows of the evaluation shard; bands:

25

–

75

percentile. By epoch

4

the BIC run is still at its epoch-

1

values on both metrics, while healthy and MAE have already taken off — but neither metric discriminates the runs at initialisation, and both can plausibly look low for legitimate reasons in early training.

Figure 11: Per-dataset, per-(framework, $L$ , $r$ ) score difference to the REVE ridge-probe baseline on OpenEEGBench. One row per dataset, one column per framework; each row uses its own diverging colour-bar so the diffs are interpretable in the dataset-native metric (balanced accuracy or $R^{2}$ ). Sidebars on the right of each row mark the absolute REVE level, the chance level, and the three supervised baselines (EEGNet, EEGConformer, ShallowFBCSPNet) trained end-to-end per task. The bottom row Average (norm.) reproduces Figure 3 (A) on the cross-dataset normalised scale. The trends summarised in Section 4 are visible dataset-by-dataset: the moderate- $r$ , short- $L$ cluster wins on most datasets, $r=\texttt{"all"}$ underperforms for JEPA on most datasets (with the exceptions of arithmetic_zyma2019 and seed-v ), and $(L,r)=(2,9\,\text{cm})$ is the aggregate JEPA optimum, although the local per-dataset peak sometimes lies elsewhere.

Figure 12: Per-dataset training trajectory of the downstream score across the

10

pre-training epochs. Each line is one of the

58

pre-training configurations (hue: framework), averaged across

3

probe seeds. Per-dataset chance (dotted grey), the REVE baseline mean

\pm

std (dashed black), and the three supervised baselines (EEGNet, EEG Conformer, ShallowFBCSPNet, dashed coloured) are overlaid for reference; the supervised baselines are trained end-to-end per task, while all FM curves use a frozen-feature ridge probe. Across datasets, both frameworks reach a near-final ranking by epoch

\sim\!5

; pathological JEPA runs at

r=\texttt{"all"}

are visible as the bundle that drops below the REVE band early and never recovers.

	Ours $(2\,\text{s},9\,\text{cm})$ , ridge		Public, ridge		Public, SGD (published)
Dataset	MAE	JEPA	REVE-Base	CBraMod	EEGPT	LaBraM	BIOT
arithmetic_zyma2019	$0.716$	$\mathbf{0.737}$	$0.614$	$0.562$	$0.551$	$0.499$	$0.553$
bcic2020-3	$0.273$	$0.277$	$\mathbf{0.281}$	$0.233$	$0.207$	$0.200$	$0.216$
bcic2a	$\mathbf{0.462}$	$0.460$	$0.436$	$0.315$	$0.296$	$0.258$	$0.264$
chbmit	$0.888$	$0.908$	$0.882$	$\mathbf{0.941}$	—	—	—
faced	$\mathbf{0.321}$	$0.280$	$0.281$	$0.212$	$0.174$	$0.125$	$0.155$

Appendix

Table 12: Downstream scores of the recommended geometry against public EEG foundation models on the

12

OpenEEGBench datasets (balanced accuracy;

R^{2}

for seed-vig ). Ridge: the analytical linear probe of this paper on fully frozen backbones. SGD: the linear probe of the original OpenEEGBench paper, whose published scores are reported (dashes: datasets it did not cover). Best score per row in bold.

Backbone	Published mask	Recommended geometry	Recommended better on
REVE-Small (core grid)	$0.710$	$0.800$	$9$ of $12$
CBraMod	$0.597$	$0.640$	$10$ of $12$
LaBraM	$0.605$	$0.642$	$7$ of $12$

Appendix

Table 13: Mean normalised OpenEEGBench score (

12

datasets

\times

5

probe seeds) of the CBraMod and LaBraM architectures re-trained under MAE on our corpus, with their published mask

(r=\texttt{"one"},L=1)

and with the recommended geometry

(r=9\,\text{cm},L=2)

, both at

\rho=0.5

. The REVE-Small row is the same contrast in the core grid, at

\rho^{\star}=0.55

Framework	Geometry $(r,L)$	$\rho=0.25$	$0.40$	$0.55$ (grid)	$0.70$	$0.85$
MAE	$(\texttt{"one"},1)$	$0.59$	$0.66$	$0.71$	$0.83$	$0.78$
MAE	$(9\,\text{cm},2)$	$0.74$	$0.82$	$0.80$	$0.81$	$0.78$
MAE	$(12\,\text{cm},16)$	—	$0.57$	$0.63$	$0.59$	$0.63$
MAE	$(\texttt{"all"},2)$	$0.61$	$0.65$	$0.65$	$0.66$	$0.56$
JEPA	$(\texttt{"one"},1)$	$0.59$	$0.64$	$0.70$	$0.72$	$0.72$
JEPA	$(9\,\text{cm},2)$	$0.79$	$0.86$	$0.83$	$0.77$	$0.73$

Appendix

Table 14: Mean normalised OpenEEGBench score (

12

datasets

\times

5

probe seeds) as a function of the target mask ratio

\rho

, for four geometries under both frameworks. The

\rho=0.55

column is the core grid.

Run	Patch	Framework	Token dim.	Mask $(r,L)$	$\sigma_{\text{token}}/\sigma_{\text{chan}}$	$\overline{\cos_{\text{intra}}}$	$\\|\mu\\|$ , ep. $1\!\to\!10$	Params enc. / tot.	GPU-h	OEB score
(A) same model size	$0.5$ s	JEPA	$512$	$(\texttt{"all"},1)$	$0.18$ / $0.17$	$0.98$ / $0.98$	$\times 2.0$ / $\times 2.2$	$12.6$ M / $21.3$ M	$39.5$	$0.598$
(B) same features per second	$0.5$ s	JEPA	$256$	$(\texttt{"all"},1)$	$0.27$ / $0.24$	$0.94$ / $0.95$	$\times 2.1$ / $\times 2.2$	$3.2$ M / $5.3$ M	$30.6$	$0.417$
Core grid, collapsed	$1.0$ s	JEPA	$512$	$(\texttt{"all"},1)$	$0.05$ / $0.08$	$1.00$ / $1.00$	$\times 2.0$ / $\times 2.0$	$12.7$ M / $21.3$ M	$16.3$	$0.320$
Core grid, collapsed	$1.0$ s	JEPA	$512$	$(\texttt{"all"},2)$	$0.02$ / $0.05$	$1.00$ / $1.00$	$\times 2.4$ / $\times 2.5$	$12.7$ M / $21.3$ M	$14.0$	$0.054$
Core grid, healthy	$1.0$ s	JEPA	$512$	$(6\,\text{cm},2)$	$0.47$ / $0.52$	$0.87$ / $0.89$	$\times 1.4$ / $\times 1.5$	$12.7$ M / $21.3$ M	$14.0$	$0.757$
Core grid, MAE control	$1.0$ s	MAE	$512$	$(\texttt{"all"},2)$	$0.83$ / $0.95$	$0.71$ / $0.72$	$\times 1.0$ / $\times 1.0$	$12.7$ M / $21.2$ M	$12.7$	$0.652$

Appendix

Table 15: Patch-length ablation: diagnostics of subsection E.2 at the final epoch, inflation of

\|\mu\|

between epochs

1

and

10

, parameters of the encoder / of the whole model, pre-training cost, and mean normalised OpenEEGBench score. Reference rows are cells of the core grid.

Explore similar work

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models
Jul 31, 2026Jinhao Li, Zhiyuan Ma, Xueqiao Han +8Electroencephalography Foundation Models Electroencephalography
Tsinghua Laboratory of Brain and Intelligence, Tsinghua University · School of Basic Medical Sciences, Tsinghua Medicine, Tsinghua University · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University +2
DARE-EEG: A Foundation Model for Mining Dual-Aligned Representation of EEG
May 18, 2026Yang Shao, Peiliang Gong, Qun Dai +1Electroencephalography Foundation Models Small-Sample Electroencephalogram Datasets
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics Nanjing, Jiangsu, China
MANAS-2: Constrained Reconstruction for EEG Foundation Models
Sep 12, 2026Arvasu Kulkarni, Aditya Ray Mishra, Jeet Bandhu Lahiri +5Electroencephalography Foundation Models 3D Masked Autoencoders
Mannas AI · Indian Institute of Technology, Mandi

Jul 31, 2026, eess.SPJ/K move · Enter open · S save

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models

Jinhao Li, Zhiyuan Ma, Xueqiao Han, Zhongye Xia+7

Tsinghua Laboratory of Brain and Intelligence, Tsinghua University · School of Basic Medical Sciences, Tsinghua Medicine, Tsinghua University · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University+2

Electroencephalography Foundation Models Electroencephalography Latent Prediction Electrocardiogram Foundation Models Pretraining

Open paper AskPDF arXiv

Electroencephalography (EEG) foundation models aim to learn reusable representations from large-scale unlabeled recordings. A common pretraining strategy is masked waveform reconstruction, but applying supervision directly to noisy EEG may encourage models to recover predictable background activity, acquisition effects, and artifacts rather than neural structure that transfers across tasks. This raises a central question: what should an EEG foundation model predict to learn transferable representations? We introduce EEG-JEPA a structured latent-prediction framework for EEG foundation modeling. Rather than reconstructing masked voltage samples, a masked context encoder and predictor infer contextual latent states produced by an exponential-moving-average target encoder that observes the complete input. EEG-JEPA organizes target design along three complementary dimensions: target content specifies what representation is predicted, target support specifies where prediction occurs over structured electrode--time regions through Neurotopology-Aware Multi-scale Electrode-Temporal Masking (N-MET), and target depth specifies at which encoder layers supervision is applied. Together, these designs shift EEG pretraining from recovering missing measurements to inferring latent states from structured electrode--time context. We evaluate EEG-JEPA through controlled objective comparisons, frozen multitask transfer, and full fine-tuning. Under the same backbone, pretraining corpus, and training duration, EEG-JEPA improves the 14-task frozen macro balanced accuracy from 40.49% to 50.42% over CBraMod-style masked waveform reconstruction. Multi-source continuation further raises this result to 52.94%, the highest average among the EEG foundation models evaluated on EEG-FM-Bench. Under protocol-matched full fine-tuning, EEG-JEPA also improves the nine-task average balanced accuracy from 68.98% to 70.65%.

Abstract

Explore similar work

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models

DARE-EEG: A Foundation Model for Mining Dual-Aligned Representation of EEG

MANAS-2: Constrained Reconstruction for EEG Foundation Models