Taming foundation model with invariance-oriented pre-training for broad-spectrum EEG analysis across signal-level, brain-state, and brain-health tasks

Yulong Dou; Han Wu; Guo Chen; Fangmao Ju; Zhiming Cui; Dinggang Shen

cs.LGAug 25, 2026

Taming foundation model with invariance-oriented pre-training for broad-spectrum EEG analysis across signal-level, brain-state, and brain-health tasks

Authors: Yulong Dou, Han Wu, Guo Chen, Fangmao Ju, Zhiming Cui, Dinggang Shen

Organizations: School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, 201210, China. · Shanghai United Imaging Intelligence Co., Ltd., Shanghai, 200230, China. · Shanghai Clinical Research and Trial Center, Shanghai, 201210, China.

Abstract

Electroencephalography (EEG) is a widely used window into human brain function, but most EEG models remain tied to a one-dataset-one-model supervised paradigm. Recent EEG foundation models offer a route toward reusable representations, but most remain reconstruction-centered, assuming that EEG content predictable from local context is necessarily transferable neural information. Here we present INCEPT, an invariance-oriented EEG foundation model trained on over 11,000 hours of unlabelled clinical EEG. Rather than prioritizing signal recovery alone, INCEPT learns representation-level stability across correlated EEG observations, separating stable neural structure and essential subject-sensitive information from the nuisance variability that dominates scalp recordings while preserving subject-, state- and condition-discriminative information. We evaluate INCEPT on a broad-spectrum benchmark of ten datasets spanning three levels of post-acquisition EEG analysis: signal-level assessment, brain-state decoding, and brain-health evaluation. INCEPT ranks first among recent EEG foundation models on 26 of 30 linear-probing metrics and 24 of 30 fine-tuning metrics, and also surpasses strong task-specific specialist encoders across diverse downstream settings. Objective ablations and representation analyses further show that invariance-oriented pre-training improves transfer and organizes subject-sensitive neural representations beyond reconstruction alone. These results establish invariance learning as a promising principle for building reusable EEG foundation models.

Figures & tables

Figure 1: Comparison of EEG modelling paradigms. Schematic comparison of task-specific supervised EEG encoders, reconstruction-centered EEG foundation models and the proposed invariance-oriented INCEPT framework. INCEPT learns compact global EEG representations through invariance-oriented contextual pre-training while retaining patch-level features as optional outputs.

Panel of Figure 2 — Figure 2: Heterogeneous downstream EEG benchmarks and transfer overview. a , Downstream EEG benchmarks spanning signal-level EEG assessment, brain-state decoding, and brain-health evaluation, with varying sample scales, class numbers, channel counts and segment durations. b , Radar plots providing a compact overview of model transfer performance across the ten downstream EEG datasets under linear probing and full fine-tuning.

Panel of Table 1 — Table 1: Dataset characteristics and cohort information for pre-training and downstream evaluation. a , Task and signal characteristics of the pre-training corpus and downstream EEG benchmarks, including original sampling rate, channel count, segment duration, sample scale, subject count and label space. Sampling rates refer to the original recordings before resampling in our preprocessing pipeline, and the TUEG sample scale denotes the amount of usable EEG retained for self-supervised pre-training after preprocessing and quality filtering. b , Source country or region, acquisition scenario and available participant information for each dataset. Age is reported as mean $\pm$ s.d. when available, either by sex or by diagnostic group according to the original metadata, and sex distribution is reported as the percentage of male participants. Dashed separators indicate the pre-training corpus, signal-level EEG assessment tasks, brain-state decoding tasks, and brain-health evaluation tasks.

Panel of Figure 3 — Figure 3: Signal-level EEG quality assessment performance across three evaluation metrics. a , TUAB abnormal EEG detection using balanced accuracy, AUROC and AUC-PR. b , TUAR artifact-type recognition using balanced accuracy, weighted F1 score and Cohen’s kappa. Bars indicate mean $\pm$ s.d. over five random seeds, with hollow circles denoting individual seeds.

Task-specific Supervised Encoders
Method	TUAB (2-class)			TUAR (4-class)
Method	Bal. Acc.	AUROC	AUC-PR	Bal. Acc.	Weighted F1	Kappa
EEGNet	78.35 ${}_{\pm 1.00\phantom{0}}$	87.33 ${}_{\pm 0.55\phantom{0}}$	87.69 ${}_{\pm 0.56\phantom{0}}$	54.42 ${}_{\pm 1.87\phantom{0}}$	64.97 ${}_{\pm 1.14\phantom{0}}$	43.71 ${}_{\pm 2.10\phantom{0}}$
ST-Transformer	76.76 ${}_{\pm 2.02\phantom{0}}$	85.06 ${}_{\pm 1.61\phantom{0}}$	83.29 ${}_{\pm 2.00\phantom{0}}$	35.84 ${}_{\pm 2.62\phantom{0}}$	50.76 ${}_{\pm 2.96\phantom{0}}$	20.96 ${}_{\pm 4.25\phantom{0}}$
EEGConformer	78.76 ${}_{\pm 0.96\phantom{0}}$	87.15 ${}_{\pm 0.89\phantom{0}}$	87.17 ${}_{\pm 1.06\phantom{0}}$	52.41 ${}_{\pm 1.49\phantom{0}}$	63.00 ${}_{\pm 1.17\phantom{0}}$	41.87 ${}_{\pm 1.85\phantom{0}}$
SPaRCNet	80.74 ${}_{\pm 0.59\phantom{0}}$	88.80 ${}_{\pm 0.57\phantom{0}}$	89.37 ${}_{\pm 0.57\phantom{0}}$	55.06 ${}_{\pm 1.31\phantom{0}}$	65.50 ${}_{\pm 1.73\phantom{0}}$	45.39 ${}_{\pm 2.86\phantom{0}}$

Table 2: Signal-level EEG quality assessment performance on TUAB and TUAR.

Panel of Figure 4 — Figure 4: Brain-state decoding performance across three evaluation metrics. a , FACED; b , SEED-V; c , PhysioNet-MI; d , ISRUC-S1. Columns show balanced accuracy, weighted F1 score and Cohen’s kappa for each dataset.

Task-specific Supervised Encoders
Method	FACED (9-class)			SEED-V (5-class)
Method	Bal. Acc.	Weighted F1	Kappa	Bal. Acc.	Weighted F1	Kappa
EEGNet	74.28 ${}_{\pm 1.08\phantom{0}}$	73.97 ${}_{\pm 1.09\phantom{0}}$	70.61 ${}_{\pm 1.24\phantom{0}}$	35.80 ${}_{\pm 0.49\phantom{0}}$	34.56 ${}_{\pm 0.84\phantom{0}}$	18.87 ${}_{\pm 0.68\phantom{0}}$
ST-Transformer	73.61 ${}_{\pm 2.64\phantom{0}}$	73.36 ${}_{\pm 2.69\phantom{0}}$	69.98 ${}_{\pm 3.04\phantom{0}}$	21.99 ${}_{\pm 0.38\phantom{0}}$	19.73 ${}_{\pm 0.83\phantom{0}}$	0 2.29 ${}_{\pm 0.42\phantom{0}}$
EEGConformer	80.68 ${}_{\pm 1.43\phantom{0}}$	80.42 ${}_{\pm 1.39\phantom{0}}$	77.95 ${}_{\pm 1.62\phantom{0}}$	33.94 ${}_{\pm 2.13\phantom{0}}$	33.28 ${}_{\pm 2.50\phantom{0}}$	16.83 ${}_{\pm 2.74\phantom{0}}$
SPaRCNet	69.74 ${}_{\pm 1.40\phantom{0}}$	69.77 ${}_{\pm 1.35\phantom{0}}$	65.48 ${}_{\pm 1.54\phantom{0}}$	29.42 ${}_{\pm 0.59\phantom{0}}$	28.51 ${}_{\pm 0.86\phantom{0}}$	11.41 ${}_{\pm 0.83\phantom{0}}$

Table 3: Brain-state decoding performance across montage-shifted EEG tasks.

Panel of Figure 5 — Figure 5: Brain-health evaluation performance across three evaluation metrics. a , Mumtaz2016; b , MentalArithmetic; c , ADFTD; d , Siena. Binary datasets use balanced accuracy, AUROC and AUC-PR, whereas ADFTD uses balanced accuracy, weighted F1 score and Cohen’s kappa.

Task-specific Supervised Encoders
Method	Mumtaz2016 (2-class)			MentalArithmetic (2-class)
Method	Bal. Acc.	AUROC	AUC-PR	Bal. Acc.	AUROC	AUC-PR
EEGNet	89.61 ${}_{\pm 0.99\phantom{0}}$	96.06 ${}_{\pm 1.38\phantom{0}}$	96.95 ${}_{\pm 0.91\phantom{0}}$	55.49 ${}_{\pm 3.24\phantom{0}}$	67.42 ${}_{\pm 3.55\phantom{0}}$	42.06 ${}_{\pm 5.58\phantom{0}}$
ST-Transformer	75.87 ±10.04	95.61 ${}_{\pm 1.46\phantom{0}}$	96.57 ${}_{\pm 0.84\phantom{0}}$	50.83 ${}_{\pm 1.67\phantom{0}}$	58.28 ${}_{\pm 3.44\phantom{0}}$	31.90 ${}_{\pm 2.99\phantom{0}}$
EEGConformer	90.66 ${}_{\pm 2.00\phantom{0}}$	96.36 ${}_{\pm 2.25\phantom{0}}$	97.25 ${}_{\pm 1.31\phantom{0}}$	60.90 ${}_{\pm 6.38\phantom{0}}$	71.64 ${}_{\pm 2.40\phantom{0}}$	49.68 ${}_{\pm 3.89\phantom{0}}$
SPaRCNet	90.69 ${}_{\pm 2.65\phantom{0}}$	97.04 ${}_{\pm 0.67\phantom{0}}$	97.80 ${}_{\pm 0.49\phantom{0}}$	64.10 ${}_{\pm 7.25\phantom{0}}$	77.70 ${}_{\pm 4.39\phantom{0}}$	56.59 ${}_{\pm 5.45\phantom{0}}$

Table 4: Brain-health evaluation performance across psychiatric, cognitive and neurological EEG tasks.

Panel of Figure 6 — Figure 6: Representation analysis of learned EEG features across neurodegenerative disease and motor imagery tasks. a - d , Representation analysis on the ADFTD neurodegenerative disease classification dataset comparing INCEPT ( a , b ) and CBraMod ( c , d ). Left panels ( a , c ) show joint t-SNE visualizations of EEG segments, where colors denote diagnostic classes and marker shapes denote adaptation protocols (circles for linear probing and triangles for full fine-tuning; large markers indicate subject-level centroids). Right panels ( b , d ) show cosine distance distributions for segment pairs from the same subject, from different subjects within the same class and from different classes. e - h , Corresponding analysis on the PhysioNet-MI motor imagery dataset for INCEPT ( e , f ) and CBraMod ( g , h ). In e and g , ellipses enclose the four class centroids from the same subject under each adaptation protocol, illustrating subject-level organization across motor imagery conditions. Distance distributions in f and h are computed for four types of trial pairs: same subject and same class, same subject and different classes, different subjects and same class, and different subjects and different classes. For all violin plots, box plots summarize the median and interquartile range, and white circles with error bars indicate mean $\pm$ s.d.

Figure 7: Quantitative ablation of self-supervised pre-training objectives. Masked modeling, invariance learning and the combined objective are compared under the same backbone and downstream full fine-tuning protocol. Points denote the mean across five random seeds, error bars indicate mean

\pm

s.d., and hollow circles show individual seed results.

Panel of Figure 8 — Figure 8: t-SNE visualization of pre-training objective ablation on ADFTD. a, ADFTD representations under linear probing, where the backbone is frozen and the embedding geometry is determined by the pre-training objective. b, ADFTD representations after full fine-tuning, where the backbone is further adapted by downstream diagnostic labels. Each point denotes an EEG segment, colors denote diagnostic classes, including Alzheimer’s disease (AD), healthy control (HC) and frontotemporal dementia (FTD), and marker shapes denote objective variants. Large markers indicate subject-level centroids. Dashed, dash-dotted and solid ellipses summarize the overall embedding regions of masked modeling, invariance learning and the combined objective, respectively.

Panel of Figure 9 — Figure 9: Pairwise-distance analysis of pre-training objective ablation on ADFTD. a, Cosine-distance distributions of ADFTD representations under linear probing. b, Cosine-distance distributions after full fine-tuning. For each objective variant, distances are computed for segment pairs from the same subject, from different subjects within the same diagnostic class and from different diagnostic classes. Violin plots show the distribution of pairwise cosine distances, box plots summarize the median and interquartile range, and white circles with error bars indicate mean $\pm$ s.d..

Figure 10: Overview of the INCEPT pre-training framework and downstream prediction. a, Dynamic spatiotemporal sampler. Given a raw EEG segment, INCEPT first patchifies each channel into non-overlapping temporal patches and then constructs correlated views with dynamic spatial and temporal extents. Macro-level views preserve broad brain-state context, micro-level views capture localized neural dynamics, and masked macro-level views introduce block-wise missing regions for contextual inference. b, Neural token embedder. Each sampled view is converted into structured EEG tokens by combining time-frequency signal embeddings, electrode spatial embeddings and within-channel temporal embeddings. Patch-level content embeddings retain local contextual information, whereas channel-level perception embeddings provide an explicit interface for summarizing neural activity at the electrode level. c, Self-supervised invariance learner. Student and teacher encoders share an alternating Transformer backbone that interleaves channel-wise self-attention for within-electrode temporal modeling with global-joint self-attention for cross-electrode contextual integration. During pre-training, contrastive invariance learning aligns perception embeddings across macro-micro and macro-masked-macro views, masked contextual learning aligns content embeddings between unmasked and masked macro-level views, and within-batch KoLeo regularization encourages whole-brain perception embeddings from different subjects to remain well distributed in latent space. d, Downstream prediction. For downstream tasks, INCEPT primarily uses channel-level perception representations, which first summarize electrode-level neural activity and can then be aggregated into compact whole-brain representations for classification. EEG patch content representations are retained as optional outputs for settings that require finer-grained contextual information.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure 11: Complete downstream performance on montage-matched datasets. a , TUAB. b , TUAR. c , Mumtaz2016. d , ADFTD. e , MentalArithmetic. Each row corresponds to one dataset and each column corresponds to one evaluation metric. For TUAB, Mumtaz2016 and MentalArithmetic, the metrics are balanced accuracy, AUROC and AUC-PR. For TUAR and ADFTD, the metrics are balanced accuracy, Cohen’s kappa and weighted F1 score. Bars indicate mean $\pm$ s.d. over five random seeds, with hollow circles denoting individual seeds. Methods are grouped as task-specific supervised encoders, EEG foundation models with linear probing and EEG foundation models with full fine-tuning.

Panel of Figure 12 — Figure 12: Complete downstream performance on montage-shifted datasets. f , Siena. g , PhysioNet-MI. h , ISRUC-S1. i , FACED. j , SEED-V. Each row corresponds to one dataset and each column corresponds to one evaluation metric. For Siena, the metrics are balanced accuracy, AUROC and AUC-PR. For PhysioNet-MI, ISRUC-S1, FACED and SEED-V, the metrics are balanced accuracy, Cohen’s kappa and weighted F1 score. Bars indicate mean $\pm$ s.d. over five random seeds, with hollow circles denoting individual seeds. Methods are grouped as task-specific supervised encoders, EEG foundation models with linear probing and EEG foundation models with full fine-tuning.

Panel of Figure 13 — Figure 13: Data and model scaling behaviour of EEG foundation pre-training. a , Effect of increasing pre-training data size on downstream balanced accuracy under the full fine-tuning protocol. Results are shown as absolute scores and normalized scores, with each downstream dataset normalized by the performance obtained using 100% of the pre-training data. Lines denote the mean across five random seeds, shaded regions indicate mean $\pm$ s.d., and hollow circles show individual seed results. b , Effect of increasing model capacity on downstream balanced accuracy under the full fine-tuning protocol, shown as absolute scores and normalized scores, with each downstream dataset normalized by the performance of the largest model. c , Parameter counts for the Transformer depth and embedding-dimension combinations used in the model-scaling analysis.

ID	Variant	PCA Aug.	Dynamic Views	Freq. Branch	SH Enc.	Channel-wise Self-Attn.
A	Basic Baseline	-	-	-	-	-
B	+ Multi-band PCA Augmentation	✓	-	-	-	-
C	+ Dynamic Multi-view Sampling	✓	✓	-	-	-
D	+ Dual-domain Signal Embedding	✓	✓	✓	-	-
E	+ Geometry-aware Electrode Encoding	✓	✓	✓	✓	-
F	+ Channel-wise Self-attention	✓	✓	✓	✓	✓

Appendix

Table 5: Matrix for staged component ablation. Check marks indicate components included in each variant. PCA Aug., multi-band PCA-based signal augmentation; Dynamic Views, construction of macro-level, micro-level and masked macro-level views with dynamic spatiotemporal configurations (i.e., varying channel and temporal extents); Freq. Branch, auxiliary frequency-domain embedding branch in addition to time-domain encoding; SH Enc., spherical-harmonics-based electrode encoding added to Cartesian coordinates; Channel-wise Self-Attn., alternating channel-wise and global-joint self-attention.

ID	Mumtaz2016 (2-Class)			FACED (9-Class)
ID	Bal. Acc.	AUROC	AUC-PR	Bal. Acc.	W. F1	Kappa
A	90.40 ±0.28	96.90 ±0.48	97.51 ±0.34	71.32 ±0.62	71.18 ±0.47	67.47 ±0.60
B	90.48 ±0.33	97.47 ±0.69	97.86 ±0.44	72.48 ±0.52	72.30 ±0.56	68.77 ±0.61
C	91.00 ±0.96	97.88 ±0.64	98.15 ±0.43	74.23 ±0.60	74.05 ±0.65	70.74 ±0.70
D	91.78 ±0.63	98.78 ±0.24	98.81 ±0.23	74.65 ±0.52	74.53 ±0.57	71.18 ±0.58
E	92.08 ±0.69	99.01 ±0.27	99.04 ±0.23	78.54 ±0.85	78.23 ±0.74	75.50 ±0.86

Appendix

Table 6: Performance of staged component ablation on Mumtaz2016 and FACED. Mumtaz2016 is a montage-matched 10-20 EEG dataset for major depressive disorder classification, while FACED is a montage-shifted emotion recognition dataset with a low-density 10-10 montage configuration. Variant IDs correspond to Table 5 . Values are reported as mean

\pm

s.d. over five random seeds.

EEG Foundation Models

Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.

See all 102 papers

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models
Jul 31, 2026Jinhao Li, Zhiyuan Ma, Xueqiao Han +8Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture
Tsinghua Laboratory of Brain and Intelligence, Tsinghua University · School of Basic Medical Sciences, Tsinghua Medicine, Tsinghua University · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University +2
Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction
Mar 17, 2026Saarang Panchavati, Uddhav Panchavati, Hiroki Nariai +2EEG Foundation Models Self-Supervised Time Series Representation Learning
Department of Medical Informatics UCLA Los Angeles, USA · Department of Cognitive Science UCSD San Diego, USA · David Geffen School of Medicine UCLA Mattel Children’s Hospital Los Angeles, USA
What Do EEG Foundation Models Capture from Human Brain Signals?
May 12, 2026Ling Tang, Qian Chen, Jilin Mei +6Frequency-Domain Feature Learning Representation Probing
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Tongji University +2

Jul 31, 2026J/K move · Enter open · S save

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models

Jinhao Li, Zhiyuan Ma, Xueqiao Han, Zhongye Xia+7

Tsinghua Laboratory of Brain and Intelligence, Tsinghua University · School of Basic Medical Sciences, Tsinghua Medicine, Tsinghua University · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University+2

Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture Predictive Representation Learning EEG Foundation Models

Open paper AskPDF arXiv

Electroencephalography (EEG) foundation models aim to learn reusable representations from large-scale unlabeled recordings. A common pretraining strategy is masked waveform reconstruction, but applying supervision directly to noisy EEG may encourage models to recover predictable background activity, acquisition effects, and artifacts rather than neural structure that transfers across tasks. This raises a central question: what should an EEG foundation model predict to learn transferable representations? We introduce EEG-JEPA a structured latent-prediction framework for EEG foundation modeling. Rather than reconstructing masked voltage samples, a masked context encoder and predictor infer contextual latent states produced by an exponential-moving-average target encoder that observes the complete input. EEG-JEPA organizes target design along three complementary dimensions: target content specifies what representation is predicted, target support specifies where prediction occurs over structured electrode--time regions through Neurotopology-Aware Multi-scale Electrode-Temporal Masking (N-MET), and target depth specifies at which encoder layers supervision is applied. Together, these designs shift EEG pretraining from recovering missing measurements to inferring latent states from structured electrode--time context. We evaluate EEG-JEPA through controlled objective comparisons, frozen multitask transfer, and full fine-tuning. Under the same backbone, pretraining corpus, and training duration, EEG-JEPA improves the 14-task frozen macro balanced accuracy from 40.49% to 50.42% over CBraMod-style masked waveform reconstruction. Multi-source continuation further raises this result to 52.94%, the highest average among the EEG foundation models evaluated on EEG-FM-Bench. Under protocol-matched full fine-tuning, EEG-JEPA also improves the nine-task average balanced accuracy from 68.98% to 70.65%.

Abstract

EEG Foundation Models

Explore similar work

EEG-JEPA: Structured Latent Prediction for EEG Foundation Models

Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction

What Do EEG Foundation Models Capture from Human Brain Signals?