cs.LGOct 23, 2026

Controlled Evaluation of EEG Foundation Models: Random Initialisation, Band-Power and Validation-Selected Probes

Authors: Mingyu Jeon, Suwan Cho, Jae Young Suh

Organizations: Modulabs, Seoul, Republic of Korea

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

EEG foundation models report downstream gains, yet their protocols seldom include three controls. These are the same encoder untrained, spectral features fitted on the same data, and a layer picked without looking at test scores. We therefore evaluate five publicly released EEG foundation models and an in-house JEPA-style model under one protocol with all three controls and several seeds, on five benchmarks and four added task families. The controls change the reading in three ways. First, picking the probed layer by its test score raises the layer-wise peak by an average of +0.034 and reverses the peak-versus-output gap in 9 of 36 cells. Second, with the layer picked on validation, random initialisation matches or beats pretraining for half of the models on sleep and emotion, and no model beats a five-band log-power baseline on emotion or motor imagery. Third, after full fine-tuning only 5 of 30 pairs gain beyond twice their seed spread, and the one clear loss of -0.290 AUROC persists under matched learning rates. From these results we draw four reporting practices, and we release every cell for re-analysis.

Figures & tables

Figure 1: The in-house model as released. Each channel passes through the sinc front-end (Eq. 1 ), which acts as a low-pass filter below one hertz as implemented, and a patch convolution; tokens add electrode-position and temporal embeddings (Eq. 2 ). The encoder reads the unmasked window, masking replaces its outputs by a mask token before the predictor, and the projected prediction is regressed onto the stop-gradient projection of the encoder output, with SIGReg on that projection (Eqs. 3 and 4 ). Only the encoder is evaluated downstream.
DatasetTaskclassestrain / val / testmetricsplit
APAVAAlzheimer’s disease vs control2347 / 157 / 159AUROCfixed, four test subjects
BCI IV 2amotor imagery42,880 / 1,152 / 1,152accuracyfixed, subjects 1–5 / 6–7 / 8–9
Lee2019motor imagery22,080 / 520 / 2,600AUROCfixed, within-subject halves
TUSLslowing vs other212,187 / 1,050 / 1,503AUROCsubject-disjoint, per seed
TUABabnormal vs normal212,866 / 3,192 / 3,942AUROCsubject-disjoint, per seed
Table 1: Original benchmarks: task, classes, windows per split (five seconds at 200 Hz, read from the evaluation outputs) and the split rule. TUSL and TUAB draw a new subject-disjoint split for every seed (sizes shown for the first seed); the other three keep one split. The TUSL positive class is rare, so AUROC is reported.
Figure 2: Layer-wise probe protocol. (a) A linear probe at every internal stage of the frozen encoder (twelve for the in-house model), with the band power anchor on the same split. (b) Per-stage mechanics: mean-pool over tokens, standardise on the training split, logistic regression. (c) The gap GG is the peak minus the output stage, with the peak s^\hat{s} selected on validation (Eq. 5 ; measured curves in Figure 3 ).
Figure 3: Layer-wise frozen probe (test AUROC) for three cells of the leakage-free benchmarks (all cells in Appendix D ). Solid: pretrained; dashed: random-init; dotted: band power; circle: test-argmax stage; star: validation-selected stage. Left: a genuine early-stage peak that the random twin reproduces. Middle: validation picks an early stage while the test argmax sits one stage before the output. Right: the largest selection penalty.
Figure 4: Added task families, frozen probe at the validation-selected stage (mean ±\pm std over 3 seeds; a deterministic probe has no bar): pretrained (circles), random-init (squares) and each model’s matched band power (grey bar). LuMamba was not run under the within-subject MI protocol. Numbers in Appendix E .
MethodAPAVABCI IV 2aLee2019TUSLTUAB
EEGNet0.723 ±\pm 0.1970.482 ±\pm 0.0420.796 ±\pm 0.0460.535 ±\pm 0.0350.863 ±\pm 0.009
FBCSP—0.2600.599——
Band power0.951–0.9850.484–0.4910.733–0.7430.645–0.6900.853–0.855
Best model, fine-tuned0.959 EEGPT0.351 CBraMod0.848 CBraMod0.758 CBraMod0.874 LaBraM
Table 2: Classical baselines on the original benchmarks, same splits: EEGNet trained from scratch (mean ±\pm std over 5 seeds), FBCSP (motor imagery only), the band-power probe (range over the models’ input pipelines) and the best pretrained model after full fine-tuning. AUROC except BCI IV 2a (accuracy, four classes).
Figure 5: Full fine-tuning gain Δ\Delta per model and benchmark. Filled markers: mean beyond twice the seed standard deviation. The in-house Lee2019 cell carries developer contact with that benchmark (Section 5.3 ).
ModelAPAVABCI IV 2aLee2019TUSLTUABmean ( kk /5)
CBraMod+0.010 ±\pm 0.141+0.052 ±\pm 0.030+0.237 ±\pm 0.030+0.147 ±\pm 0.141+0.034 ±\pm 0.012+0.096 (5)
LaBraM+0.143 ±\pm 0.166+0.047 ±\pm 0.041+0.057 ±\pm 0.054+0.059 ±\pm 0.075+0.061 ±\pm 0.008+0.073 (5)
EEGPT+0.018 ±\pm 0.045+0.003 ±\pm 0.040−- 0.005 ±\pm 0.014+0.154 ±\pm 0.119−- 0.004 ±\pm 0.010+0.033 (3)
LUNA−- 0.290 ±\pm 0.035+0.041 ±\pm 0.085+0.084 ±\pm 0.058−- 0.049 ±\pm 0.157+0.004 ±\pm 0.009−- 0.042 (3)
LuMamba+0.026 ±\pm 0.030−- 0.032 ±\pm 0.044+0.002 ±\pm 0.008−- 0.052 ±\pm 0.110+0.012 ±\pm 0.012−- 0.009 (3)
In-house (JEPA)+0.091 ±\pm 0.042+0.061 ±\pm 0.022+0.013 ±\pm 0.010+0.072 ±\pm 0.091−- 0.017 ±\pm 0.009+0.044 (4)
Table 3: Gain Δ\Delta of full fine-tuning, pretrained minus random-init at the best-validation epoch (test metric), mean ±\pm population std over 5 seeds, on the five original benchmarks; bold = mean beyond twice its seed std. kk = datasets with a positive mean. The in-house Lee2019 cells carry developer contact with that benchmark (Section 5.3 ).
Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

ModelHugging Face repositoryFileRevision
EEG-OmniJEPA (ours, REVE 25k)not releasedckpts/step_025000_weights_only.pt9567ab6053
LaBraM-Basebraindecode/labram-pretrainedmodel.safetensors0563b6c626
CBraModbraindecode/cbramod-pretrainedmodel.safetensors584cdc4159
LUNA-BasePulpBio/LUNALUNA_base.safetensors999c1af0fd
LuMamba LeJEPA-onlyPulpBio/LuMambaLuMamba_LeJEPAOnly_128slices.safetensorsfd3da57e34
EEGPT-Basebraindecode/eegpt-pretrainedmodel.safetensorse41cb3ae2c
Appendix
Table 4: Model checkpoints: repository, file and the revision pinned when the paper is built (hf.lock.json).
Modelpretrained ratefed ratepretrained windowfed window
CBraMod200200305
LaBraM200200≤\leq 256 one-second patches15
EEGPT25625044
LUNA25625655
LuMamba25625655
Appendix
Table 5: Input each public model was pretrained on (its paper or official release) and the input its adapter feeds in this audit, after the shared chain of Table 1 . Sampling rate in Hz, window in seconds.
ItemValue
Sampling rate / window200 Hz, five seconds, common-average reference, band-pass and notch filtering
Bandpower baselinelog band power in five bands (0.5–4, 4–8, 8–13, 13–30, 30–75 Hz) per channel, then the same probe
Linear probemean-pool over tokens, standardise on the training split, L2 logistic regression ( C=1.0C=1.0 , at most 3000 iterations)
Stage selectionargmax of the validation metric over stages; test metric reported at that stage
Full fine-tuningeach model’s own head; learning rate 5e-5 (pretrained) / 5e-4 (random-init); up to 50 epochs, batch 32; best-validation epoch reported
Seeds5 (original benchmarks) or 3 (added tasks); population std throughout
Appendix
Table 6: Protocol constants, read from the evaluation code at build time.
Signal0–1 Hz1–4 Hz4–8 Hz8–13 Hz13–30 Hz30–75 Hz
Input37.429.06.37.56.912.7
Front-end as released91.27.20.80.30.30.1
Same filters, nn in samples0.29.416.020.418.634.9
Appendix
Table 7: What the released in-house front-end passes. Top: share of spectral power per band (%) in Lee2019 (first 32 windows, all channels), at the input, at the output of the released front-end, and at the output of the same filters with the time index in samples ( nn instead of n/fsn/f_{s} in Eq. 1 ; not retrained). Bottom: gain (output over input amplitude) for pure sines, for four of the filters.
Figure 6: Layer-wise frozen linear probe for every pretrained cell of the three leakage-free benchmarks (rows: datasets; columns: models; x-axis: probed stage from input to output). Solid: pretrained; dashed: random-init; dotted: band power; circle: test-argmax stage; star: validation-selected stage.
Modelpretrainedrandom-init
 APAVABCI IV 2aLee2019APAVABCI IV 2aLee2019
CBraMod+0.110 (+0.000)+0.003 (+0.042)−- 0.013 (+0.041)+0.011 (+0.041)−- 0.004 (+0.020)+0.006 (+0.007)
LaBraM−- 0.113 (+0.181)−- 0.001 (+0.014)−- 0.018 (+0.041)+0.000 (+0.000)−- 0.010 (+0.010)+0.007 (+0.000)
EEGPT+0.073 (+0.191)−- 0.006 (+0.023)+0.019 (+0.000)+0.015 (+0.024)−- 0.007 (+0.016)−- 0.008 (+0.014)
LUNA+0.023 (+0.062)−- 0.016 (+0.021)+0.000 (+0.018)−- 0.023 (+0.128)+0.023 (+0.000)−- 0.000 (+0.007)
LuMamba−- 0.139 (+0.145)−- 0.016 (+0.036)+0.009 (+0.000)−- 0.002 (+0.020)−- 0.007 (+0.007)+0.000 (+0.002)
Appendix
Table 8: Frozen probe on the three leakage-free original benchmarks, both initialisations (36 cells): validation-selected peak minus output stage (selection penalty = test-argmax minus validation-selected peak, in parentheses). APAVA and Lee2019 report AUROC, BCI IV 2a accuracy; the in-house Lee2019 cells carry developer contact with that benchmark.
ModelERP P300 (AUROC)Sleep 5-cl. (bal. acc.)Emotion 8-cl. (macro-F1)MI within-subj. (acc.)
 pre / randΔ\Deltapre / randΔ\Deltapre / randΔ\Deltapre / randΔ\Delta
CBraMod0.768 / 0.703+0.0650.596 / 0.461+0.1350.165 / 0.139+0.0270.344 / 0.288+0.057
LaBraM0.742 / 0.684+0.0580.527 / 0.479+0.0480.168 / 0.136+0.0320.354 / 0.301+0.052
EEGPT0.588 / 0.641−- 0.0520.552 / 0.584−- 0.0330.151 / 0.154−- 0.0030.278 / 0.379−- 0.100
LUNA0.615 / 0.568+0.0460.568 / 0.593−- 0.0240.165 / 0.174−- 0.0080.316 / 0.344−- 0.028
LuMamba0.628 / 0.581+0.0470.565 / 0.601−- 0.0350.162 / 0.167−- 0.005——
Appendix
Table 9: Frozen linear probe on the four added task families: validation-selected stage, test metric, mean of 3 seeds; Δ\Delta = pretrained −- random-init, computed before rounding; last row = five-band log-power baseline on the same split. Seed std ≤\leq 0.024; on the within-subject protocols the split is deterministic, so pretrained is a point estimate and only random-init carries a seed band. The band-power baseline is computed on each model’s own preprocessed input; each model is compared with its own baseline (seed mean, std ≤\leq 0.023), and the last row gives the range over models. MI within-subject: 5 models, as LuMamba was not run under that protocol.
ModelERP P300Sleep 5-cl.Emotion 8-cl.MI within-subj.
 pretrainedrandom-initpretrainedrandom-initpretrainedrandom-initpretrainedrandom-init
CBraMod0.768 ±\pm 0.0000.703 ±\pm 0.0060.596 ±\pm 0.0110.461 ±\pm 0.0240.165 ±\pm 0.0060.139 ±\pm 0.0110.344 ±\pm 0.0000.288 ±\pm 0.011
LaBraM0.742 ±\pm 0.0000.684 ±\pm 0.0110.527 ±\pm 0.0170.479 ±\pm 0.0230.168 ±\pm 0.0130.136 ±\pm 0.0040.354 ±\pm 0.0000.301 ±\pm 0.014
EEGPT0.588 ±\pm 0.0000.641 ±\pm 0.0050.552 ±\pm 0.0120.584 ±\pm 0.0220.151 ±\pm 0.0060.154 ±\pm 0.0060.278 ±\pm 0.0000.379 ±\pm 0.003
LUNA0.615 ±\pm 0.0000.568 ±\pm 0.0170.568 ±\pm 0.0090.593 ±\pm 0.0040.165 ±\pm 0.0010.174 ±\pm 0.0080.316 ±\pm 0.0000.344 ±\pm 0.011
LuMamba0.628 ±\pm 0.0000.581 ±\pm 0.0050.565 ±\pm 0.0110.601 ±\pm 0.0110.162 ±\pm 0.0020.167 ±\pm 0.007——
Appendix
Table 10: Table 9 with seed standard deviations (3 seeds; a std of 0.000 marks a deterministic within-subject pretrained probe).
ModelDatasetmean-pool finalattention finaldd
LUNAAPAVA0.9050.886 ±\pm 0.035−- 0.019
LUNABCI IV 2a0.3260.309 ±\pm 0.003−- 0.017
LUNALee20190.5560.523 ±\pm 0.009−- 0.033
In-house (JEPA)APAVA0.6310.562 ±\pm 0.004−- 0.069
In-house (JEPA)BCI IV 2a0.2860.275 ±\pm 0.017−- 0.011
In-house (JEPA)Lee20190.5060.499 ±\pm 0.004−- 0.007
Appendix
Table 11: Trainable attention-pooling readout on frozen features (3 seeds) against the mean-pool linear probe at the output stage. dd = attention minus mean-pool; a non-positive dd means the native-style readout does not recover the output layer.
ModelAPAVABCI IV 2aLee2019
CBraMod−- 0.017 ±\pm 0.021−- 0.015 ±\pm 0.006+0.004 ±\pm 0.001
LaBraM+0.039 ±\pm 0.013−- 0.005 ±\pm 0.018−- 0.021 ±\pm 0.001
EEGPT−- 0.132 ±\pm 0.009−- 0.019 ±\pm 0.012−- 0.003 ±\pm 0.004
LUNA−- 0.044 ±\pm 0.006−- 0.019 ±\pm 0.019−- 0.026 ±\pm 0.002
LuMamba−- 0.044 ±\pm 0.002−- 0.007 ±\pm 0.009−- 0.026 ±\pm 0.002
In-house (JEPA)−- 0.036 ±\pm 0.006−- 0.026 ±\pm 0.002−- 0.001 ±\pm 0.004
Appendix
Table 12: One-hidden-layer MLP probe minus linear probe at the output stage, pretrained encoders, mean ±\pm std over 3 seeds. Positive values would indicate non-linearly re-encoded information.
ModelDatasetchosen LRtunedcanonicalpenalty
CBraModAPAVA5e-50.8830.883+0.000
 BCI IV 2a2e-50.4400.363+0.077
 Lee20191e-40.8370.840−- 0.003
LaBraMAPAVA2e-50.9320.951−- 0.019
 BCI IV 2a2e-50.3190.309+0.010
 Lee20191e-40.6340.578+0.055
Appendix
Table 13: Tuned anchor for full fine-tuning (one seed): each model’s learning rate chosen from 2e-5, 5e-5, 1e-4 by minimum validation loss, against the canonical rate of the main matrix. The penalty is what the canonical protocol gives away; a positive value means tuning would have helped.
Learning rate (pretrained / random-init)pretrainedrandom-initΔ\Delta
5e-5 / 5e-4 (asymmetric, main matrix)0.663 ±\pm 0.0520.935 ±\pm 0.044−- 0.272
2e-5 / 2e-50.763 ±\pm 0.0780.947 ±\pm 0.023−- 0.184
5e-5 / 5e-50.663 ±\pm 0.0520.950 ±\pm 0.027−- 0.287
1e-4 / 1e-40.824 ±\pm 0.0590.948 ±\pm 0.028−- 0.124
Appendix
Table 14: LUNA on APAVA under full fine-tuning, re-run on one machine: the asymmetric schedule of the main matrix and three shared learning rates (AUROC at the best-validation epoch, mean ±\pm std over five seeds). The asymmetric row is a fresh run and differs from the Table 3 value within seed noise.
ModelDataset10%25%50%100%
CBraModAPAVA+0.136 ±\pm 0.067+0.052 ±\pm 0.036+0.083 ±\pm 0.022+0.185 ±\pm 0.000
CBraModBCI IV 2a+0.048 ±\pm 0.012+0.037 ±\pm 0.027+0.024 ±\pm 0.008+0.009 ±\pm 0.000
CBraModLee2019+0.010 ±\pm 0.007+0.013 ±\pm 0.013+0.013 ±\pm 0.012+0.016 ±\pm 0.000
LaBraMAPAVA+0.149 ±\pm 0.097+0.151 ±\pm 0.062+0.074 ±\pm 0.017+0.067 ±\pm 0.000
LaBraMBCI IV 2a+0.030 ±\pm 0.026+0.009 ±\pm 0.013+0.022 ±\pm 0.024+0.036 ±\pm 0.000
LaBraMLee2019−- 0.005 ±\pm 0.012+0.023 ±\pm 0.013+0.027 ±\pm 0.011+0.037 ±\pm 0.000
Appendix
Table 15: Pretrained minus random-init validation-selected peak as the labelled training fraction shrinks, mean ±\pm std over 3 seeds, from a separate feature-extraction run (one initialisation per cell). The original benchmarks use fixed splits and the linear probe is deterministic, so at the full budget the seed acts on nothing and the std is zero; seeds act only through the subsampling below it. A low-label advantage would show as a gap that grows to the left.
Modelcross peakcross finalwithin peakwithin final
CBraMod0.3000.2970.3440.329
LaBraM0.3270.3280.3540.369
EEGPT0.2810.2870.2780.305
LUNA0.3110.3260.3160.311
In-house (JEPA)0.3380.2890.3700.273
Appendix
Table 16: BCI IV 2a pretrained frozen probe (accuracy, four classes, chance 0.25) under the subject-independent and the within-subject protocol, same models, seed and hyperparameters; validation-selected peak and output stage.
TTboth-posfrozen-onlyfullft-onlyboth-negmixed
0.005173217
0.011431012
0.021230015
Appendix
Table 17: Four-mode label of the thirty model-dataset pairs at three thresholds TT (a gain with ∣Δ∣≤T|\Delta|\leq T counts as mixed). The frozen gain is the released test-argmax probe gain (five seeds), the fine-tuning gain the best-validation-epoch gain of Table 3 .
ModelAPAVABCI IV 2aLee2019TUSLTUAB
CBraMod0.900 / 0.8460.343 / 0.2940.545 / 0.5140.778 / 0.6840.861 / 0.788
LaBraM0.989 / 0.8520.343 / 0.3060.566 / 0.5110.754 / 0.6070.850 / 0.774
EEGPT0.831 / 0.9410.307 / 0.3430.531 / 0.5760.740 / 0.6420.821 / 0.836
LUNA0.991 / 0.9510.331 / 0.3290.574 / 0.5640.663 / 0.6170.838 / 0.846
LuMamba0.981 / 0.9580.332 / 0.3050.570 / 0.5830.647 / 0.6550.842 / 0.851
In-house (JEPA)0.638 / 0.6230.338 / 0.3320.708 / 0.7020.698 / 0.6710.717 / 0.717
Appendix
Table 18: Frozen linear probe on the five original benchmarks as released (test-argmax peak stage, five seeds): pretrained / random-init peak; last row the five-band log-power baseline. Within-subject and fixed splits give a zero seed std for the pretrained probe. Validation-selected values for the three leakage-free benchmarks are in Table 8 ; the in-house Lee2019 cells carry developer contact with that benchmark.
QuantityValueSource (repository path:line)
raw-EEG probe, Lee2019 (v5 stage diagnostic)0.599docs/supplementary/experiment_results_summary.md:44
v5 encoder-output probe (channel mixer)0.509docs/supplementary/experiment_results_summary.md:47
v5 mixer query std at start0.020docs/_v1/history.md:322
v5 mixer query std after 45K steps0.023docs/_v1/history.md:322
steps over which the query std stayed flat45Kdocs/_v1/history.md:322
v5 run stopped55Kdocs/_v1/history.md:322
Appendix
Table 19: Development-record values of the in-house model (Section 4.1 ). No raw output survives for these; each value is copied from the project document line shown, and the build fails when the value is no longer on that line.
More in this topic

EEG Foundation Models

Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.

See all 101 papers

Explore similar work

CardsList

  1. What EEG Foundation Models Encode: Dataset Identity and a Negative-Control Suite for Clinical Benchmarks

    Jul 27, 2026Marzieh ZareCross-Subject EEG DecodingElectroencephalography

    Universit´e Laval, School of Psychology, Quebec City, QC, Canada · NeuroGenis Inc., Toronto, ON, Canada

  2. Probing Spatial and Spectral Grounding in EEG Foundation Models

    Oct 21, 2026Kleanthis Avramidis, Ivan Tashev, Dimitra EmmanouilidouEEG Foundation Models

    Signal Analysis and Interpretation Lab, University of Southern California, USA · Microsoft Research, Redmond, WA, USA

  3. A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models

    May 27, 2026Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +2Neural Network GeneralizationEEG Decoding

    Signal Analysis and Interpretation Laboratory, University of Southern California, USA