Backbone-Informed Adaptation after EEG Pretraining: A Capacity-Controlled Study
Organizations: Department of Computer Science, Vanderbilt University · Department of Biomedical Informatics and Data Science, Yale School of Medicine
Abstract
Electroencephalography (EEG) foundation models learn representations that combine information across channels and time. This structure motivates adapters that model the same relationships during downstream learning. However, it is not clear whether such a design outperforms other candidate adapters. Moreover, in order to arrive at a fair comparison between candidate adapters, we also need to consider their trainable parameters. Here, we ask whether a backbone-informed (native) adapter makes better use of a given capacity by comparing it against a token-wise multilayer perceptron control that transforms each feature vector independently. We approximately match their trainable parameter counts and keep the pretrained encoder fixed. Across two EEG foundation models and tasks covering emotion recognition, sleep staging, and EEG event classification, the native adapter shows no consistent advantage. For one backbone, differences between the designs are small relative to variation across runs. For the other, the token-wise control performs better on emotion recognition and EEG event classification. On sleep staging, the native adapter's advantage over the token-wise control is reduced when we also train the component that processes sequences of EEG segments. The relative performance of the two adapters depends on the backbone, task, and downstream training protocol. These findings motivate evaluating backbone-informed adapters against capacity-matched alternatives.
Figures & tables
| Dataset | Task | Classes | Comparisons | Split unit | ||
|---|---|---|---|---|---|---|
| FACED | Emotion | 32 | 10 | 9 | RQ1, RQ2 | Subject/source file |
| ISRUC | Sleep staging | 6 | 30 | 5 | RQ1, RQ2 | Subject |
| TUEV | Event classification | 16 | 5 | 6 | RQ1, RQ2 | Record/prefix |
| PhysioNet-MI | Motor imagery | 64 | 4 | 4 | RQ2 | Subject |
| SEED-V | Emotion | 62 | 1 | 5 | RQ2 (CBraMod) | Within-subject trial |
| (a) Balanced accuracy | ||||||
|---|---|---|---|---|---|---|
| Backbone | Dataset | Probe | Native | Token-wise | RQ1 | RQ2 |
| CBraMod | FACED | |||||
| ISRUC | ||||||
| TUEV | ||||||
| LaBraM | FACED | |||||
| ISRUC † | ||||||
| Dataset | Metric | Native | Branch | Token | Native Branch | Branch Token |
|---|---|---|---|---|---|---|
| FACED | BA | |||||
| FACED | F1 | |||||
| ISRUC | BA | |||||
| ISRUC | F1 | |||||
| TUEV † | BA | |||||
| TUEV † | F1 |
| Frozen sequence encoder | Trainable sequence encoder | |||
|---|---|---|---|---|
| Condition | BA | F1 | BA | F1 |
| No-adapter baseline | ||||
| Native adapter | ||||
| Token-wise control | ||||
| Branch-preserving MLP | ||||
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Backbone | Dataset | Seed | Native BA | Token-wise BA | BA | Native F1 | Token-wise F1 | F1 |
|---|---|---|---|---|---|---|---|---|
| CBraMod | FACED | 42 | .3952 | .3873 | .3951 | .3873 | ||
| CBraMod | FACED | 1024 | .3875 | .3853 | .3879 | .3860 | ||
| CBraMod | FACED | 3407 | .3902 | .3913 | .3891 | .3891 | ||
| CBraMod | ISRUC | 42 | .7541 | .7547 | .7472 | .7506 | ||
| CBraMod | ISRUC | 1024 | .7446 | .7500 | .7408 | .7464 | ||
| CBraMod | ISRUC | 3407 | .7570 | .7520 | .7575 | .7482 |
| Dataset | Input shape | Split definition | Train / Val. / Test |
|---|---|---|---|
| FACED | Subjects 000–079 / 080–099 / 100–122 | 6,720 / 1,680 / 1,932 | |
| ISRUC | Subjects 1–80 / 81–90 / 91–100 | 3,559 / 468 / 435 | |
| TUEV | Record/prefix split; original evaluation set used for testing | 68,712 / 15,220 / 29,421 | |
| SEED-V | Within-subject trials 0–4 / 5–9 / 10–14 | 34,432 / 42,960 / 40,352 | |
| PhysioNet-MI | 70 / 19 / 20 subjects | 6,300 / 1,734 / 1,803 |
| Setting | Architecture | Optimization | Capacity | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Backbone | Dataset | Condition | Heads | Adapter LR | Head LR | Batch | Training epochs | Adapter params | |
| CBraMod | FACED | Native | 64 | 4 | 64 | 50 | 59,610 | ||
| CBraMod | FACED | Token-wise | 148 | – | 64 | 50 | 59,949 | ||
| CBraMod | ISRUC | Native | 64 | 4 | 8 | 20 | 59,610 | ||
| CBraMod | ISRUC | Token-wise | 148 | – | 8 | 20 | 59,949 | ||
| CBraMod | TUEV | Native | 64 | 4 | 8 | 20 | 59,610 | ||
| Backbone | Dataset | Downstream head | Trainable params |
|---|---|---|---|
| CBraMod | FACED | Flatten; MLP; ELU and dropout after the two hidden layers | 128,404,009 |
| CBraMod | ISRUC | Per-epoch flatten; projection and GELU; one Transformer layer ( , 4 heads, feed-forward width 2,048, dropout ); classifier | 21,587,461 |
| CBraMod | TUEV | Flatten; MLP; ELU and dropout after the two hidden layers | 16,202,406 |
| LaBraM | FACED | Mean patch pooling; frozen LayerNorm; classifier | 1,809 |
| LaBraM | ISRUC | Per-epoch mean pooling; frozen LayerNorm; newly initialized, frozen Transformer layer ( , 4 heads, feed-forward width 800); classifier | 1,005 |
| LaBraM | TUEV | Mean patch pooling; frozen LayerNorm; classifier | 1,206 |
| Backbone | Class | Native R | Token-wise R | R | Native F1 | Token-wise F1 | F1 | |
|---|---|---|---|---|---|---|---|---|
| CBraMod | SPSW | 567 | .039 | .039 | .065 | .061 | ||
| CBraMod | GPED | 4677 | .353 | .356 | .465 | .468 | ||
| CBraMod | PLED | 1998 | .310 | .323 | .369 | .385 | ||
| CBraMod | EYEM | 329 | .585 | .585 | .697 | .700 | ||
| CBraMod | ARTF | 2204 | .542 | .539 | .364 | .368 | ||
| CBraMod | BCKG | 19646 | .901 | .906 | .872 | .876 |
| (a) Balanced accuracy | ||||||||
|---|---|---|---|---|---|---|---|---|
| Backbone | Dataset | Probe | Native | Token-wise | Generic | LoRA | Upper | Dense |
| CBraMod | FACED | .378 .002 | .391 .004 | .388 .003 | .390 .003 | .417 .007 | .403 .005 | .572 .011 |
| CBraMod | ISRUC | .756 .003 | .752 .006 | .752 .002 | – | – | – | – |
| CBraMod | TUEV | .466 .031 | .455 .022 | .458 .008 | .461 .012 | .555 .009 | .557 .031 | .620 .024 |
| LaBraM | TUEV | .3658 ‡ | .392 .018 | .431 .001 | .437 .013 | .572 .023 | .541 .020 | – |
| Dataset | Condition | Regime | BA | Macro-F1 | Qualification | |
|---|---|---|---|---|---|---|
| FACED | Frozen probe | Frozen | .159 .000 | .150 .000 | 3 | Probe |
| FACED | Native C+P, rank 64 | Frozen | .161 .010 | .141 .005 | 3 | Primary |
| FACED | Token-wise | Frozen | .195 .003 | .194 .002 | 3 | Primary control |
| FACED | LoRA | LoRA | .282 .032 | .280 .032 | 3 | Context |
| FACED | Upper-2 | Upper layer | .297 .001 | .297 .002 | 3 | Context |
| ISRUC | Frozen probe | Frozen | .652 .002 | .635 .002 | 3 | Probe |
| Dataset | Seed | B BA | A B BA | B T BA | B F1 | A B F1 | B T F1 |
|---|---|---|---|---|---|---|---|
| FACED | 42 | ||||||
| FACED | 1024 | ||||||
| FACED | 3407 | ||||||
| ISRUC | 42 | ||||||
| ISRUC | 1024 | ||||||
| ISRUC | 3407 |
| Backbone | Native BA | MLP BA | BA | Native F1 | MLP F1 | F1 | BA MLP | F1 MLP |
|---|---|---|---|---|---|---|---|---|
| CBraMod | .4548 .0223 | .4637 .0168 | .0315 | .4720 .0216 | .4781 .0063 | .0235 | 2/3 | 2/3 |
| LaBraM | .3916 .0182 | .4311 .0033 | .0208 | .3676 .0260 | .4336 .0030 | .0288 | 3/3 | 3/3 |
| Backbone | Dataset | Axes | Epochs | Batch | Backbone/upper LR | Head/adapter LR | WD |
|---|---|---|---|---|---|---|---|
| CBraMod | FACED | C+P | 50 | 64 | .05 | ||
| CBraMod | TUEV | C+P | 20 | 8 | .05 | ||
| CBraMod | SEED-V | C | 40 | 32 | .03 | ||
| LaBraM | PhysioNet-MI | C+P | 40 | 64 | .05 |
| Backbone | Dataset | Frozen | Trainable | ||
|---|---|---|---|---|---|
| BA | F1 | BA | F1 | ||
| CBraMod | FACED | ||||
| CBraMod | ISRUC | – | – | ||
| CBraMod | TUEV | ||||
| LaBraM | TUEV ‡ | – | – | ||
| LaBraM | PhysioNet-MI | ||||
| Setting | Condition | Same | Median shift | Mean shift |
|---|---|---|---|---|
| CBraMod–FACED | Native | 2/3 | 0.0 | 4.0 |
| CBraMod–FACED | Token-wise | 2/3 | 0.0 | 1.7 |
| CBraMod–TUEV | Native | 0/3 | 3.0 | 4.0 |
| CBraMod–TUEV | Token-wise | 0/3 | 7.0 | 6.0 |
| LaBraM–TUEV | Native | 0/3 | 6.0 | 5.3 |
| LaBraM–TUEV | Token-wise | 0/3 | 3.0 | 3.0 |
| Backbone | Dataset | BA | F1 | BA | F1 |
|---|---|---|---|---|---|
| CBraMod | PhysioNet-MI | 1/3 | 1/3 | ||
| LaBraM | PhysioNet-MI | 3/3 | 2/3 | ||
| CBraMod | SEED-V | 0/3 | 0/3 |
| Backbone | Dataset | Seed | Balanced accuracy | Macro-F1 | ||||
|---|---|---|---|---|---|---|---|---|
| Native | Probe | Native | Probe | |||||
| CBraMod | FACED | 42 | .3952 | .3751 | .3951 | .3762 | ||
| CBraMod | FACED | 1024 | .3875 | .3791 | .3879 | .3796 | ||
| CBraMod | FACED | 3407 | .3902 | .3790 | .3891 | .3808 | ||
| CBraMod | ISRUC | 42 | .7541 | .7543 | .7472 | .7466 | ||
| CBraMod | ISRUC | 1024 | .7446 | .7544 | .7408 | .7509 | ||
| Condition | Seed | Epoch | BA | F1 |
|---|---|---|---|---|
| No-adapter baseline | 42 | 25 | 0.7580 | 0.7477 |
| No-adapter baseline | 1024 | 23 | 0.7598 | 0.7536 |
| No-adapter baseline | 3407 | 23 | 0.7567 | 0.7493 |
| Native adapter | 42 | 23 | 0.7672 | 0.7598 |
| Native adapter | 1024 | 23 | 0.7739 | 0.7660 |
| Native adapter | 3407 | 26 | 0.7726 | 0.7665 |
| Contrast | BA | BA | Macro-F1 | F1 |
|---|---|---|---|---|
| Native Token-wise | 3/3 | 2/3 | ||
| Native Branch | 3/3 | 3/3 | ||
| Branch Token-wise | 1/3 | 1/3 | ||
| Native Baseline | 3/3 | 3/3 |
| Dataset | Seed | Condition | Ratio mean | Ratio SD | |||
|---|---|---|---|---|---|---|---|
| FACED | 42 | Native | 0.0308 | 0.0279 | – | 0.6723 | 0.0433 |
| FACED | 42 | Token-wise | – | – | 0.0376 | 0.7350 | 0.0458 |
| FACED | 1024 | Native | 0.0304 | 0.0291 | – | 0.6841 | 0.0416 |
| FACED | 1024 | Token-wise | – | – | 0.0368 | 0.7278 | 0.0440 |
| FACED | 3407 | Native | 0.0307 | 0.0281 | – | 0.6680 | 0.0426 |
| FACED | 3407 | Token-wise | – | – | 0.0390 | 0.7290 | 0.0435 |
Electroencephalography
Electroencephalography (EEG) studies brain-generated electrical activity to understand cognition and support diagnosis, monitoring, and brain-computer interfaces.
See all 200 papers