SAMBA: Toward an EEG Foundation Model via Differential Mamba and Coordinate-Driven Spatial Embedding
Organizations: Emotiv Research, Melbourne, Australia · Emotiv Research, Sydney, Australia
Abstract
Modeling long-context electroencephalography (EEG) signals is critical for learning generalizable EEG representations, as EEG recordings are acquired at high sampling rates and often span extended durations that capture evolving neural states beyond short task- specific windows. Transformer-based models have shown promise in modeling short sequences of a few seconds; however, their quadratic complexity limits scalability to longer contexts. Moreover, substantial variability in electrode montages across datasets and recording devices poses a major challenge for building EEG models that generalize across domains. We propose SAMBA, a self-supervised learning framework with a Mamba- based U-shaped encoder-decoder architecture, which effectively captures long-range temporal dependencies and montage variability in EEG data. Leveraging the inherent ability of Mamba in processing long context sizes, we introduce: (1) Temporal Semantic Random Masking for semantic-level sequence reconstruction, (2) a Multi-Head Differential Mamba module to suppress redundancy and emphasize salient temporal structures, and (3) a Coordinate-Driven Spatial Embedding that learns unified embeddings in a three- dimensional Euclidean space, enabling robustness across devices. Experiments on thirteen EEG datasets across diverse tasks, electrode configurations, and sequence durations demonstrate that SAMBA achieves competitive or improved performance compared to recent state-of-the-art methods, while maintaining low memory consumption and efficient inference. The learned spatial weight maps exhibit close alignment with task-relevant neurophysiological regions, highlighting the interpretability of SAMBA. The code is available at: https://github.com/Jiazhen-Hong/SAMBA
Figures & tables
| Models | Crowdsourced | DriverDistraction | STEW | |||
|---|---|---|---|---|---|---|
| ACC | AUROC | ACC | AUROC | ACC | AUROC | |
| BENDR | 70.46 4.14 | 0.7056 0.04 | 68.40 3.08 | 0.5521 0.03 | 63.03 1.07 | 0.6303 0.01 |
| MAEEG | 75.21 2.11 | 0.7501 0.02 | 68.37 2.60 | 0.5529 0.02 | 67.99 1.86 | 0.6858 0.02 |
| BIOT | 76.23 4.56 | 0.7633 0.04 | 63.93 1.28 | 0.6333 0.01 | 67.54 2.08 | 0.6769 0.04 |
| EEG2Rep | 81.66 2.93 | 0.8167 0.03 | 76.88 2.55 | 0.6559 0.02 | 69.04 1.04 | 0.6910 0.01 |
| SAMBA (Linear) | 86.72 0.02 | 0.9336 0.00 | 77.97 0.90 | 0.7092 0.01 | 70.62 0.03 | 0.7704 0.00 |
| Models | Model Size | Balanced ACC | AUROC |
|---|---|---|---|
| SPaRCNet Jing et al. (2023) | 0.79M | 78.96 0.18 | 0.8676 0.0012 |
| ContraWR Yang et al. (2023b) | 1.6M | 77.46 0.41 | 0.8456 0.0074 |
| CNN-LSTM Li et al. (2022) | 2.4M | 78.48 0.38 | 0.8569 0.0051 |
| CNN-Transformer Peh et al. (2022) | 3.2M | 77.77 0.22 | 0.8461 0.0013 |
| BIOT Yang et al. (2023a) | 3.2M | 79.59 0.57 | 0.8815 0.0043 |
| ST-Transformer Song et al. (2021) | 3.5M | 79.66 0.23 | 0.8707 0.0019 |
| Initialization | Crowdsourced | DriverDistraction | STEW | |||
|---|---|---|---|---|---|---|
| ACC | AUROC | ACC | AUROC | ACC | AUROC | |
| Random | 69.66 | 0.7943 | 63.38 | 0.6377 | 57.21 | 0.6033 |
| In-domain | ( 23.58) | ( 0.1850) | ( 16.80) | ( 0.0468) | ( 13.68) | ( 0.1829) |
| Cross-domain | ( 16.17) | ( 0.1414) | ( 10.66) | ( 0.0313) | ( 16.39) | ( 0.1951) |
| Dataset | # of | # of | Sampling | Time | SAMBA-E | SAMBA-T | EEGPT | LaBraM | BIOT | MMM | CBraMod | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Class | Chan. | Rate (Hz) | (s) | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | |
| PhysionetMI | 5 | 64 | 160 | 3 | 28.29 | 0.3070 | 27.41 | 0.3006 | 23.77 | 0.2631 | 35.79 | 0.3575 | 27.14 | 0.2832 | 19.33 | 0.2229 | 20.32 | 0.2983 |
| GrosseWentrup | 2 | 128 | 500 | 7 | 53.50 | 0.5349 | 57.83 | 0.5781 | 50.67 | 0.5037 | 53.67 | 0.5344 | 50.33 | 0.4530 | 50.83 | 0.5080 | 51.17 | 0.5098 |
| BNCI2014-001 | 4 | 22 | 250 | 4 | 32.00 | 0.3220 | 30.57 | 0.3033 | 25.41 | 0.2500 | 28.53 | 0.2807 | 27.31 | 0.2388 | 24.71 | 0.2461 | 25.98 | 0.1436 |
| P300-A | 2 | 64 | 240 | 0.8 | 56.26 | 0.6181 | 58.49 | 0.6366 | 55.79 | 0.6129 | 54.15 | 0.6234 | 49.48 | 0.6117 | 55.20 | 0.6102 | 49.88 | 0.6254 |
| P300-B | 2 | 64 | 240 | 0.8 | 57.76 | 0.6303 | 59.22 | 0.6424 | 54.00 | 0.5986 | 56.17 | 0.6326 | 50.88 | 0.6868 | 53.32 | 0.5853 | 50.49 | 0.6532 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset Source | EEG Task | Dataset | # of Channels | Sampling Rate (Hz) | Time Steps | Samples |
| Emotiv a | Emotion | DREAMER | 14 | 128 | 256 | 17,246 |
| Eye State | Alpha | 14 | 128 | 256 | 11,866 | |
| Eye State | Crowdsourced | 14 | 128 | 256 | 12,296 | |
| Mental Workload | STEW | 14 | 128 | 256 | 28,512 | |
| Distraction | DriverDistraction | 14 | 128 | 256 | 66,197 | |
| Attention | Attention | 14 | 128 | 256 | 21,894 |
| Model Variant | Modified Component | ACMSE | B-ACC (%) | AUROC |
|---|---|---|---|---|
| SAMBA | – | 82.64 | 0.9054 | |
| Setting 1 | TSR random mask | 79.79 ( 2.84) | 0.8629 ( 0.04) | |
| Setting 2 | TSR SSP | 79.27 ( 3.34) | 0.8689 ( 0.04) | |
| Setting 3 | MDM Mamba2 block | 80.01 ( 2.63) | 0.8687 ( 0.04) | |
| Setting 4 | Remove residual in MDM | 81.41 ( 1.23) | 0.8867 ( 0.02) | |
| Setting 5 | Remove | 80.18 ( 2.46) | 0.8677 ( 0.04) |
| Dataset | SAMBA-E | SAMBA-E (mean) | SAMBA* | SAMBA* (mean) | ||||
|---|---|---|---|---|---|---|---|---|
| Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | Acc. | W-F1 | |
| PhysionetMI | 28.29 | 0.3070 | 29.83 | 0.3226 | 27.41 | 0.2985 | 29.04 | 0.3140 |
| GrosseWentrup | 53.50 | 0.5349 | 57.72 | 0.5772 | 57.50 | 0.5749 | 57.78 | 0.5776 |
| BNCI2014-001 | 32.00 | 0.3220 | 33.28 | 0.3285 | 29.54 | 0.2954 | 32.25 | 0.3198 |
| P300-A | 56.26 | 0.6181 | 53.35 | 0.5927 | 58.01 | 0.6323 | 56.22 | 0.6179 |
| P300-B | 57.76 | 0.6303 | 53.37 | 0.5929 | 57.23 | 0.6260 | 54.75 | 0.6051 |
EEG Foundation Models
Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.
See all 120 papers