MAGE: Towards Generalizable Multi-timescale EEG Representations
Organizations: LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France · LIST3N, Université de Technologie de Troyes · German Research Center for Artificial Intelligence (DFKI), Germany · Institute for AI Innovation and Societal Impact, Hanoi University of Science and Technology, Vietnam
Abstract
Electroencephalography (EEG) data plays an important role in understanding human brain activity, with various major applications. While current approaches, which largely build on EEG foundation models, have demonstrated strong performance across datasets and tasks, they typically rely on a fixed temporal scale. This design choice limits their ability to capture characteristics of neural dynamic behavior, often spanning across multiple timescales, which are naturally captured in EEG recordings. To address this limitation, we propose MAGE (Multi-timescale Adaptive Generalization for EEG), a foundation model that dynamically adapts to capture and integrate information across different timescales. Specifically, MAGE combines two key mechanisms: (i) a novel multi-timescale attention mechanism, capable of learning representations across multiple temporal scales and (ii) a trainable aggregation module, that dynamically identifies and adapts to the most informative timescale of the input signal. Experimental results across seven datasets demonstrate that MAGE consistently outperforms existing EEG foundation models while using substantially fewer parameters. Moreover, our results support the hypothesis that different downstream tasks benefit from distinct temporal scales.
Figures & tables
| Dataset | Metrics | Medformer | LaBram | BIOT | EEGPT | CBraMod | CSBrain | MAGE | MAGE-froz |
| 1.4M | 5.8M | 3.2M | 25.0M | 4.0M | 8.9M | 0.6M | 0.6M | ||
| BCIC-IV-2a | bAcc | 52.68 ±6.06 | 49.75 ±8.73 | 49.92 ±11.74 | 50.49 ±6.38 | 46.09 ±6.63 | 55.11 ±8.63 | 56.78 ±6.08 | 60.81 ±10.05 |
| Kappa | 36.91 ±8.08 | 33.00 ±11.64 | 33.23 ±15.65 | 33.98 ±8.50 | 28.12 ±8.83 | 40.14 ±11.50 | 42.38 ±8.11 | 47.75 ±13.40 | |
| wF1 | 52.08 ±6.34 | 47.64 ±10.12 | 48.06 ±13.36 | 49.61 ±6.77 | 45.02 ±6.96 | 54.12 ±8.96 | 55.61 ±6.60 | 59.30 ±11.72 | |
| PhysioNet | bAcc | 59.01 ±3.39 | 58.82 ±4.40 | 55.02 ±3.33 | 53.80 ±2.91 | 52.22 ±4.01 | 60.62 ±3.14 | 62.69 ±3.59 | 64.08 ±3.51 |
| Kappa | 45.36 ±4.52 | 45.11 ±5.86 | 40.05 ±4.43 | 38.41 ±3.88 | 36.30 ±5.35 | 47.52 ±4.18 | 50.27 ±4.77 | 52.12 ±4.66 |
| Dataset | Metrics | 250ms only | 500ms only | 1s only | MAGE-router | MAGE |
| FACED | Balanced Acc | 50.93 ±2.82 | 49.78 ±1.80 | 46.65 ±1.84 | 49.87 ±2.45 | 51.30 ±3.11 |
| Cohen’s Kappa | 44.41 ±3.19 | 43.14 ±1.97 | 39.79 ±1.98 | 43.22 ±2.78 | 44.89 ±3.43 | |
| Weighted F1 | 50.43 ±2.91 | 49.20 ±1.75 | 46.36 ±1.53 | 49.36 ±2.45 | 50.97 ±2.96 | |
| Physionet-MI | Balanced Acc | 60.08 ±0.16 | 60.65 ±0.98 | 60.55 ±0.96 | 58.83 ±1.35 | 61.15 ±0.86 |
| Cohen’s Kappa | 46.81 ±0.21 | 47.58 ±1.30 | 47.41 ±1.28 | 45.13 ±1.79 | 48.23 ±1.15 | |
| Weighted F1 | 60.17 ±0.35 | 60.66 ±0.96 | 60.47 ±1.05 | 58.83 ±1.42 | 61.16 ±0.90 |
| Dataset | Metrics | MAGE | Inter Attention | Uniform Weight |
| FACED | Balanced Acc | 51.42 ±2.11 | 50.79 ±3.04 | 50.85 ±2.48 |
| Cohen’s Kappa | 44.97 ±2.31 | 44.27 ±3.33 | 44.43 ±2.73 | |
| Weighted F1 | 50.94 ±1.97 | 50.26 ±2.93 | 50.52 ±2.38 | |
| PhysioNet-MI | Balanced Acc | 62.69 ±3.59 | 60.48 ±0.87 | 60.90 ±1.02 |
| Cohen’s Kappa | 50.27 ±4.77 | 47.31 ±1.17 | 47.87 ±1.35 | |
| Weighted F1 | 62.71 ±3.54 | 60.68 ±0.87 | 61.09 ±1.05 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Space | Meaning |
|---|---|---|
| Number of channels | ||
| Number of timestamps | ||
| Number of patch sizes | ||
| Patch size at index | ||
| Number of segments at index | ||
| Feature dimension |
| Component | Hyperparameters | Settings |
| EEG sample | Channels | 19 |
| Time points | 6000 (30s @ 200Hz) | |
| Patch sizes | 50 100 200 (250ms 500ms 1s) | |
| Mask ratio | 0.7 | |
| Mask token | Constant | |
| Encoder | Layers (depth) | 6 |
| Dataset | Rate | Number of Channels | Duration | Labels |
| BCIC-IV-2A | 250 Hz | 22 | 4s | 4-class |
| PhysioNet-MI | 160 Hz | 64 | 4s | 4-class |
| FACED | 250 Hz | 32 | 10s | 9-class |
| Sleep-EDFx | 100 Hz | 2 | 30s | 5-class |
| TUAB | 250 Hz | 16 | 10s | 2-class |
| KaggleERN | 200 Hz | 56 | 2s | 2-class |
| Models | Training Paradigm | Sampling rate (Hz) | Params. (M) |
| Medformer | supervised | 200 | 1.4 |
| LaBraM | self- supervised | 200 | 5.8 |
| BIOT | self- supervised | 200 | 3.2 |
| EEGPT | self- supervised | 256 | 25.0 |
| CBraMod | self- supervised | 200 | 4.0 |
| CSBrain | self- supervised | 200 | 8.9 |
| Dataset | Metrics | MAGE | MAGE wo noise |
| FACED | Balanced Acc | 51.30 ±3.11 | 50.35 ±2.62 |
| Cohen’s Kappa | 44.89 ±3.43 | 43.80 ±3.02 | |
| Weighted F1 | 50.97 ±2.96 | 49.84 ±2.80 | |
| PhysioNet-MI | Balanced Acc | 61.15 ±0.86 | 61.57 ±1.15 |
| Cohen’s Kappa | 48.23 ±1.15 | 48.78 ±1.51 | |
| Weighted F1 | 61.16 ±0.90 | 61.66 ±1.14 |
| Dataset | Metrics | MAGE | MAGE-uniform pretrain |
| PhysioNet-MI | Balanced Acc | 61.15 ±0.86 | 61.15 ±0.27 |
| Cohen’s Kappa | 48.23 ±1.15 | 48.23 ±0.37 | |
| Weighted F1 | 61.16 ±0.90 | 61.31 ±0.25 | |
| FACED | Balanced Acc | 51.30 ±3.11 | 46.92 ±2.13 |
| Cohen’s Kappa | 44.89 ±3.43 | 40.19 ±2.39 | |
| Weighted F1 | 50.97 ±2.96 | 46.88 ±2.11 |
| Dataset | Metrics | Mask | Mask | Mask | Mask |
| FACED | Balanced Acc | 50.83 ±3.84 | 51.22 ±3.44 | 51.30 ±3.11 | 51.30 ±2.21 |
| Cohen’s Kappa | 44.31 ±4.24 | 44.83 ±3.77 | 44.89 ±3.43 | 44.80 ±2.49 | |
| Weighted F1 | 50.41 ±3.76 | 50.79 ±3.24 | 50.97 ±2.96 | 50.61 ±2.20 |
| Dataset | Metrics | Learnable Channel Embedding | RoPE |
| FACED | Balanced Acc | 51.30 ±3.11 | 50.99 ±3.00 |
| Cohen’s Kappa | 44.89 ±3.43 | 44.43 ±3.26 | |
| Weighted F1 | 50.97 ±2.96 | 50.34 ±2.85 |
| Dataset | Metrics | MAGE-500h | MAGE-2500h |
| PhysioNet-MI | Balanced Acc | 61.15 ±0.86 | 62.57 ±0.73 |
| Cohen’s Kappa | 48.23 ±1.15 | 50.12 ±0.98 | |
| Weighted F1 | 61.16 ±0.90 | 62.62 ±0.75 | |
| FACED | Balanced Acc | 51.30 ±3.11 | 52.02 ±2.22 |
| Cohen’s Kappa | 44.89 ±3.43 | 45.68 ±2.40 | |
| Weighted F1 | 50.97 ±2.96 | 51.62 ±1.94 |
EEG Foundation Models
Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.
See all 121 papers