Neuron-Level Architecture Growth: A Controlled Evaluation for EEG Time-Series Decoding
Authors: Adam Mounir, Stella Douka, Arnault H. Caillet, Bruno Aristimunha, Sylvain Chevallier
Organizations: Inria TAU – LISN, Université Paris-Saclay, France · Yneuro, Paris, France · Imperial College London, UK · University of California San Diego, CA, USA
Convolutional EEG decoders are trained at a fixed width, usually set by their authors on other data. Growing methods add neurons during training where the loss could decrease the most, but whether they improve compared to a reference width is untested on EEG. Here, we grow three convolutional backbones on 12 motor-imagery datasets under three protocols and compare each with its reference model per subject. The growing ShallowFBCSPNet scores 2.9 points above its reference model with only half the parameters (0.57x), SCCNet changes by at most 1.2 points. Deep4Net growing models show decreased accuracy, but they require adaptation that prevent to compare faithfully the results. These differences follow the selection step, which keeps a candidate neuron relying on a dynamic threshold from singular values decomposition. Overall, these results suggest that growth helps when its criterion can rank the candidate neurons, and that the rate of skipped neuron addition tells where a decoder can be grown small from scratch.
Figures & tables
Figure 1: (a) The desired update vgoal splits into a reachable part and a residual, the expressivity bottleneck, which growth covers. (b) On EEG, both read as scalp patterns (schematic).
Backbone
Growable layer
Width
Reference
Shallow
temporal → spatial
8→40
ShallowFBCSPNet, 40
SCCNet
spatial → spatio-temp.
4→22
SCCNet, 22
Deep
conv2a → conv2b
8→32
Deep4Net, 4 stages
Table 1: Backbones, the layer that grows and its width range. Only the first two rows are width-matched.
Protocol
Net
Δgrow [95 % CI]
MDE
ahead
Within-sess.
Shallow
+2.9
[+2.3,+3.5]
0.9
240/324
Deep
−4.0
[−4.9,−3.1]
1.3
107/324
SCCNet
+0.4†
[−0.2,+0.9]
0.8
172/324
Cross-sess.
Shallow
+2.8
[+1.9,+3.7]
1.3
85/116
Deep
−3.6
[−5.1,−2.2]
2.0
28/116
SCCNet
+1.2
[+0.4,+2.1]
1.2
72/116
Table 2: Growth is ahead for Shallow and behind for Deep in every protocol. Δgrow : growing minus reference score, in points. Brackets: 95 % bootstrap intervals over subjects. Ahead : subjects on which the growing backbone scores higher. MDE: minimum detectable effect at 80 % power. † marks the two cells whose effect falls below it.
Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.
Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian +2
Sigma Nova Paris, France · Goethe University Frankfurt Frankfurt, Germany
EEG decoding tasks can rely on different temporal dynamics and cross-channel relationships. We test whether specialized modules improve a fully fine-tuned EEG foundation model by augmenting CBraMod with cross-depth Attention Residuals (AttnRes) and two soft-routed expert banks. Across matched three-seed experiments on FACED, ISRUC, SEED-V, and PhysioNet-MI, the complete model changes mean balanced accuracy relative to full fine-tuning by -0.12, +1.27, +0.77, and -1.27 points, respectively. AttnRes alone improves mean balanced accuracy on three datasets, whereas adding experts on top of AttnRes helps only FACED and SEED-V. These gains come with substantial overhead: AttnRes requires 2.11 to 2.88x runtime and 1.78 to 2.67x memory, while the complete model requires 2.41 to 3.04x runtime and 1.86 to 2.85x memory. Overall, the added modules produce dataset-dependent, sometimes opposing effects rather than consistent gains over full fine-tuning.
Mingyang Jiang, Yamin Li, Daniel Moyer +3
Department of Computer Science, Vanderbilt University · Department of Biomedical Informatics and Data Science, Yale University
Intracranial electroencephalography (iEEG) is widely used to record electrical activity directly from electrodes inside the human brain, making it an attractive modality for neural decoding. However, progress in iEEG decoding, especially toward general-purpose foundation models, remains difficult to measure reliably: datasets are task- or institution-specific, limiting evidence of generalization across tasks and recording environments, and preprocessing choices can strongly influence performance, making model improvements difficult to distinguish from preprocessing gains. Thus, we introduce iMINDBench, an iEEG Multi-Institution Neural Decoding Benchmark that evaluates models on a shared suite of fifteen decoding tasks across three naturalistic movie-watching datasets. The benchmark additionally defines standardized preprocessing tracks and fixed evaluation splits to support consistent model comparisons. Using iMINDBench, we find that the evaluated pretrained systems generally outperform baselines within their respective preprocessing tracks, while strong spectral baselines remain competitive across institutional datasets. In our scaling study, adding up to 25 times more supervised data from other subjects or institutions yields only small or task-dependent gains over within-session training. Together, these findings highlight the need for iEEG models that improve on strong preprocessing baselines and make more effective use of data across subjects and institutions. Project website: https://imindbench.github.io/