Brain encoding models aim to predict neural responses to stimuli. The most widely used approach fits readouts from frozen pretrained backbones. Using neural data to train large models from scratch or fully fine-tune them after task-pretraining has proven difficult because neural data is costly and limited. We show that parameter-efficient fine-tuning (PEFT) enables us to effectively adapt very large pretrained backbones for brain encoding with a small number of trainable parameters. To make it computationally practical, we exploit the backbone's causal structure to predict multiple timepoints in a single pass instead of using sliding windows, significantly reducing feature-extraction cost. This enables efficient fine-tuning of multimodal backbones with up to 104B parameters, the largest model adapted for brain encoding to our knowledge. Trained with a low-rank adapter (LoRA) and a lightweight readout, our approach consistently improves brain encoding performance across nine diverse multimodal backbones. On the same Qwen3-Omni backbone as the prior state-of-the-art (SoTA), our method BrainLoRA achieves the best published performance on the Algonauts 2025 Challenge. We also show the adapted multimodal backbones remain interpretable: modality attribution recovers the canonical sensory topography that prior work often obtains by construction from separate unimodal encoders, and neural adaptation increases the contribution of audio inputs and cross-modal mixing within the backbone. To scale out the number of adapted models, we develop a multi-LoRA kernel that accelerates joint training of multiple LoRAs on a shared backbone. This enables an ensemble of 200+ adapted models that ranks first on the public Algonauts post-challenge leaderboard at the time of writing. Together, our results highlight PEFT as an efficient and powerful paradigm for brain encoding.
Figures & tables
Figure 1: From frozen backbones to PEFT, and three directions to scale PEFT for brain encoding. (a) The dominant paradigm: a frozen pretrained backbone with readouts trained for brain encoding. The backbone prior is strong, but it is not specialized for brain encoding. (b) We advocate using PEFT to adapt strong pretrained backbones, retaining the powerful prior while allowing flexible neural adaptation. This suggests three complementary scaling directions: Scale up to stronger priors leveraging more capable backbones; Scale down the size of trainable adapters for brain encoding with limited data; and Scale out the number of adapters with different specializations.
Figure 2: One causally masked pass replaces a sliding window. The baseline re-encodes an overlapping window per fMRI timestep; a single pass over the window gives features at every timestep.
Figure 3: Adaptation improves performance on nine tested backbones. All backbones share the same training configuration except for the feature extraction layer (App. 15 ). LoRA improves the performance consistently.
Trainable
Pearson r
Model
params
val
ID
OOD
Single model
TRIBE † ( d’Ascoli et al., 2026 )
–
–
0.303
0.196
MIRAGE † ( Gokce et al., 2026a )
1.1B
0.319
0.310
0.217
Linear + frozen backbone
52.3M
0.291
0.287
0.187
Linear + full finetuning
20.3B
0.323
0.320
0.217
Table 1: Main results on CNeuroMod/Algonauts 2025. Validation is on Friends season 6; ID and OOD are the challenge server’s scores on held-out Friends season 7 and OOD movies. Two rows within 0.00077 are not separated by seed variability (App. 10 ). Rows marked † are cited authors’ own numbers quoted from the papers. Our method BrainLoRA achieves a new state of the art.
Configuration
s06
Frozen backbone
0.2907
+ LoRA r16 (ours)
0.3263
without closed-form re-fit
0.3202
Rank 32
0.3268
Gated-MLP readout
0.3269
Cross-attention readout
0.3310
Table 2: Ablation. LoRA r16+linear readout, 60s context lead, closed-form re-fit. The first block builds our main configuration; the rest vary one choice from the main configuration. Rows within the 0.00077 seed spread are not separable.
Figure 4: Modality attribution recovers the canonical topography, and adaptation reweights and mixes the modalities. (a) Contribution to encoding accuracy of the video, audio, and text content, on our LoRA+linear model, left hemisphere (lateral above, medial below). (b) Modality contribution by region. The Shapley terms from the three modalities and a small layout residual sum to the region’s r . (c) Change in modality contribution from the frozen to the adapted model. Audio contribution increases the most, while text falls. (d) Share of the prediction carried from each stimulus modality (left) to the backbone’s output at each modality’s tokens (right), which is the readout’s input. Crossing ribbons are cross-modal mixing within the backbone, which nearly doubles after adaptation.
Figure 5: Scaling out: training a population of adapted models. (a) Training-step time for N rank-16 adapters on a shared backbone: one fused batched gather-matmul vs. N sequential runs. The fused step matches the cost of a single adapter on the same batch (App. 9 ). (b) Ensemble performance vs. ensemble size, averaged over random subsets of the 261 adapted models, which is nearly a straight line in the log-log space (App. 13 ). (c) Performance of 2,000 random 20-model ensembles vs. the performance predicted linearly on two properties of their members: mean single-model accuracy (quality) and mean pairwise residual correlation (diversity). Neither factor predicts ensemble accuracy alone ( R2 0.23 and 0.05), but together they predict the performance well ( R2=0.97 ).
Extract
MIRAGE head (1,123M)
Linear (52.3M)
Train
Stimulus design
(GPU-h)
frozen
LoRA
+ re-fit
LoRA
+ re-fit
s/step
(GPU-h)
Sliding window, full-res. media
∼ 700
0.3187
–
–
–
–
–
–
Sliding window, 336×224 media
∼ 700
0.3178
–
–
–
–
–
–
Single pass, whole clip
1.99
0.3149
–
–
–
–
–
–
Single pass, whole clip, 336×224
1.99
0.3137
–
–
–
–
–
–
+ window-local (149 s), no lead
1.99
0.3091
–
–
0.3148
0.3206
9.7
4.0
Table 3: From MIRAGE’s sliding window to a single causal pass. Friends s06 subject-mean Pearson over four subjects. The sliding-window protocol re-encodes the preceding stimulus at every timestep ( ∼ 2,800 forward passes per clip); a single causal pass encodes each clip or window once, and a context lead supplies the preceding stimulus that the first timesteps of a window would otherwise lack. Step time and training cost are for the linear readout. The sliding-window cost is the ∼ 700 GPU-hours reported by the baseline’s authors, which we reproduced on our hardware (App. 9 ). The frozen MIRAGE head was trained at a 30 s lead and has no longer-lead entries; with LoRA the head was trained at 30 and 60 s, and its re-fit re-solves the per-subject output layer in closed form. Both LoRA column pairs show seed 0. The linear readout has no whole-clip entries because whole-clip encoding is the limit of the context length sweep.
Figure 6: Accuracy within the window. Accuracy against TR position inside the 149 s window for three input designs, scored with the same frozen MIRAGE head on Friends s06 (606 complete interior windows; the band is the s.e.m. over windows for the no-lead configuration). The dashed box marks the first 10 s, where the designs differ. Accuracy here is the per-TR spatial correlation between predicted and measured 1000-parcel patterns, which is defined at a single time point; it is not comparable to the per-parcel headline metric. Without a lead, a window starts 0.110 below its plateau and needs about five TRs to reach it. With a 30 s lead it starts 0.048 below plateau, and with the whole clip 0.051. The first window of each clip is excluded because of a clip-onset transient.
Figure 7: Accuracy against cost. Held-out Friends S7 accuracy against trainable parameters (a) and against end-to-end compute including feature extraction (b) , both on log scales. Published baselines are squares and our models circles; an N -model ensemble is plotted at N times a member’s cost.
Frozen + 1,123M readout
LoRA + 52.3M readout
Hardware and precision
Device
1 × A800-80GB
Precision
bf16 backbone, fp32 patch-embed conv
Stimulus encoding
Protocol
sliding window
single causal pass
Forward passes / clip
2,758
5.1
Table 4: Compute for the two single-model configurations. Both columns use one A800-80GB, the same 331-clip corpus and the same 336 × 224 media cache (built once on CPU and shared). Sliding window re-encodes a capped prefix (60 s audio, 4 s video, 1,024 words) at every cutoff; single causal pass encodes each 149 s window once with a 60 s lead. Sliding-window extraction was measured over the 49 validation clips and scaled to 331 clips. Baseline training is our reproduction of the published head recipe. The closed-form re-fit is included in our column because all reported accuracies are post-re-fit. Extraction is 352 × cheaper and the end-to-end cost 83 × lower.
Figure 8: Multi-adapter kernel benchmark. Qwen3-Omni truncated at the feature extraction layer, rank-16 adapters on q/k/v/o , sequence length 128, one sample per adapter, one A800, median of 5 timed steps after 2 warm-up steps. (a) One fused batched gather-matmul over N different adapters (blue) matches the same batch through one adapter (grey); running the N jobs one at a time (orange) grows linearly in N . (b) The ratio of the fused pass to the single-adapter batch is 1.00× throughout, up to run-to-run noise of ±0.03 .
Rank
Readout lr
Runs
s06 (mean)
s.d.
16
3 ×10−4
4
0.3262
0.00007
32
3 ×10−4
1
0.3268
–
32
5 ×10−4
9
0.3257
0.00109
64
5 ×10−4
4 (+1 diverged)
0.3237
0.00128
128
5 ×10−4
0 (+2 diverged)
–
–
Table 5: LoRA rank. Shipped recipe otherwise unchanged ( Qwen3-Omni , linear readout, extraction layer 32, 60 s lead, LoRA lr 6×10−4 , 1,429 steps), Friends s06 subject-mean after the closed-form re-fit. Rank and readout learning rate were not varied independently; the matched comparison is the two rows at 3×10−4 . Diverged runs are counted in parentheses.
Feature-extraction layer
Frozen
Trained
16
0.2682
–
24
0.2839
–
30
0.2889
0.3252
31
0.2885
0.3260
32
0.2881
0.3263
33
0.2872
0.3258
Table 6: Feature-extraction layer. Frozen is the closed-form ridge readout on unadapted features (no adapter, no training); trained is the full recipe, one seed per layer, after the re-fit. The frozen screen peaks at layer 30; the trained runs at layers 31–34 lie within 0.0006 of one another, inside the 0.00077 seed spread of this configuration.
LoRA lr
Readout lr
s06 (pre-re-fit)
0
2e-5
0.2818
1e-3
1e-4
0.3071
1e-3
3e-4
0.3131
6e-4
1e-3
0.3076
6e-4
3e-4
0.3134
Table 7: Learning rates. Measured in an earlier configuration without context lead or closed-form re-fit, so these are pre-re-fit validation scores about 0.013 below the final recipe, comparable to each other but not to Tab. 1 . The per-modality projection uses 2× the readout rate. The first row freezes the adapter (readout-only control).
Step
200
400
600
800
1000
1200
1400
1600
1800
2000
2200
2400
2600
2800
2858
s06
0.2789
0.2856
0.2877
0.2921
0.2977
0.3007
0.3052
0.3009
0.2999
0.2996
0.3017
0.3019
0.3031
0.3030
0.3028
Table 8: A second epoch overfits. Validation score through a 2858-step (two-epoch) run, same configuration as Tab. 7 . The peak is at step 1400, the end of the first pass over the 1,429 training windows. Matched one- versus two-epoch pairs give −0.0147 at readout lr 3×10−4 , −0.0047 at readout lr 1×10−4 .
Prefix ends at
audio
vision
text
140 s
0.9579
0.9895
0.9960
160 s
0.9569
0.9908
0.9970
180 s
0.9553
0.9901
0.9969
200 s
0.9584
0.9906
0.9973
same span twice
1.0000
1.0000
1.0000
Table 9: Cosine between representations at shared timesteps, prefix pass against single pass, on friends_s01e01a with a 209 s span, feature extraction layer 32.
Configuration
Members
Seeds
shipped recipe
34
15 seeds
rank 32, readout lr 0.0005
27
13 seeds
lead 30
16
8 seeds
backbone Qwen3.8-27B , chunk 2, no audio
10
6 seeds
readout cross-attn (tokens), tokens stream
8
3 seeds
bag 0.6
6
2 seeds
Table 10: The 261-model pool. Each row is a distinct configuration; members differing only in random seed are grouped, with the seed count given. Entries list only where a member departs from the shipped recipe ( Qwen3-Omni backbone, linear readout, feature extraction layer 32, rank 16, 60 s lead, 1,429 steps). Counts sum to 261.
Method
Params
s06
Δ frozen
% of LoRA gain
Δ / M params
Frozen backbone (readout only)
0
0.2907
—
—
—
IA 3 , attention + FFN
0.057M
0.3127
+0.0220
62%
0.3833
IA 3 , + MoE experts
3.18M
0.3177
+0.0270
76%
0.0085
Bottleneck adapter, FFN only ( r=32 )
4.26M
0.3164
+0.0257
72%
0.0060
Bottleneck adapter, attention + FFN ( r=32 )
8.52M
0.3190
+0.0283
79%
0.0033
LoRA r=16 (ours)
8.91M
0.3263
+0.0356
100%
0.0040
Table 11: Three PEFT families at one protocol. Friends s06 subject-mean after the closed-form re-fit, at the main configuration ( Qwen3-Omni , linear readout, feature extraction layer 32, 60 s lead, 1,429 steps, one seed), with only the adaptation method changed. The column Params is backbone-side trainable parameters, read from each checkpoint; the 52.3M readout is common to every row and is excluded. The column of LoRA is the gain over the frozen backbone as a fraction of LoRA’s, and the last column is that gain per million adapter parameters. IA 3 learns three element-wise rescaling vectors per block (keys, values, FFN intermediate) and adds no weights; a bottleneck adapter inserts x+Wupgelu(Wdownx) after the attention sub-layer and the FFN (Houlsby) or after the FFN alone (Pfeiffer). Each family runs at its own screened rate ( 1×10−2 , 3×10−4 , 6×10−4 ), since a PEFT rate does not transfer between families.
Backbone
Params
Extraction layer
Frozen
+LoRA
Δ
+cross-attn
Phi-4-MM
4.7B
20
0.2521
0.2841
+0.0320
0.2888
Qwen2.5-Omni 3B
6.0B
30
0.2807
0.3139
+0.0331
0.3175
Gemma 3n E4B
7.8B
25
0.2631
0.3026
+0.0395
0.3049
LCO-Omni 7B
8.9B
26
0.2683
0.3199
+0.0517
0.3200
MiniCPM-o
9.4B
28
0.2594
0.3249
+0.0655
0.3265
Qwen2.5-Omni 7B
11.2B
24
0.2748
0.3178
+0.0431
0.3214
Table 12: The nine omni-modal backbones. Total parameters from each checkpoint’s own index ( Qwen3-Omni and Ming are mixture-of-experts, with 3B and 6B active per token); feature extraction layer is chosen per backbone. All arms share the recipe of § 5.2 : rank-16 LoRA, 60 s context lead, closed-form re-fit, Friends s06 subject-mean.
Figure 9: Cross-seed agreement. Mean agreement between adapters over the 15 seeds of the main configuration (105 pairs). (a) At successive levels of the model, from near zero for the weights to near one for the prediction. Levels below the first are cosines between the two seeds’ changes relative to the frozen model, averaged over the audio, video, and text streams. (b) Agreement at every decoder layer, per stream. It rises through the adapted stack, peaks at the feature extraction layer, and declines slowly through the 16 frozen layers above it.
Configuration
Training targets
s06
LoRA + linear (ours)
aligned
0.3263
Frozen backbone
aligned
0.2907
Frozen backbone
shifted
0.2617
LoRA + linear
shifted
0.1874
Table 13: Shifted-target negative control. Friends s06 subject-mean after the closed-form re-fit on aligned training data, one run per configuration. Differences are paired per parcel with a functional-block bootstrap interval; the last column counts the parcels on which the first configuration scores higher.
Held-out
frozen
3-subj., held out
3-subj., in-sample
4-subject
vs. frozen
vs. in-sample
vs. 4-subj.
sub-01
0.2875
0.3210
0.3215
0.3229
+0.0335
-0.0005
-0.0019
sub-02
0.2968
0.3286
0.3286
0.3301
+0.0317
-0.0000
-0.0015
sub-03
0.3130
0.3485
0.3490
0.3497
+0.0355
-0.0005
-0.0012
sub-05
0.2734
0.3034
0.3014
0.3025
+0.0301
+0.0021
+0.0010
Table 14: Transfer to an unseen subject. Validation score per subject, with the readout solved in closed form, under: the frozen backbone; an adapter trained on the other three subjects only (bold, the transfer); the mean of the three 3-subject adapters that did train on the subject (in-sample); and the shared 4-subject adapter. The last three columns compare the held-out score with each of the others.
subject-mean r
% of noise ceiling
frozen
LoRA
gain
frozen
LoRA
gain, points [95% CI]
Algonauts, validation
0.2907
0.3263
+0.0356
81.3
92.0
+10.7 [+10.0, +11.4]
NNDb, readout transferred
0.0991
0.1223
+0.0232
43.5
54.6
+11.1 [+10.5, +11.6]
NNDb, readout re-fitted
0.0914
0.1252
+0.0338
40.3
55.7
+15.4 [+14.6, +16.2]
Table 15: Transfer to a second dataset. The same LoRA and frozen-backbone models on Algonauts validation and on NNDb (20 participants, one film; different site, scanner, preprocessing and TR). Transferred applies the Algonauts readout unchanged; re-fitted solves a fresh readout per NNDb participant under cross-validation over contiguous film time. r columns are subject means over all 1,000 parcels. Ceiling columns give r as a percentage of each dataset’s noise ceiling, over the 922 (Algonauts) and 816 (NNDb) parcels with an estimable ceiling, with a 95% parcel-block bootstrap on the gain. Paired over the 800 parcels both datasets admit, transferred minus Algonauts is +0.55 [ +0.01 , +1.06 ] points.
Figure 10: Performance against the noise ceiling on NNDb. 20 participants, readout re-fitted per participant on the fixed backbone, over the 816 parcels with an estimable ceiling. (a) Subject-mean r per parcel against the corrected noise ceiling (dashed line). (b) Mean percentage of the ceiling per Yeo network.
calibration data from the new subject (min)
Algonauts, held-out subject
0
5
10
20
40
79
159
all
Ridge from scratch
–
0.1495
0.1967
0.2273
0.2493
0.2687
0.2869
0.3254
Low-rank deviation ( r=16 )
0.2574
0.2572
0.2578
0.2591
0.2603
0.2621
0.2642
–
Anchored ridge
0.2574
0.2597
0.2641
0.2718
0.2789
0.2889
0.2997
–
LoRA − frozen (anchored)
+0.0236
+0.0254
+0.0260
+0.0272
+0.0273
+0.0298
+0.0326
+0.0326
Table 16: Calibrating a new subject with little data. Validation r of a subject whose fMRI was never used for training the adapter, with its readout fitted on K windows (149 s each) of its own data, shown in minutes. Bold marks the best estimator at each budget. The last row is the gain of the adapted model over the frozen backbone under anchored ridge, which beats it for every subject and draw (20/20) and every NNDb participant (20/20) at every budget. Top: the four 3-subject adapters of Tab. 14 , mean over the four held-out subjects and five random draws of the calibration windows; all is the held-out column of Tab. 14 , solved on all of that subject’s training data. Bottom: the LoRA + linear model of Tab. 1 on NNDb, 20 participants, a fixed 7-window test segment and a contiguous calibration run, averaged over placements at the start and end of the film. Its 0 min scores therefore differ from Tab. 15 , which scores the whole film.
region
n
frozen
LoRA
gain [95% CI]
rel. %
Yeo 7 networks
DorsAttn
122
0.330
0.380
+0.0502 [+0.0468, +0.0533]
15
Vis
162
0.398
0.438
+0.0401 [+0.0364, +0.0440]
10
Cont
129
0.279
0.317
+0.0379 [+0.0347, +0.0412]
14
SalVentAttn
121
0.241
0.277
+0.0352 [+0.0319, +0.0397]
15
Default
212
0.318
0.351
+0.0330 [+0.0306, +0.0355]
10
Table 17: Gain from the adapter by region. Validation r with and without LoRA in the same configuration (the frozen arm trains the readout with the same rates and steps), over the n parcels of each region: the seven Yeo networks, then ten ROIs defined from parcel centroids. Arms are paired per parcel; gain intervals are functional-block bootstraps over parcels, and the last column is the gain relative to the frozen score. One run per arm; averaging the LoRA map over 15 seeds leaves both orderings unchanged.
Figure 11: Validation r by region, with and without the adapter. Frozen backbone and the same configuration with LoRA, over the n parcels of each region (Tab. 17 ); bars show the gain, printed at the right. (a) The seven Yeo networks. (b) Ten coordinate-defined ROIs. One run per configuration, paired per parcel; every gain’s 95% functional-block bootstrap interval excludes zero.
Figure 12: All subsets of the three input streams. Subject-mean Pearson r on the 250 validation windows used for attribution, with absent streams replaced by their training-mean embeddings. The top bar of (a) is the full model (dashed line). (a) Any single stream recovers most of the full score, and removing one costs far less than that stream achieves alone: the streams are largely redundant. (b) Cost of removing each stream, before and after adaptation. The video cost barely changes, the audio cost grows, and the transcript cost falls by more than a third. This removes a stream from a trained model and differs from the no-audio configuration of Tab. 2 , which is trained without audio throughout.
Figure 13: Audio contribution by region, before and after adaptation. Subject-mean contribution of the audio stream to encoding accuracy for the frozen and LoRA + linear models, with episode block-bootstrap intervals; labels give the relative change. Auditory cortex gains least; the gain is largest in regions that relied least on audio before adaptation.
pool
val. r
explained %
missed %
cross %
missed
main single model (LoRA + linear)
0.3263
60.5
36.2
3.3
0.0480
3-member mean
0.3262
60.8
36.5
2.6
0.0485
261-member uniform mean
0.3401
66.2
32.5 [30.0, 35.3]
1.3
0.0431
Table 18: What the pool still misses. The cross-subject partition of the stimulus-locked (inter-subject) covariance on the validation episodes, as a percentage of it: the part shared by the predictions (explained), the part shared by the residuals (missed) and the cross terms, for three pools; the interval is a functional-block bootstrap over parcels, and the last column is the missed covariance in units of a unit-variance parcel.
Block
δb (s)
wb (s)
covers (s)
full
clipped
zero
1
0.5
2.0
[0.5,2.5)
98
1
1
2
2.5
2.0
[2.5,4.5)
99
1
0
3
4.5
4.0
[4.5,8.5)
97
2
1
4
8.5
8.0
[8.5,16.5)
92
5
3
5
16.5
16.0
[16.5,32.5)
81
10
9
6
32.5
32.0
[32.5,64.5)
60
21
19
Table 19: Lag blocks. Each block is a (delay, width) pair in seconds. Block b of TR j uniformly averages the feature frames in [cj−(δb+wb)f,cj−δbf) , where cj is the TR’s block center and f=2 Hz. A block that reaches before the window start is clipped, or is a zero row if it lies entirely before it. The last three columns count, over the 100 TRs of a window, the rows at full width, clipped, and zero (158 of 800 rows are zero).
Backbone
Hugging Face repository
Revision
Phi-4-MM
microsoft/Phi-4-multimodal-instruct
467a5d77dcfa
Qwen2.5-Omni 3B
Qwen/Qwen2.5-Omni-3B
f75b40e3da20
Gemma 3n E4B
unsloth/gemma-3n-E4B-it
45e9fb1dd0e3
LCO-Omni 7B
LCO-Embedding/LCO-Embedding-Omni-7B
3d38f58aae12
MiniCPM-o
openbmb/MiniCPM-o-4_5
503e754207c9
Qwen2.5-Omni 7B
Qwen/Qwen2.5-Omni-7B
ae9e1690543f
Table 20: Backbone revisions. Hugging Face repository and commit for every backbone in Tab. 12 , as loaded by our runs.
Recent progress in task-optimized neural networks has established encoding models as a powerful tool for predicting brain responses to naturalistic stimuli, yet most existing approaches rely on unimodal representations. The emergence of omni-modal foundation models and rich multimodal neural datasets enables encoding models that jointly integrate visual, auditory, and linguistic information across subjects. We introduce MIRAGE, a brain encoding framework for predicting whole-brain fMRI responses to naturalistic audiovisual stimuli. MIRAGE achieves state-of-the-art performance via a native multimodal backbone and adaptive feature gating across layers. These representations are then combined with a transformer-based brain encoder and a subject-specific linear head over the cortical parcels. Controlled comparisons show that natively multimodal features consistently outperform post-hoc aggregation of independent unimodal features, across architectural levels and backbones. Beyond predictive accuracy, the learned attention weights are directly inspectable to interpret the modality-specific gating profile over the backbone, and each modality traces a distinct anatomical pattern across cortex. Together, these results propose adaptive layer-wise aggregation of natively multimodal features as a generalizable, interpretable, and accurate approach for whole-brain encoding.
Abdulkadir Gokce, Badr AlKhamissi, Martin Schrimpf
Scaling EEG foundation models requires pooling data across heterogeneous electrode montages, a prerequisite both for larger pretraining corpora and for downstream deployment. We present the first systematic comparison of four channel adaptation methods (Conv1d projection, spherical spline interpolation (SSI), source-space decomposition, and Riemannian re-centering) across five pretrained EEG foundation models (5M--157M parameters), five downstream tasks, and two training regimes with 10--15 random seeds each. We find that rigid-montage models (BENDR, Neuro-GPT) require external adaptation, while flexible models (EEGPT, CBraMod) match or exceed it natively when fine-tuned but benefit from external methods under frozen-encoder deployment. A probe-SFT asymmetry exists: external adaptation can cause severe negative transfer during fine-tuning of flexible models. The optimal method is architecture-dependent (Conv1d for BENDR, SSI/Riemannian for Neuro-GPT, source-space decomposition for depression detection), and 5M-parameter CBraMod outperforms models up to 31× larger on 4/5 datasets, consistent with independent findings that compact EEG-specific architectures can match larger models.
Kuntal Kokate, Bruno Aristimunha, Dung Truong +1
Swartz Center for Computational Neuroscience, Institute for Neural Computation, University of California San Diego, La Jolla, CA 92093, USA · Yneuro, Paris, France · Centre de Recherche Cerveau et Cognition, CNRS, Université Paul Sabatier, Toulouse, France
Recent electroencephalography (EEG) foundation models increasingly emphasize model scale, potentially limiting their deployment on resource-constrained devices and in everyday environments. We argue that transferable EEG features can instead be learned with a compact architecture and introduce BrainCodes, a lightweight foundation model pretrained through self-supervised learning on more than 800 hours of recordings from over 1,000 participants. Our architecture combines a quantization-based neural tokenizer with a spatiotemporal encoder that supports arbitrary electrode configurations. The pretraining objective explicitly encourages representations that capture spatial connectivity through channel positional encoding, and sensitivity to narrowband neural oscillations through an auxiliary training objective. Our model matches or surpasses the state of the art across diverse neurophysiological monitoring tasks, including sleep staging, attention assessment, and cognitive load estimation. BrainCodes transfers effectively to low-resource wearable devices while using up to 97% fewer trainable parameters than comparable architectures. Beyond predictive accuracy, our evaluation framework characterizes the geometry and invariances of learned EEG representations across measurable spatial, temporal, and spectral signal properties.
Kleanthis Avramidis, Ivan Tashev, Dimitra Emmanouilidou
Signal Analysis and Interpretation Lab, University of Southern California, Los Angeles, CA 90007, USA · Microsoft Research, Redmond, WA, USA