Electroencephalography (EEG) foundation models learn transferable representations for brain-computer interfaces, but existing approaches treat tasks and labels as discrete targets and fail to use the semantic structure of natural-language task instructions to guide representation learning. We present LEAF, a foundation model for EEG-Language Alignment with Semantic Task Instruction and Querying. LEAF integrates task-aware semantic guidance to produce structured and linguistically aligned EEG embeddings, thereby enhancing decoding robustness and transferability. In the EEG pretraining stage, we introduce a joint Spectral-Temporal Reconstruction (STR) framework that captures the coupled spectral rhythms and temporal dynamics of EEG signals. STR applies randomized spectral perturbation to enhance frequency robustness and uses two complementary temporal objectives to learn both contextual and sequential structure. In the EEG-Language alignment stage, we propose the Instruction-conditioned Q-Former (IQF). This query-based cross-attention transformer injects instruction embeddings into EEG tokens and achieves semantic alignment with textual label embeddings through learnable queries. We evaluate LEAF on 16 downstream datasets spanning motor imagery, emotion recognition, steady-state visual evoked potentials, covert speech, and mental-workload decoding. Under supervised finetuning, LEAF achieves the best balanced accuracy on 12 of the 16 datasets and the strongest average performance across the evaluated paradigms. Further analyses show that instruction conditioning reduces intra-class variation, increases inter-class separation, and aligns EEG representations more closely with textual class prototypes. These findings support the use of task instructions as semantic priors for transferable EEG decoding. Code is available at https://github.com/Jiang-Muyun/LEAF.
Figures & tables
ID
Dataset
Split
Train
Validation
Test
(#cls) Textual labels
1
BCIC-IV-2a [ 35 ]
CS
Subj. 1–7
20% of train
Subj. 8–9
(4) Left, Right, Foot, Tongue
2
OpenBMI-MI [ 17 ]
CS
Subj. 1–42
20% of train
Subj. 43–54
(2) Right, Left
3
BCIC-Upperlimb [ 10 ]
CS
Subj. 1–11
20% of train
Subj. 12–15
(3) Cylin, Sphe, Lumbrical
4
SHU-MI [ 23 ]
CS
Subj. 1–20
20% of train
Subj. 21–25
(2) Right, Left
5
HighGamma [ 30 ]
CS
Subj. 1–10
20% of train
Subj. 11–14
(2) Left, Right
6
Cho2017 [ 3 ]
CS
Subj. 1–40
20% of train
Subj. 41–49
(2) Left, Right
Table 1: Downstream datasets, evaluation splits, and textual class labels. CS: cross-subject; CT: cross-trial. The class count is shown in parentheses.
Table 2: Task-specific finetuning performance on eight motor imagery datasets.
Table 3: Task-specific finetuning performance on eight datasets spanning emotion recognition, SSVEP, covert speech, and mental workload.
Figure 1: Effect of instruction conditioning on EEG representations: (A) intra-class variance, (B) inter-class distance, and (C) KDE-based embedding distributions with textual class prototypes.
Figure 2: Topography visualization on downstream datasets, illustrating the spatial EEG activation patterns captured by LEAF across different BCI task categories.
Figure 3: PCA visualization of Q-Former query evolution on four representative EEG decoding tasks. Queries diverge from their shared initialization and gradually form task-specific configurations across layers.
Figure 4: LEAF ablations: (A) EEG encoder depth, (B) Q-Former depth, (C) text encoder, and (D) instruction conditioning. Values are mean balanced accuracy.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Index
Target
Source channel names assigned to the target
1
Fpz
Fpz, AFpz
2
AFz
AFz, AFFz
3
Fz
Fz
4
FCz
FCz, FFCz
5
Cz
Cz, FCCz
6
CPz
CPz, CCPz
Appendix
Table 4: Complete source-to-template channel mapping for LEAF.
Figure 5: Standardized 65-channel EEG montage used by LEAF. Inputs from different datasets are mapped to this common 10–10 electrode configuration by source-channel assignment and averaging, with spatial interpolation for missing electrodes.
Group
Parameter
Value
Input
Channels
65
Sampling rate
200 Hz
Window length / stride
100 / 100 samples (0.5 s; non-overlapping)
Tokenizer
Convolutional filters
64
Temporal kernel / padding
(1,40) / (0,20)
Spatial kernel
(65,1)
Appendix
Table 5: LEAF architecture with the MPNet text encoder.
Training setting
Pretraining
Instruction tuning
Finetuning
Peak LR: CNN (Fast)
10−6
10−3
10−3
Peak LR: Transformers (Slow)
10−7
2×10−4
10−4
Warmup epochs
1
1
1
Total epochs
10
100
100
Global batch size
512
512
128
Weight decay
10−2
10−3
10−3
Appendix
Table 6: LEAF stage-specific training defaults.
Figure 6: LEAF finetuning from a shared instruction-tuned checkpoint: (a) replace the Q-Former with an MLP head; (b) retain the encoder and Q-Former and apply cross-entropy to cosine similarities with fixed text prototypes, adding no trainable parameters.
ID
Dataset
(A) Encoder + MLP (Main) New parameters: MLP head
(B) Q-Former + cosine CE New parameters: none
B.Acc
Kappa
B.Acc
Kappa
1
BCIC-IV-2a
0.6381±0.0120
0.5175±0.0160
0.6620±0.0136
0.5494±0.0182
2
OpenBMI-MI
0.8134±0.0031
0.6268±0.0062
0.8017±0.0018
0.6035±0.0036
3
BCIC-Upperlimb
0.5483±0.0038
0.3225±0.0057
0.4828±0.0038
0.2242±0.0058
4
SHU-MI
0.6134±0.0008
0.2268±0.0016
0.5897±0.0053
0.1792±0.0106
5
HighGamma
0.7982±0.0035
0.5964±0.0070
0.7899±0.0104
0.5797±0.0208
Appendix
Table 7: LEAF finetuning comparison for the two paths in Fig. 6 . B.Acc and Kappa are mean ± standard deviation; the average row gives equal-weight means across 16 datasets.
Figure 7: UMAP projection of EEG embeddings and textual class prototypes across downstream datasets under task-and-target instruction. Colors indicate datasets; stars indicate textual class prototypes.
Figure 8: Per-dataset views of the shared UMAP projection. Each panel highlights one downstream dataset in color against the embeddings of all datasets in gray; stars indicate textual class prototypes.
Freq.
Rand.
Causal
OpenBMI-MI
SEED
✓
0.8078 / 0.6156
0.6882 / 0.5487
✓
✓
0.7896 / 0.5792
0.6771 / 0.5298
✓
✓
0.8033 / 0.6066
0.6957 / 0.5499
✓
✓
✓
0.7851 / 0.5702
0.6760 / 0.5169
Appendix
Table 8: Ablation of STR masking strategies. Values are B-Acc / Kappa.
Number of Queries Nq
MI-OpenBMI
SEED
B.Acc
Kappa
B.Acc
Kappa
4
0.6913 ± 0.0025
0.3826 ± 0.0050
0.6896 ± 0.0071
0.5358 ± 0.0184
8
0.7278 ± 0.0026
0.4556 ± 0.0052
0.7011 ± 0.0068
0.5543 ± 0.0176
12
0.7213 ± 0.0025
0.4426 ± 0.0050
0.7004 ± 0.0069
0.5536 ± 0.0175
Appendix
Table 9: Direct inference ablation study on the number of learnable queries Nq evaluated on two datasets using Balanced Accuracy and Cohen’s Kappa.
Variant
OpenBMI-MI
SEED
B.Acc
Kappa
B.Acc
Kappa
Concat
0.6897 ± 0.0028
0.3794 ± 0.0056
0.6658 ± 0.0073
0.4976 ± 0.0189
FiLM-only
0.7052 ± 0.0026
0.4104 ± 0.0052
0.6817 ± 0.0070
0.5239 ± 0.0182
Cross-attention only
0.7136 ± 0.0025
0.4272 ± 0.0050
0.6919 ± 0.0069
0.5412 ± 0.0178
IQF (Ours)
0.7278 ± 0.0026
0.4556 ± 0.0052
0.7011 ± 0.0068
0.5543 ± 0.0176
Appendix
Table 10: Ablation of cross-modal interaction strategies in IQF, evaluated under multi-task instruction tuning.
Figure 9: Zero-shot direct inference on held-out motor-imagery datasets. Balanced accuracy on Dreyer2023 and Weibo2014 using MPNet-base or Qwen3-Embedding-4B as the frozen text encoder. The models are evaluated without adaptation under three levels of semantic guidance: no instruction, task instruction, and task instruction with target class names. The dashed horizontal line denotes chance performance (0.5).
Figure 10: Comparison of KDE visualization of features between incorrect and correct instructions.
Table 11: Direct inference evaluation across instruction sentences for motor imagery and emotion tasks. Summary rows report the mean and sample standard deviation across the ten listed paraphrases.
Recent advances in EEG foundation models have demonstrated the potential of large-scale pretraining to enable generalizable neural decoding across subjects, recording environments, and datasets. However, dominant pretraining paradigms face key challenges: masked autoencoding tends to prioritize low-level signal reconstruction over task-relevant semantics, while autoregressive modeling creates a mismatch between continuous neural dynamics and discrete token spaces. To address these challenges, new strategies are needed to effectively align continuous EEG representations with natural-language semantics and enable their integration with large language models. Accordingly, we propose Brain Latent Predictive Model (BLPM), an EEG-language foundation model that reformulates heterogeneous EEG decoding tasks as a continuous semantic embedding prediction problem. BLPM introduces a Continuous EEG Latent Predictive (CELP) encoder that learns transferable representations through latent target prediction. Building on these representations, a Multi-Query Semantic Decomposition (MQSD) module extracts task-relevant information and aligns continuous EEG representations with textual semantics within a shared latent space according to their semantic relationships. Experiments across multiple benchmarks demonstrate consistent generalization performance across diverse tasks, establishing continuous latent semantic prediction as an effective paradigm for EEG-language foundation models.
Myeong-Ju Cho, Hye-Bin Shin, Seo-Hyun Lee +1
Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea
Electroencephalography (EEG) decoding models often generalize poorly across datasets and subjects due to domain shifts in acquisition protocols and individual neurophysiology. We propose EEG-PRIME, a two-stage EEG foundation model for cross-dataset multi-task decoding. EEG-PRIME combines masked pretraining with prototype-aligned instruction tuning to enable instruction-aware and subject-invariant decoding across diverse BCI paradigms. During pretraining, an EEG encoder learns transferable representations through masked reconstruction with frequency-cutoff spectral augmentation. During instruction tuning, EEG-PRIME incorporates task-semantic, dataset-specific, and subject-invariant conditioning. The resulting conditioning signal modulates the Q-Former through Layer-wise Query Modulation, while frozen text embeddings of class labels serve as prototypes for cosine-similarity-based prediction across heterogeneous label spaces. Experiments on sixteen datasets covering motor imagery, emotion recognition, ADHD detection, covert speech, and mental workload show consistent improvements over state-of-the-art baselines and prior EEG foundation models under cross-subject settings. On two additional held-out datasets, EEG-PRIME achieves balanced accuracy comparable to within-session calibration models without target-domain optimization, calibration, or linear probing, demonstrating promising zero-shot transfer capability.
Shuailei Zhang, Muyun Jiang, Wei Zhang +5
College of Computing and Data Science, Nanyang Technological University, Singapore · Southeast University, China · Centre for AI in Medicine, Nanyang Technological University, Singapore
Electroencephalography (EEG) is a widely used tool for studying brain function, with applications in clinical neuroscience, diagnosis, and brain-computer interfaces (BCIs). Recent EEG foundation models trained on large unlabeled corpora aim to learn transferable representations, but their effectiveness remains unclear; reported improvements over smaller task-specific models are often modest, sensitive to downstream adaptation and fine-tuning strategies, and limited under linear probing. We hypothesize that one contributing factor is the reliance on signal reconstruction as the primary self-supervised learning (SSL) objective, which biases representations toward high-variance artifacts rather than task-relevant neural structure. To address this limitation, we explore an SSL paradigm based on Joint Embedding Predictive Architectures (JEPA), which learn by predicting latent representations instead of reconstructing raw signals. We introduce Laya, the first EEG foundation model based on LeJEPA. We show that latent prediction yields representations that encode semantic structure in EEG: Laya embeddings track clinically meaningful state changes such as seizure onset, are resilient to noise, and achieve the strongest mean clinical accuracy under frozen linear probing, with particular gains on tasks where relevant neural patterns are subtle and easily obscured by artifacts. Controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver of these gains.
Saarang Panchavati, Uddhav Panchavati, Hiroki Nariai +2
Department of Medical Informatics UCLA Los Angeles, USA · Department of Cognitive Science UCSD San Diego, USA · David Geffen School of Medicine UCLA Mattel Children’s Hospital Los Angeles, USA