Brain-Token Learning: Microstate-Based Tokenization and Multi-Scale Interaction for Long-Horizon EEG Sequence Modeling
Authors: Weishan Ye, Yue Pan, Li Zhang, Gan Huang, Zhen Liang
Organizations: School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China, also with the Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
Electroencephalography (EEG) provides a non-invasive window into dynamic brain activity, yet modeling long-horizon EEG sequences remains challenging due to their high temporal complexity, substantial variability across subjects, and the lack of biologically meaningful sequence representations. Existing tokenization strategies, such as fixed-window and patch-based representations, discretize EEG signals according to artificial temporal boundaries, which may disrupt intrinsic brain-state dynamics. In this work, we propose Brain-Token Learning, a neuroscience-inspired framework that introduces Brain Tokenization for long-horizon EEG sequence modeling. Instead of partitioning EEG signals into predefined temporal segments, Brain Tokenization represents EEG as sequences of recurrent microstate-derived brain tokens, where each token corresponds to a quasi-stable large-scale brain state with variable temporal duration. Based on these biologically grounded tokens, we further develop a multi-scale token interaction module consisting of Latent State Aggregation and State Transition Modeling to jointly capture global brain-state context and local microstate transitions. We evaluate Brain-Token on five heterogeneous EEG datasets, including the newly collected long-horizon NeuroLong dataset and four affective or clinical EEG datasets (SEED, DEAP, MDD, and NSSI). Extensive experiments demonstrate that Brain-Token consistently outperforms conventional CNN/LSTM architectures, Transformer-based models, and domain adaptation methods across diverse EEG scenarios. Further analysis verifies the effectiveness of microstate-based tokenization and multi-scale interaction for learning robust and interpretable EEG representations. These results establish Brain-Token as a biologically grounded tokenization paradigm for long-horizon EEG sequence modeling.
Figures & tables
Fig. 1: Comparison of tokenization paradigms across natural language processing, computer vision, and EEG.
Fig. 2: Comparison between conventional EEG patch tokens and the proposed Brain Tokens.
Fig. 3: Overview of the proposed Brain-Token learning framework, including brain tokenization, token embedding, token interaction, and token classification.
Attribute
NeuroLong
SEED
DEAP
MDD
NSSI
Label Type
Subjective / Stimulus
Stimulus
Subjective
Clinical attribute
Clinical behavior
Subjects
49
15
32
56
104
Trials / Conditions
12 videos
15 videos
40 videos
EC / EO
B1 / B2
Classes
2/3/5 + 2
3
2
2
2
Channels
63
62
32
19
63
Sampling Rate
1000 Hz
200 Hz
512 Hz
256 Hz
500 Hz
TABLE I: Summary of heterogeneous EEG datasets and evaluation protocols used in this study.
Fig. 4: Label construction strategies and label definitions across heterogeneous EEG datasets, including NeuroLong, SEED, DEAP, MDD, and NSSI.
Fig. 5: Dataset-specific microstate topographic templates obtained from NeuroLong, SEED, DEAP, MDD, and NSSI datasets.
Method
CNN [ 82 ]
LSTM [ 18 ]
Transformer [ 4 ]
DANN [ 83 ]
DeepCORAL [ 84 ]
Brain-Token
NeuroLong (Subjective Labels)
2-class
68.71 ± 11.97
71.60 ± 11.16
82.99 ± 07.89
83.50 ± 08.67
83.16 ± 08.50
88.27 ± 07.32
3-class
60.20 ± 08.63
62.07 ± 09.38
76.36 ± 11.08
76.36 ± 11.21
75.34 ± 10.51
78.57 ± 09.82
5-class
52.89 ± 11.11
53.91 ± 11.55
65.82 ± 12.51
65.48 ± 13.98
65.98 ± 13.02
67.35 ± 11.89
NeuroLong (Stimulus Labels)
2-class
67.01 ± 02.36
69.56 ± 05.96
90.65 ± 08.01
90.14 ± 08.19
91.50 ± 07.98
91.16 ± 05.93
TABLE II: Comparison of different methods on heterogeneous EEG classification tasks under subject-independent evaluation (accuracy % ± std.).
Method
LSA Only
STM Only
w/o Token Interaction
w/o Domain Adaptation
Full Model
NeuroLong (Subjective Labels)
2-class
86.39 ± 08.20
78.57 ± 10.91
78.91 ± 10.12
88.43 ± 07.86
88.27 ± 07.32
3-class
77.04 ± 10.45
70.24 ± 08.91
69.22 ± 10.94
77.38 ± 10.51
78.57 ± 09.82
5-class
67.86 ± 13.98
60.88 ± 12.16
61.05 ± 12.64
66.33 ± 11.90
67.35 ± 11.89
NeuroLong (Stimulus Labels)
2-class
87.07 ± 06.31
88.78 ± 08.17
86.56 ± 09.93
90.65 ± 05.98
91.16 ± 05.93
TABLE III: Ablation results of key components in Brain-Token on heterogeneous EEG datasets (accuracy % ± std.).
Target Dataset
Task
NeuroLong-MS
SEED4MS
DEAP4MS
MDD4MS
NSSI6MS
NeuroLong
Subject labels (2-class)
88.27 ± 07.32
69.39 ± 11.47
81.12 ± 10.48
82.31 ± 08.52
82.14 ± 08.07
Subject labels (3-class)
78.57 ± 09.82
55.44 ± 11.36
60.88 ± 11.19
61.90 ± 08.91
59.01 ± 07.69
Subject labels (5-class)
67.35 ± 11.89
46.94 ± 13.33
36.22 ± 12.32
36.73 ± 12.48
36.90 ± 12.82
Stimulus labels (2-class)
91.16 ± 05.93
66.16 ± 12.87
66.50 ± 14.33
73.30 ± 16.32
67.01 ± 13.46
SEED
Stimulus labels (3-class)
74.22 ± 08.39
82.22 ± 09.64
69.78 ± 08.02
68.89 ± 09.64
74.67 ± 08.15
DEAP
Subjective labels (2-class)
64.22 ± 06.54
61.88 ± 06.37
66.72 ± 05.21
63.91 ± 05.93
64.14 ± 06.54
TABLE IV: Cross-dataset transferability of microstate templates across heterogeneous EEG classification tasks (accuracy, %). Each column represents the source microstate template, and each row denotes the target dataset. Underlined values indicate self-template settings, while bold values denote the best performance in each row.
Method
Parameters
Model Size
FLOPs
CNN
86.98K
0.33 MB
2.58 G
LSTM
166.53K
0.64 MB
4.92 G
Transformer
100.48K
0.38 MB
118.15 G
DANN
109.12K
0.42 MB
118.15 G
DeepCORAL
100.48K
0.38 MB
118.15 G
Brain-Token (Ours)
206.40K
0.79 MB
13.61 G
TABLE V: Comparison of computational complexity among different EEG sequence modeling methods.
Electroencephalography (EEG) studies brain-generated electrical activity to understand cognition and support diagnosis, monitoring, and brain-computer interfaces.
Learning universal representations from electroencephalogram (EEG) signals is a cutting-edge approach in the field of neuroinformatics and brain-computer interfaces (BCIs). Conventionally, EEG is treated as a multivariate temporal signal, where time- or frequency-domain features are extracted for representation learning. This paper investigates a simple yet effective EEG representation, i.e., microstates. Microstates represent the building blocks of brain activity patterns at a microscopic time scale. We build a universal microstate tokenizer from a large medical EEG dataset by clustering continuous EEG signals into sequences of discrete microstates. The microstate tokenizer is then adopted universally across a series of downstream tasks, including sleep staging, emotion recognition, and motor imagery classification. Experimental results show that EEG representation learning with microstates outperforms traditional time-domain and frequency-domain features under different models and across different tasks. Further analysis shows that microstates offer greater interpretability and scalability, thereby opening up applications in both cognitive neuroscience and clinical research.
Xinyang Tian, Ruitao Liu, Ziyi Ye +3
Institute for Interdisciplinary Information Sciences, Tsinghua University Beijing, China · Institute of Trustworthy Embodied AI, Fudan University Shanghai, China · Beijing Five Seasons Medical Technology Co., Ltd. Beijing, China
Self-supervised foundation models have recently shown strong potential for electroencephalogram (EEG)-based analysis. However, existing approaches struggle to capture the inherently multi-scale temporal structure of EEG signals, where local neural patterns and long-range dependencies jointly encode task-relevant information. This limitation hampers cross-scale representation learning and generalization across diverse downstream tasks. To address this challenge, we propose MSBraM, a Multi-Scale self-supervised Brain foundation Model designed to learn hierarchical EEG representations. MSBraM follows a two-stage pretraining framework. First, a multi-scale neural tokenizer discretizes raw EEG signals into semantic codes at different temporal resolutions via vector-quantized reconstruction. Second, the model is pretrained to predict masked codes using a curriculum multi-scale masking strategy, progressively integrating fine-grained local patterns with global temporal context. We pretrain MSBraM on over 2,400 hours of EEG data and evaluate it across 10 downstream tasks on 12 public datasets. Extensive experiments show that MSBraM achieves superior performance on other state-of-the-art pretrained models, demonstrating strong generalization and transferability. These results indicate that explicitly modeling multi-scale temporal dynamics is critical for effective EEG foundation models.
Electroencephalography (EEG) is recorded continuously over hours, with relevant dynamics spanning timescales from milliseconds to hours. Most EEG foundation models nevertheless process fixed windows independently, limiting their ability to capture information encoded in long-timescale dynamics. State-space architectures enable persistent recurrent processing, but long-range information remains implicitly compressed in recurrent states. We present NeurDuo-EEG, a causal EEG foundation model with channel-resolved persistent memory. NeurDuo-EEG introduces multi-timescale memory management with learned consolidation and selective retrieval, enabling persistent modelling of continuous EEG with fixed-size state. It is pre-trained on 3,955 hours of EEG from 17 public datasets using multichannel autoregressive prediction of discrete spectral codes. Across three short-window and two long-sequence downstream tasks, NeurDuo-EEG achieves the best performance on four of five benchmarks, including all three short-window tasks and seizure detection, where AUC-PR improves from 0.285 to 0.471 over the strongest non-NeurDuo baseline. NeurDuo-EEG also remains competitive on sleep staging and supports efficient streaming inference, with nearly constant per-chunk latency as the available history grows to one hour. Notably, the Small variant achieves this with only 4.7M backbone parameters. These results demonstrate the value of persistent, multi-timescale modelling for both long-sequence and short-window EEG analysis. Our code is available at https://github.com/YifaNNW/NeurDuo-EEG.
Yifan Wang, Haiping Liu, Yang Cui +13
University of Manchester · ETH Zürich · Shanghai Jiao Tong University +5