LI-DSN: A Layer-wise Interactive Dual-Stream Network for EEG Decoding
Authors: Chenghao Yue, Zhiyuan Ma, Zhongye Xia, Xinche Zhang, Yisi Zhang, Xinke Shen, Sen Song
Organizations: School of Life Sciences, Tsinghua University, Beijing 100084, China · School of Biomedical Engineering, Tsinghua Laboratory of Brain and Intelligence, and IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen 518055, China · Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, 100084, China
Electroencephalography (EEG) provides a non-invasive window into brain activity, offering high temporal resolution crucial for understanding and interacting with neural processes through brain-computer interfaces (BCIs). Current dual-stream neural networks for EEG often process temporal and spatial features independently through parallel branches, delaying their integration until a final, late-stage fusion. This design inherently leads to an "information silo" problem, precluding intermediate cross-stream refinement and hindering spatial-temporal decompositions essential for full feature utilization. We propose LI-DSN, a layer-wise interactive dual-stream network that facilitates progressive, cross-stream communication at each layer, thereby overcoming the limitations of late-fusion paradigms. LI-DSN introduces a novel Temporal-Spatial Integration Attention (TSIA) mechanism, which constructs a Spatial Affinity Correlation Matrix (SACM) to capture inter-electrode spatial structural relationships and a Temporal Channel Aggregation Matrix (TCAM) to integrate cosine-gated temporal dynamics under spatial guidance. Furthermore, we employ an adaptive fusion strategy with learnable channel weights to optimize the integration of dual-stream features. Extensive experiments across eight diverse EEG datasets, encompassing motor imagery (MI) classification, emotion recognition, and steady-state visual evoked potentials (SSVEP), consistently demonstrate that LI-DSN significantly outperforms 13 state-of-the-art (SOTA) baseline models, showcasing its superior robustness and decoding performance. The code will be publicized after acceptance.
Figures & tables
Fig. 1: Exploration of dual-stream interaction strategies for EEG analysis, where Ft and Fs raw temporal and spatial features respectively, Ft′~ and Fs′~ denote enhanced features after interaction. (a) Dual-stream architecture with cross-flow interactions between temporal and spatial streams. (b) Baseline with no cross-stream interaction before fusion. (c) Interaction outputs integrated only into the spatial stream. (d) Interaction outputs integrated only into the temporal stream. (e) Bidirectional interaction feeding both streams.
Paradigm
Dataset
Number of Subjects
Number of EEG Channels
Sampling Rate (Hz)
Trial Length (seconds)
Number of Total Trials
Task Types
MI
BNCI2014001
9
22
250
4
1,296
left/right hand
BNCI2014002
14
15
512
5
1,400
right hand/both feet
BNCI2014004
9
3
250
5
1,080
left/right hand
Zhou2016
4
14
250
5
409
left/right hand
Emotion Recognition
SEED
15
62
200
2
7,920
negative/positive
FACED
123
32
250
4
10,332
negative/positive
TABLE I: Summary of datasets and task types across different paradigms.
Fig. 2: The overall architecture of the proposed LI-DSN. The raw EEG signals are first processed by parallel Temporal and Spatial Tokenizers to extract initial embeddings. These embeddings are then fed into N blocks, where each block consists of a Feed-Forward Network (FFN) followed by the Temporal-Spatial Integration Attention (TSIA) module to enable progressive cross-stream interaction. The refined features are subsequently aggregated by the Adaptive Fusion module and passed through an MLP classification head to generate outputs for various EEG tasks (e.g., MI classification, Emotion Recognition, SSVEP). The detailed architectures of the five key components, namely the Spatial Tokenizer, Temporal Tokenizer, FFN Block, TSIA and Adaptive Fusion, are illustrated in the sub-panels.
Task
Dataset
SCNN
DCNN
EEGNet
FBCNet
CTNet
MSVTNet
IFNet
Conformer
DBConformer
LI-DSN
CO
BNCI2014001
73.57 ± 2.37
59.29 ± 1.64
69.05 ± 1.02
68.97 ± 1.28
73.49 ± 2.12
74.60 ± 3.03
77.94 ± 0.87
78.57 ± 0.72
80.16 ± 1.83
80.95 ± 1.09
BNCI2014002
79.07 ± 2.01
64.07 ± 2.68
66.07 ± 2.76
69.50 ± 1.02
71.00 ± 1.78
78.93 ± 1.42
78.29 ± 1.73
76.21 ± 1.52
79.14 ± 0.16
81.57 ± 1.33
BNCI2014004
68.15 ± 1.42
62.41 ± 1.58
68.43 ± 2.21
65.46 ± 1.56
71.57 ± 1.32
71.30 ± 1.50
73.43 ± 1.71
72.04 ± 1.61
75.09 ± 1.54
75.19 ± 0.23
Zhou2016
75.03 ± 6.18
78.03 ± 2.41
80.13 ± 3.35
63.33 ± 2.31
76.81 ± 5.14
70.32 ± 1.42
81.70 ± 2.14
73.87 ± 4.45
82.49 ± 2.42
84.65 ± 1.87
CV
BNCI2014001
78.22 ± 0.21
61.59 ± 1.12
71.86 ± 1.03
74.74 ± 0.72
75.97 ± 0.77
81.28 ± 0.72
82.62 ± 0.92
83.62 ± 1.01
83.66 ± 0.42
83.79 ± 0.69
BNCI2014002
81.24 ± 0.52
67.17 ± 0.32
68.90 ± 1.37
74.53 ± 0.72
75.50 ± 0.93
80.91 ± 0.52
80.21 ± 0.52
79.30 ± 0.31
81.36 ± 0.33
82.57 ± 0.44
TABLE II: MI classification accuracies (%) of different methods on the BNCI2014001, BNCI2014002, BNCI2014004, and Zhou2016 datasets under CO, CV, and LOSO scenarios. ( Bold : best; Underline : second best)
Method
SEED
FACED
ACC(±std)↑
F1(±std)↑
ACC(±std)↑
F1(±std)↑
GCBNet
68.42±17.21
52.15±35.68
57.38±5.23
69.12±5.36
EEGNet
69.55±11.28
65.84±22.08
62.60±4.35
76.45±3.31
TSception
66.23±18.12
62.05±28.26
61.92±8.83
70.19±3.58
CTNet
68.71±11.62
63.75±25.41
62.08±4.20
76.10±3.17
MSVTNet
64.36±8.21
59.86±21.78
62.91±6.17
76.79±4.84
TABLE III: Emotion recognition results of different methods on the SEED and FACED datasets. ( Bold : best; Underline : second best)
Method
Nakanishi2015
Kalunga2016
ACC(±std)↑
F1(±std)↑
ACC(±std)↑
F1(±std)↑
EEGNet
96.09±0.22
95.94±0.22
69.73±0.68
67.87±0.94
CTNet
96.19±0.11
96.06±0.14
69.18±0.45
67.05±0.30
MSVTNet
90.46±0.27
90.37±0.32
70.84±0.49
68.35±0.54
Conformer
94.16±0.34
94.16±0.34
65.17±0.42
63.10±0.52
Deformer
96.26±0.33
96.19±0.34
70.01±0.35
68.97±0.51
TABLE IV: SSVEP results of different methods on the Nakanishi2015 and Kalunga2016 datasets. ( Bold : best; Underline : second best)
Model Variant
2014001
2014002
Zhou2016
w/o Positional Embedding
74.79±1.18
74.67±0.14
84.87±1.19
w/o Interaction (Late Fusion)
73.25±1.32
72.05±0.15
81.51±1.38
w/o TSIA (Simple Concat)
72.37±1.72
70.56±0.23
80.31±1.65
w/o Cosine Gate
75.22±1.15
75.29±0.08
84.01±1.21
w/o Ele-Pos Embedding
75.65±1.20
75.34±0.12
83.92±1.26
w/o Adaptive Fusion
73.69±1.45
73.90±0.14
82.14±1.41
TABLE V: The ablation study of the proposed module and strategy on three MI classification datasets.
Fig. 3: Comparison of integration method under the LOSO protocol across four MI classification datasets.
Fig. 4: Impact of architectural hyperparameters on performance under the LOSO protocol. (a) and (c) illustrate the effect of block depth (Ns,Nt) on the SEED and FACED datasets respectively. (b) and (d) display the effect of feature embedding width on the SEED and FACED datasets.
Fig. 5: Comparison of model performance, parameters, and MFLOPS on BNCI2014001, with model size indicated by circle area.
Fig. 6: Visualization of the TSIA matrices and channel-temporal interactions. Subfigures (a), (b), (e), and (f) correspond to the Nakanishi2015 dataset, while (c), (d), (g), and (h) correspond to the Kalunga2016 dataset. Specifically, (a) and (c) illustrate the spatial attention heatmap of SACM; (b) and (d) display the spatial attention heatmap of TCAM. Regarding the feature interaction, (e) and (g) depict the channel-temporal feature matrices before TSIA interaction, whereas (f) and (h) visualize the matrices after interaction.
Fig. 7: Visualization of saliency maps across three representative datasets. Subfigures (a)-(d) correspond to BNCI2014001, (e)-(h) represent the Kalunga2016 dataset, and (i)-(l) depict the SEED dataset. Within each group, the first three subfigures display activation patterns of representative subjects to illustrate individual variability, while the last subfigure presents the mean saliency map averaged across all subjects in the respective dataset.
In real-world applications of noninvasive electroencephalography (EEG), specialized decoders often show limited generalizability across diverse tasks under subject-independent settings. One central challenge is that task-relevant EEG signals often follow different temporal organization patterns across tasks, while many existing methods rely on task-tailored architectural designs that introduce task-specific temporal inductive biases. This mismatch makes it difficult to adapt temporal modeling across tasks without changing the model configuration. To address these challenges, we propose DSAINet, an efficient dual-scale attentive interaction network for general EEG decoding. Specifically, DSAINet constructs shared spatiotemporal token representations from raw EEG signals and models diverse temporal dynamics through parallel convolutional branches at fine and coarse scales. The resulting representations are then adaptively refined by intra-branch attention to emphasize salient scale-specific patterns and by inter-branch attention to integrate task-relevant features across scales, followed by adaptive token aggregation to yield a compact representation for prediction. Extensive experiments on five downstream EEG decoding tasks across ten public datasets show that DSAINet consistently outperforms 13 representative baselines under strict subject-independent evaluation. Notably, this performance is achieved using the same architecture hyperparameters across datasets. Moreover, DSAINet achieves a favorable accuracy-efficiency trade-off with only about 77K trainable parameters and provides interpretable neurophysiological insights. The code is publicly available at https://github.com/zy0929/DSAINet.
Zhiyuan Ma, Zeyuan Li, Zihao Qiu +6
School of Biomedical Engineering, Tsinghua Laboratory of Brain and Intelligence, and IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · School of Basic Medical Sciences and Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China · Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing 100084, China +1
Decoding brain states requires models that capture both the evolution of neural activity and interactions among groups of brain regions. Existing EEG methods often treat recordings as multivariate time series or represent functional connectivity with pairwise graphs, leaving dynamic higher-order interactions largely unmodeled. We introduce the Temporal Simplicial Neural Network (T-SNN), which represents EEG recordings as sequences of evolving simplicial complexes. By combining simplicial convolutions with recurrent updates, T-SNN jointly learns higher-order interactions and their temporal evolution. On the seven-class SEED-VII emotion recognition task, T-SNN outperforms convolutional, recurrent, graph-based, and Transformer methods in both trial-wise and cross-subject evaluations. Incorporating eye-movement features further improves performance, demonstrating the framework's potential for multimodal brain-state decoding.
Nikita Malik, Shubhajit Roy, Mohit Kataria +4
Bharti School of Telecommunication Technology and Management Indian Institute of Technology, Delhi · Department of Computer Science & Engineering Indian Institute of Technology, Gandhinagar · Yardi School of Artificial Intelligence Indian Institute of Technology, Delhi +4
Motor imagery (MI) brain--computer interfaces (BCIs) have emerged as a promising approach for establishing flexible communication pathways between the human brain and external devices , particularly for individuals affected by stroke or neurodegenerative disorders. Reliable decoding of motor-imagery electroencephalography (MI-EEG) remains challenging because EEG recordings contain substantial noise and exhibit complex, weakly informative relationships with the underlying brain activity. Although deep learning provides an effective means of learning representations directly from EEG signals, its application to MI-EEG feature learning remains comparatively limited. This study introduces a hybrid deep-learning architecture that integrates a convolutional neural network (CNN) with a bidirectional long short-term memory (bi-LSTM) network. The CNN is used to learn high-level spatial and temporal representations directly from raw MI-EEG recordings, whereas the bi-LSTM models temporal dependencies and relationships among the extracted features. The proposed approach is evaluated using both a publicly available dataset and a privately acquired dataset obtained with an EEG acquisition system. The experimental results indicate that the CNN&bi-LSTM architecture provides robust performance for both two- and three-class motor-imagery classification and demonstrates promising subject-independent decoding capability across the evaluated methods.
Athanasios Karagounis
School of Science, Department of Digital Industry Technologies GR34400, Psachna, Greece