Organizations: Carnegie Mellon University, Pittsburgh, PA, USA · Kyung Hee University, Yongin, South Korea · Korea Advanced Institute of Science and Technology, Deajeon, South Korea · University of Pittsburgh, Pittsburgh, PA, USA
Cross-subject generalization in EEG-based brain-computer interfaces (BCIs) remains challenging due to individual variability in neural signals. We investigate whether spectral representations offer more stable features for cross-subject transfer than temporal waveforms. Through correlation analyses across three EEG paradigms (SSVEP, P300, and Motor Imagery), we find that spectral features exhibit consistently higher cross-subject similarity than temporal signals. Motivated by this observation, we introduce ASPEN, a hybrid architecture that combines spectral and temporal feature streams via multiplicative fusion, requiring cross-modal agreement for features to propagate. Experiments across six benchmark datasets reveal that ASPEN is able to dynamically achieve the optimal spectral-temporal balance depending on the paradigm. ASPEN achieves the best unseen-subject accuracy on three of six datasets and competitive performance on others, demonstrating that multiplicative multimodal fusion enables effective cross-subject generalization.
Figures & tables
Figure 1: ASPEN architecture. Raw EEG is processed through parallel temporal and spectral streams, combined via multiplicative fusion before classification.
Figure 2: Cross-session (left) and cross-subject (right) correlation comparison between temporal and spectral representations. Spectral features exhibit consistently higher cross-subject similarity across all datasets.
Figure 3: Detailed view of temporal stream, spectral stream, and multiplicative fusion components.
Dataset
nperseg
noverlap
nfft
Best Fusion
Best Acc
Mult Acc
Δ
Wang 2016 SSVEP
256
128
256
Global Attn
72.76
69.47
-3.3
Lee2019 SSVEP
256
128
1024
Bilinear
86.68
85.71
-1.0
BI2014b P300
32
16
512
Bilinear
73.52
66.12
-7.4
BNCI2014-009 P300
128
120
256
Multiplicative
89.82
89.82
–
BNCI2014-001 MI
512
256
512
Multiplicative
30.73
30.73
–
Lee2019 MI
32
30
32
Multiplicative
75.70
75.70
–
Table 1: Ablation study summary. Best STFT parameters and fusion strategy per dataset, selected by unseen-subject accuracy. Best Acc = best fusion accuracy on held-out subjects (%), Mult Acc = multiplicative fusion accuracy (%), Δ = absolute difference. Global Attn = Global Attention, Bilinear = Low-rank Bilinear.
Method
Task
Dataset
Cross-
EEGNet
EEGConf.
MultiDiff.
TSformer-SA
CTNet
SPEN
ASPEN
SSVEP
Wang2016
Session
81.96 ± 6.02
56.32 ± 4.15
91.74 ± 1.62
47.96 ± 4.16
88.37 ± 4.20
85.71 ± 1.57
73.98 ± 1.46
Subject
74.25 ± 5.27
49.95 ± 2.81
87.95 ± 2.56
39.93 ± 5.25
83.60 ± 0.82
78.82 ± 6.41
67.20 ± 4.83
Lee2019
Session
95.04 ± 0.35
92.00 ± 0.94
93.67 ± 0.67
79.98 ± 6.60
95.36 ± 0.61
70.58 ± 0.80
95.50 ± 0.33
Subject
86.51 ± 0.09
81.99 ± 1.05
85.04 ± 0.76
63.38 ± 6.43
87.25 ± 0.69
58.51 ± 0.98
87.53 ± 0.29
P300
BI2014b
Session
64.74 ± 2.71
78.84 ± 3.46
81.46 ± 4.56
82.25 ± 3.04
76.06 ± 6.45
84.15 ± 0.93
77.95 ± 5.52
Table 2: Final results across tasks and models. Cross-subject generalization accuracy (%). Bold indicates best cross-subject performance per dataset. Mean ± STD across three seeds.
Figure 4: Stream contributions and feature correlation ( ρ ) across datasets. Low correlation values confirm that streams capture distinct information.
Figure 5: Grad-CAM visualization of feature importance for P300 classification. Correct prediction (top) shows focused attention on physiologically relevant low-frequency bands. Misclassification (bottom) reveals scattered attention towards high-frequency noise artifacts.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Dataset
Subj.
Chan.
Cls.
Bandpass
Epoch
SR
SSVEP : Frequency-
Wang2016
35
64
26
6–90 Hz
1.0s
250 Hz
tagged visual decoding
Lee2019
54
62
4
6–90 Hz
1.0s
250 Hz
P300 : Binary target
BI2014b
38
32
2
0.1–30 Hz
1.0s
256 Hz
vs. non-target ERP
BNCI2014_009
10
16
2
1–24 Hz
1.0s
256 Hz
MI : Motor
BNCI2014_001
9
22
4
4–40 Hz
4.0s
250 Hz
imagery decoding
Lee2019
54
22
2
4–40 Hz
4.0s
250 Hz
Appendix
Table 3: EEG Dataset Specifications and Preprocessing Parameters. Z-score normalization was applied to all datasets to standardize signal amplitudes across different subjects and sessions, ensuring that high-voltage artifacts or individual physiological variations do not disproportionately influence model training.
val
seen
seen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
loss
128
64
256
83.05
80.06
0.77
71.71
71.90
71.71
96.08
1.15
128
64
512
82.34
81.20
0.69
69.23
69.25
69.23
96.16
1.17
128
64
1024
81.34
80.06
0.76
65.46
65.71
65.46
95.30
1.26
128
96
256
83.76
82.76
0.70
70.83
70.91
70.83
96.21
1.13
128
96
512
82.48
82.91
0.66
71.31
71.41
71.31
96.57
1.10
Appendix
Table 4: STFT Ablation SSVEP
val
seen
seen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
loss
64
32
256
61.06
61.09
0.95
48.14
48.22
48.14
72.63
1.28
64
32
512
67.09
67.00
0.85
53.70
53.74
53.70
77.73
1.18
64
32
1024
66.19
65.75
0.87
53.12
53.20
53.12
77.35
1.14
64
48
256
69.34
68.22
0.85
54.77
54.77
54.77
78.36
1.21
64
48
512
69.28
68.13
0.81
55.52
55.56
55.52
78.56
1.14
Appendix
Table 5: STFT Ablation Lee2019 SSVEP
val
seen
seen
unseen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
pr_auc
loss
32
16
32
82.14
15.40
1.13
16.67
28.50
99.67
54.10
18.75
1.18
32
16
256
82.14
19.10
1.12
19.35
28.35
95.72
52.85
18.02
1.17
32
16
512
82.14
19.17
1.13
20.29
28.44
95.07
51.54
17.54
1.18
32
24
32
82.00
24.08
1.11
22.92
28.05
90.13
53.30
19.84
1.16
32
24
256
82.14
32.48
1.12
26.64
28.37
87.17
53.06
18.34
1.16
Appendix
Table 6: STFT Ablation BI2014b P300
val
seen
seen
unseen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
pr_auc
loss
128
64
256
74.44
83.32
0.23
83.33
90.91
100.00
60.09
87.14
0.23
128
64
512
72.83
83.61
0.23
83.64
91.05
99.86
59.84
86.38
0.23
128
64
1024
72.87
83.32
0.23
83.33
90.91
100.00
58.68
86.10
0.23
128
96
256
78.32
83.57
0.23
84.34
91.40
99.86
62.86
87.18
0.23
128
96
512
78.16
83.40
0.23
83.60
91.03
99.88
64.72
88.37
0.23
Appendix
Table 7: STFT Ablation BNCI2014 P300
val
seen
seen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
loss
256
128
512
35.42
29.46
1.34
24.13
19.51
24.13
50.03
1.39
256
128
1024
39.58
35.71
1.32
27.08
26.13
27.08
51.73
1.49
256
128
2048
35.12
35.71
1.34
23.44
20.72
23.44
51.72
1.41
256
192
512
35.42
33.63
1.37
26.91
21.55
26.91
51.57
1.40
256
192
1024
38.39
35.71
1.34
28.47
23.27
28.47
53.26
1.41
Appendix
Table 8: STFT Ablation MI
val
seen
seen
unseen
unseen
unseen
unseen
unseen
nperseg
noverlap
nfft
acc
acc
loss
acc
f1
recall
auc
loss
32
16
32
57.34
51.06
0.67
51.88
66.81
96.89
56.01
0.69
32
16
256
55.75
51.31
0.68
51.46
66.23
95.18
55.16
0.69
32
16
512
55.00
49.31
0.68
50.84
66.78
98.82
54.10
0.69
32
24
32
57.66
53.81
0.67
52.52
66.80
95.54
56.37
0.69
32
24
256
55.47
50.41
0.68
51.20
66.42
96.54
54.53
0.69
Appendix
Table 9: STFT Ablation Lee2019 MI
STFT Config
Strategy
Val Acc
Unseen Acc
Unseen F1
Unseen Rec
Unseen AUC
nperseg=128 noverlap=64 nfft=256
Static
71.510
68.109
68.419
68.109
96.504
Global Attn
73.932
69.311
69.490
69.311
97.114
Spatial Attn
69.943
64.663
64.718
64.663
95.877
GLU
76.211
68.990
69.355
68.990
96.537
Multiplicative
71.795
67.147
67.440
67.147
95.512
Bilinear
69.658
64.663
64.687
64.663
95.171
Appendix
Table 10: Fusion Strategy Ablation on Wang 2016 SSVEP
STFT Config
Strategy
Val Acc
Unseen Acc
Unseen F1
Unseen Rec
Unseen AUC
nperseg=128 noverlap=64 nfft=512
Static
94.906
85.964
85.955
85.964
97.716
Global Attn
94.906
83.250
83.245
83.250
96.912
Spatial Attn
94.906
84.821
84.800
84.821
97.363
GLU
95.219
85.214
85.244
85.214
97.534
Multiplicative
95.250
85.518
85.498
85.518
97.460
Bilinear
94.562
85.411
85.394
85.411
97.338
Appendix
Table 11: Fusion Strategy Ablation on Lee2019 SSVEP
STFT Config
Strategy
Val Acc
Unseen Acc
Unseen F1
Unseen Rec
Unseen AUC
nperseg=32 noverlap=16 nfft=512
Static
69.121
68.202
32.084
45.066
60.730
Global Attn
65.372
67.599
33.220
48.355
62.038
Spatial Attn
65.372
69.737
33.654
46.053
63.271
GLU
62.645
63.322
29.801
46.711
55.929
Multiplicative
65.167
66.118
25.721
35.197
57.451
Bilinear
74.506
73.520
30.101
34.211
59.848
Appendix
Table 12: Fusion Strategy Ablation on BI2014b P300
STFT Config
Strategy
Val Acc
Unseen Acc
Unseen F1
Unseen Rec
Unseen AUC
nperseg=128 noverlap=96 nfft=1024
Static
85.301
87.751
92.520
90.903
89.936
Global Attn
86.210
88.580
93.089
92.292
90.390
Spatial Attn
85.756
88.619
93.070
91.713
90.766
GLU
83.485
87.191
92.042
88.889
91.374
Multiplicative
86.581
89.410
93.624
93.310
90.953
Bilinear
86.664
89.275
93.529
93.009
90.647
Appendix
Table 13: Fusion Strategy Ablation on BNCI2014-009 P300
STFT Config
Strategy
Val Acc
Unseen Acc
Unseen F1
Unseen Rec
Unseen AUC
nperseg=512 noverlap=384 nfft=1024
Static
47.917
28.472
26.085
28.472
54.502
Global Attn
43.750
27.083
26.188
27.083
53.981
Spatial Attn
36.607
24.479
22.860
24.479
50.203
GLU
36.012
24.479
24.411
24.479
50.760
Multiplicative
47.321
28.299
27.440
28.299
55.923
Bilinear
50.000
28.993
28.586
28.993
56.170
Appendix
Table 14: Fusion Strategy Ablation on BNCI2014-001 MI
The development of generalizable electroencephalography (EEG) decoding models is essential for robust brain-computer interfaces (BCI) and objective neural biomarkers in mental health. Conventional approaches have been hindered by poor cross-subject and cross-task generalization, owing to high inter-subject variability and non-stationary neural signals. We address this challenge with a zero-shot cross-subject decoding framework on the large-scale Healthy Brain Network dataset, benchmarking a convolutional neural network baseline, a hybrid LSTM, and a Transformer-based foundation model. To adapt the Transformer for regression while averting catastrophic forgetting, we propose a novel progressive unfreezing strategy. The baseline yielded an nRMSE of 0.9991, whereas our fine-tuned Transformer achieved 0.9799 on unseen subjects. This work advances scalable, calibration-free EEG decoding for computational psychiatry and behavioral prediction.
Baimam Boukar Jean Jacques, Brandone Fonya, Nchofon Tagha Ghogomu +2
In cross-subject EEG emotion decoding, neighboring windows can inform the representation of a target window, but they contain both background variation and sustained task signals. This creates a challenge for contextual representation learning because subtracting shared activity can also remove useful information. We introduce the Morlet Spectral Transformer (MST), which conditions target-window tokens on a structured spectral summary before attention. MST averages the Morlet log-amplitude spectra of unlabeled same-trial neighbors and uses frequency-specific electrode projections to combine the target spectrum with its reference-relative residual. This retains access to absolute activity while incorporating context into a fixed attended target-token grid. Without external pretraining, MST achieves 66.5%, 40.7%, 36.1%, and 27.9% accuracy on SEED, SEED-IV, SEED-V, and SEED-VII under leave-one-subject-out evaluation, averaged over three random seeds, outperforming the evaluated pretrained and from-scratch baselines. On FACED, MST achieves 17.52% accuracy in nine-class cross-subject evaluation, exceeding the strongest evaluated baseline by 1.15 percentage points. We also performed the ablation studies across all 15 SEED subjects to evaluate the contribution of different components of MST.
Personalised music-based interventions offer a powerful means of supporting motor rehabilitation by dynamically tailoring auditory stimuli to provide external timekeeping cues, modulate affective states, and stabilise gait patterns. Generalisable Brain-Computer Interfaces (BCIs) thus hold promise for adapting these interventions across individuals. However, inter-subject variability in EEG signals, further compounded by movement-induced artefacts and motor planning differences, hinders the generalisability of BCIs and results in lengthy calibration processes. We propose Individual Tangent Space Alignment (ITSA), a novel pre-alignment strategy incorporating subject-specific recentering, distribution matching, and supervised rotational alignment to enhance cross-subject generalisation. Our hybrid architecture fuses Regularised Common Spatial Patterns (RCSP) with Riemannian geometry in parallel and sequential configurations, improving class separability while maintaining the geometric structure of covariance matrices for robust statistical computation. Using leave-one-subject-out cross-validation, `ITSA' demonstrates significant performance improvements across subjects and conditions. The parallel fusion approach shows the greatest enhancement over its sequential counterpart, with robust performance maintained across varying data conditions and electrode configurations. The code will be made publicly available at the time of publication.
Nicole Lai-Tan, Xiao Gu, Marios G. Philiastides +1