ASPEN: Spectral-Temporal Fusion for Cross-Subject Brain Decoding

Megan Lee; Seung Ha Hwang; Inhyeok Choi; Shreyas Darade; Mengchun Zhang; Kateryna Shapovalenko

cs.LGFeb 18, 2026

ASPEN: Spectral-Temporal Fusion for Cross-Subject Brain Decoding

Authors: Megan Lee, Seung Ha Hwang, Inhyeok Choi, Shreyas Darade, Mengchun Zhang, Kateryna Shapovalenko

Organizations: Carnegie Mellon University, Pittsburgh, PA, USA · Kyung Hee University, Yongin, South Korea · Korea Advanced Institute of Science and Technology, Deajeon, South Korea · University of Pittsburgh, Pittsburgh, PA, USA

Abstract

Cross-subject generalization in EEG-based brain-computer interfaces (BCIs) remains challenging due to individual variability in neural signals. We investigate whether spectral representations offer more stable features for cross-subject transfer than temporal waveforms. Through correlation analyses across three EEG paradigms (SSVEP, P300, and Motor Imagery), we find that spectral features exhibit consistently higher cross-subject similarity than temporal signals. Motivated by this observation, we introduce ASPEN, a hybrid architecture that combines spectral and temporal feature streams via multiplicative fusion, requiring cross-modal agreement for features to propagate. Experiments across six benchmark datasets reveal that ASPEN is able to dynamically achieve the optimal spectral-temporal balance depending on the paradigm. ASPEN achieves the best unseen-subject accuracy on three of six datasets and competitive performance on others, demonstrating that multiplicative multimodal fusion enables effective cross-subject generalization.

Figures & tables

Figure 1: ASPEN architecture. Raw EEG is processed through parallel temporal and spectral streams, combined via multiplicative fusion before classification.

Figure 2: Cross-session (left) and cross-subject (right) correlation comparison between temporal and spectral representations. Spectral features exhibit consistently higher cross-subject similarity across all datasets.

Figure 3: Detailed view of temporal stream, spectral stream, and multiplicative fusion components.

Dataset	nperseg	noverlap	nfft	Best Fusion	Best Acc	Mult Acc	$\Delta$
Wang 2016 SSVEP	256	128	256	Global Attn	72.76	69.47	-3.3
Lee2019 SSVEP	256	128	1024	Bilinear	86.68	85.71	-1.0
BI2014b P300	32	16	512	Bilinear	73.52	66.12	-7.4
BNCI2014-009 P300	128	120	256	Multiplicative	89.82	89.82	–
BNCI2014-001 MI	512	256	512	Multiplicative	30.73	30.73	–
Lee2019 MI	32	30	32	Multiplicative	75.70	75.70	–

Table 1: Ablation study summary. Best STFT parameters and fusion strategy per dataset, selected by unseen-subject accuracy. Best Acc = best fusion accuracy on held-out subjects (%), Mult Acc = multiplicative fusion accuracy (%),

\Delta

= absolute difference. Global Attn = Global Attention, Bilinear = Low-rank Bilinear.

Task	Dataset	Cross-	EEGNet	EEGConf.	MultiDiff.	TSformer-SA	CTNet	SPEN	ASPEN
			Method
SSVEP	Wang2016	Session	81.96 $\pm$ 6.02	56.32 $\pm$ 4.15	91.74 $\pm$ 1.62	47.96 $\pm$ 4.16	88.37 $\pm$ 4.20	85.71 $\pm$ 1.57	73.98 $\pm$ 1.46
	Wang2016	Subject	74.25 $\pm$ 5.27	49.95 $\pm$ 2.81	87.95 $\pm$ 2.56	39.93 $\pm$ 5.25	83.60 $\pm$ 0.82	78.82 $\pm$ 6.41	67.20 $\pm$ 4.83
	Lee2019	Session	95.04 $\pm$ 0.35	92.00 $\pm$ 0.94	93.67 $\pm$ 0.67	79.98 $\pm$ 6.60	95.36 $\pm$ 0.61	70.58 $\pm$ 0.80	95.50 $\pm$ 0.33
	Lee2019	Subject	86.51 $\pm$ 0.09	81.99 $\pm$ 1.05	85.04 $\pm$ 0.76	63.38 $\pm$ 6.43	87.25 $\pm$ 0.69	58.51 $\pm$ 0.98	87.53 $\pm$ 0.29
P300	BI2014b	Session	64.74 $\pm$ 2.71	78.84 $\pm$ 3.46	81.46 $\pm$ 4.56	82.25 $\pm$ 3.04	76.06 $\pm$ 6.45	84.15 $\pm$ 0.93	77.95 $\pm$ 5.52

Table 2: Final results across tasks and models. Cross-subject generalization accuracy (%). Bold indicates best cross-subject performance per dataset. Mean ± STD across three seeds.

Figure 4: Stream contributions and feature correlation ( $\rho$ ) across datasets. Low correlation values confirm that streams capture distinct information.

Figure 5: Grad-CAM visualization of feature importance for P300 classification. Correct prediction (top) shows focused attention on physiologically relevant low-frequency bands. Misclassification (bottom) reveals scattered attention towards high-frequency noise artifacts.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Dataset	Subj.	Chan.	Cls.	Bandpass	Epoch	SR
SSVEP : Frequency-	Wang2016	35	64	26	6–90 Hz	1.0s	250 Hz
tagged visual decoding	Lee2019	54	62	4	6–90 Hz	1.0s	250 Hz
P300 : Binary target	BI2014b	38	32	2	0.1–30 Hz	1.0s	256 Hz
vs. non-target ERP	BNCI2014_009	10	16	2	1–24 Hz	1.0s	256 Hz
MI : Motor	BNCI2014_001	9	22	4	4–40 Hz	4.0s	250 Hz
imagery decoding	Lee2019	54	22	2	4–40 Hz	4.0s	250 Hz

Appendix

Table 3: EEG Dataset Specifications and Preprocessing Parameters. Z-score normalization was applied to all datasets to standardize signal amplitudes across different subjects and sessions, ensuring that high-voltage artifacts or individual physiological variations do not disproportionately influence model training.

			val	seen	seen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	loss
128	64	256	83.05	80.06	0.77	71.71	71.90	71.71	96.08	1.15
128	64	512	82.34	81.20	0.69	69.23	69.25	69.23	96.16	1.17
128	64	1024	81.34	80.06	0.76	65.46	65.71	65.46	95.30	1.26
128	96	256	83.76	82.76	0.70	70.83	70.91	70.83	96.21	1.13
128	96	512	82.48	82.91	0.66	71.31	71.41	71.31	96.57	1.10

Appendix

Table 4: STFT Ablation SSVEP

			val	seen	seen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	loss
64	32	256	61.06	61.09	0.95	48.14	48.22	48.14	72.63	1.28
64	32	512	67.09	67.00	0.85	53.70	53.74	53.70	77.73	1.18
64	32	1024	66.19	65.75	0.87	53.12	53.20	53.12	77.35	1.14
64	48	256	69.34	68.22	0.85	54.77	54.77	54.77	78.36	1.21
64	48	512	69.28	68.13	0.81	55.52	55.56	55.52	78.56	1.14

Appendix

Table 5: STFT Ablation Lee2019 SSVEP

			val	seen	seen	unseen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	pr_auc	loss
32	16	32	82.14	15.40	1.13	16.67	28.50	99.67	54.10	18.75	1.18
32	16	256	82.14	19.10	1.12	19.35	28.35	95.72	52.85	18.02	1.17
32	16	512	82.14	19.17	1.13	20.29	28.44	95.07	51.54	17.54	1.18
32	24	32	82.00	24.08	1.11	22.92	28.05	90.13	53.30	19.84	1.16
32	24	256	82.14	32.48	1.12	26.64	28.37	87.17	53.06	18.34	1.16

Appendix

Table 6: STFT Ablation BI2014b P300

			val	seen	seen	unseen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	pr_auc	loss
128	64	256	74.44	83.32	0.23	83.33	90.91	100.00	60.09	87.14	0.23
128	64	512	72.83	83.61	0.23	83.64	91.05	99.86	59.84	86.38	0.23
128	64	1024	72.87	83.32	0.23	83.33	90.91	100.00	58.68	86.10	0.23
128	96	256	78.32	83.57	0.23	84.34	91.40	99.86	62.86	87.18	0.23
128	96	512	78.16	83.40	0.23	83.60	91.03	99.88	64.72	88.37	0.23

Appendix

Table 7: STFT Ablation BNCI2014 P300

			val	seen	seen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	loss
256	128	512	35.42	29.46	1.34	24.13	19.51	24.13	50.03	1.39
256	128	1024	39.58	35.71	1.32	27.08	26.13	27.08	51.73	1.49
256	128	2048	35.12	35.71	1.34	23.44	20.72	23.44	51.72	1.41
256	192	512	35.42	33.63	1.37	26.91	21.55	26.91	51.57	1.40
256	192	1024	38.39	35.71	1.34	28.47	23.27	28.47	53.26	1.41

Appendix

Table 8: STFT Ablation MI

			val	seen	seen	unseen	unseen	unseen	unseen	unseen
nperseg	noverlap	nfft	acc	acc	loss	acc	f1	recall	auc	loss
32	16	32	57.34	51.06	0.67	51.88	66.81	96.89	56.01	0.69
32	16	256	55.75	51.31	0.68	51.46	66.23	95.18	55.16	0.69
32	16	512	55.00	49.31	0.68	50.84	66.78	98.82	54.10	0.69
32	24	32	57.66	53.81	0.67	52.52	66.80	95.54	56.37	0.69
32	24	256	55.47	50.41	0.68	51.20	66.42	96.54	54.53	0.69

Appendix

Table 9: STFT Ablation Lee2019 MI

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=128 noverlap=64 nfft=256	Static	71.510	68.109	68.419	68.109	96.504
	Global Attn	73.932	69.311	69.490	69.311	97.114
	Spatial Attn	69.943	64.663	64.718	64.663	95.877
	GLU	76.211	68.990	69.355	68.990	96.537
	Multiplicative	71.795	67.147	67.440	67.147	95.512
	Bilinear	69.658	64.663	64.687	64.663	95.171

Appendix

Table 10: Fusion Strategy Ablation on Wang 2016 SSVEP

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=128 noverlap=64 nfft=512	Static	94.906	85.964	85.955	85.964	97.716
	Global Attn	94.906	83.250	83.245	83.250	96.912
	Spatial Attn	94.906	84.821	84.800	84.821	97.363
	GLU	95.219	85.214	85.244	85.214	97.534
	Multiplicative	95.250	85.518	85.498	85.518	97.460
	Bilinear	94.562	85.411	85.394	85.411	97.338

Appendix

Table 11: Fusion Strategy Ablation on Lee2019 SSVEP

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=32 noverlap=16 nfft=512	Static	69.121	68.202	32.084	45.066	60.730
	Global Attn	65.372	67.599	33.220	48.355	62.038
	Spatial Attn	65.372	69.737	33.654	46.053	63.271
	GLU	62.645	63.322	29.801	46.711	55.929
	Multiplicative	65.167	66.118	25.721	35.197	57.451
	Bilinear	74.506	73.520	30.101	34.211	59.848

Appendix

Table 12: Fusion Strategy Ablation on BI2014b P300

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=128 noverlap=96 nfft=1024	Static	85.301	87.751	92.520	90.903	89.936
	Global Attn	86.210	88.580	93.089	92.292	90.390
	Spatial Attn	85.756	88.619	93.070	91.713	90.766
	GLU	83.485	87.191	92.042	88.889	91.374
	Multiplicative	86.581	89.410	93.624	93.310	90.953
	Bilinear	86.664	89.275	93.529	93.009	90.647

Appendix

Table 13: Fusion Strategy Ablation on BNCI2014-009 P300

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=512 noverlap=384 nfft=1024	Static	47.917	28.472	26.085	28.472	54.502
	Global Attn	43.750	27.083	26.188	27.083	53.981
	Spatial Attn	36.607	24.479	22.860	24.479	50.203
	GLU	36.012	24.479	24.411	24.479	50.760
	Multiplicative	47.321	28.299	27.440	28.299	55.923
	Bilinear	50.000	28.993	28.586	28.993	56.170

Appendix

Table 14: Fusion Strategy Ablation on BNCI2014-001 MI

STFT Config	Strategy	Val Acc	Unseen Acc	Unseen F1	Unseen Rec	Unseen AUC
nperseg=32 noverlap=24 nfft=32	Static	76.969	74.714	76.266	81.250	83.905
	Global Attn	77.031	74.589	75.520	78.393	83.287
	Spatial Attn	78.000	74.607	75.287	77.357	83.576
	GLU	77.844	74.125	74.566	75.857	83.285
	Multiplicative	78.344	75.500	77.149	82.714	84.523
	Bilinear	77.625	74.214	72.620	68.393	84.000

Appendix

Table 15: Fusion Strategy Ablation on Lee2019 MI

Cross-Subject EEG Decoding

Cross-subject EEG decoding develops methods that classify or predict brain activity for people not seen during training.

See all 76 papers

Zero-Shot Neural Priors for Generalizable Cross-Subject and Cross-Task EEG Decoding
Jun 12, 2026Baimam Boukar Jean Jacques, Brandone Fonya, Nchofon Tagha Ghogomu +2Cross-Subject EEG Decoding Zero-Shot Learning
Carnegie Mellon University
Context-aware tokenization for Cross-subject Emotion Decoding from EEG
May 30, 2026Jiaxin Qing, Lexin LiCross-Subject EEG Decoding EEG Decoding
University of California, Berkeley
Cross-Subject and Cross-Montage EEG Transfer Learning via Individual Tangent Space Alignment and Spatial-Riemannian Feature Fusion
Aug 11, 2025Nicole Lai-Tan, Xiao Gu, Marios G. Philiastides +1Cross-Subject EEG Decoding Brain-Computer Interfaces

Jun 12, 2026J/K move · Enter open · S save

Zero-Shot Neural Priors for Generalizable Cross-Subject and Cross-Task EEG Decoding

Baimam Boukar Jean Jacques, Brandone Fonya, Nchofon Tagha Ghogomu, Pauline Nyaboe+1

Carnegie Mellon University

Cross-Subject EEG Decoding Zero-Shot Learning EEG Decoding EEG Foundation Models

Open paper AskPDF arXiv

The development of generalizable electroencephalography (EEG) decoding models is essential for robust brain-computer interfaces (BCI) and objective neural biomarkers in mental health. Conventional approaches have been hindered by poor cross-subject and cross-task generalization, owing to high inter-subject variability and non-stationary neural signals. We address this challenge with a zero-shot cross-subject decoding framework on the large-scale Healthy Brain Network dataset, benchmarking a convolutional neural network baseline, a hybrid LSTM, and a Transformer-based foundation model. To adapt the Transformer for regression while averting catastrophic forgetting, we propose a novel progressive unfreezing strategy. The baseline yielded an nRMSE of 0.9991, whereas our fine-tuned Transformer achieved 0.9799 on unseen subjects. This work advances scalable, calibration-free EEG decoding for computational psychiatry and behavioral prediction.

Abstract

Cross-Subject EEG Decoding

Explore similar work

Zero-Shot Neural Priors for Generalizable Cross-Subject and Cross-Task EEG Decoding

Context-aware tokenization for Cross-subject Emotion Decoding from EEG

Cross-Subject and Cross-Montage EEG Transfer Learning via Individual Tangent Space Alignment and Spatial-Riemannian Feature Fusion