Beyond Modeling Order: Dual-Path Spatiotemporal Representation Learning for Cross-Subject EEG Emotion Recognition

Zehui Xiao; Yi Zheng; Guixing Fu; Wenkai Lu; Lingfei He; Tao Tan; Yue Liu; Silun Wang

cs.LGOct 26, 2026

Beyond Modeling Order: Dual-Path Spatiotemporal Representation Learning for Cross-Subject EEG Emotion Recognition

Authors: Zehui Xiao, Yi Zheng, Guixing Fu, Wenkai Lu, Lingfei He, Tao Tan, Yue Liu, Silun Wang,

Organizations: School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · School of Computer Science and Technology, Tianjin University, Tianjin 300354, China · Faculty of Applied Sciences, Macao Polytechnic University, Macao, China · Shenzhen Dr. Brain Technologies Co., Ltd., Shenzhen 518057, China

Abstract

Cross-subject generalization remains a major challenge for EEG-based emotion recognition due to pronounced inter-subject variability in neural activity and emotion-related spatiotemporal patterns. Existing methods mainly alleviate inter subject differences through distribution alignment or contrastive learning, while paying limited attention to representation bias associated with fixed spatiotemporal modeling orders. We hypothesize that high-level emotion representations of the same EEG signal should remain consistent across different spatiotemporal modeling paths. Based on this hypothesis, we propose a Dual-Path Spatiotemporal Contrastive Learning (DP-STCL) framework for cross-subject EEG emotion recognition. DP-STCL employs two complementary encoders with independent parameters, namely a Temporal-to-Spatial (TS) path and a Spatial to-Temporal (ST) path, to model EEG spatiotemporal information under different integration orders. During representation learning, the same emotional stimulus is used to establish cross subject associations, while three complementary relations jointly constrain representation variations across subjects and modeling paths. This promotes stable and discriminative stimulus-related emotion representations. To further handle limited labeled target subject data, we introduce a LoRA-based few-shot adaptation strategy with low parameter overhead. Extensive cross-subject experiments on SEED, SEED-IV, SEED-V, and FACED show that DP-STCL consistently outperforms the compared methods, demonstrating its effectiveness in improving cross-subject EEG emotion recognition.

Figures & tables

Fig. 1: Overall framework of the proposed DP-STCL for cross-subject EEG emotion recognition. The framework consists of cross-subject pretraining with dual-path spatiotemporal encoding, same-path cross-subject contrastive learning, and cross-path semantic alignment, followed by few-shot subject adaptation with LoRA and emotion recognition for the target subject.

Fig. 2: Illustration of the multiscale dilated temporal convolution module. Parallel temporal convolution branches with different dilation rates capture temporal dependencies at multiple receptive-field scales, and their outputs are concatenated to form the enhanced representation.

Fig. 3: Illustration of the proposed collaborative representation learning scheme, including same-path cross-subject contrastive learning and cross-path semantic alignment.

Method	SEED (Acc%)	SEED-IV (Acc%)	SEED-V (Acc%)
DE+MLP	$79.9\pm 8.7$	$59.3\pm 17.2$	$59.3\pm 17.2$
DSAN [ 43 ]	$84.1\pm 7.6$	$62.2\pm 14.2$	$59.3\pm 16.8$
DMMR [ 44 ]	$88.3\pm 5.6$	$72.7\pm 8.0$	/
MS-MDA [ 45 ]	$84.5\pm 7.5$	$64.3\pm 11.1$	$58.2\pm 12.3$
ADAST [ 46 ]	$87.3\pm 2.1$	$53.7\pm 13.6$	$50.7\pm 14.0$
GMoE [ 47 ]	$84.6\pm 9.3$	$71.2\pm 8.5$	$76.3\pm 9.2$

TABLE I: Cross-Subject Emotion Recognition Performance on the SEED, SEED-IV, and SEED-V Datasets. Results marked with † are reproduced; others are from original papers.

Method	FACED (cls2) (Acc%)	FACED (cls9) (Acc%)	Source of Result
DE+MLP	$60.9\pm 3.2$	$21.7\pm 2.2$	Baseline
SimCLR	/	$30.2\pm 2.7$	Reproduced Results in [ 33 ]
MSHCL	$77.0\pm 3.6$	$45.7\pm 3.4$	Original Results in [ 50 ]
CLISA	$67.8\pm 4.1$	$42.4\pm 3.1$	Reproduced Results in [ 13 ]
DAEST	$77.8\pm 5.8$	$62.3\pm 8.1$	Reproduced Results in [ 51 ]
DP-STCL	$\mathbf{80.5\pm 4.4}$	$\mathbf{66.5\pm 7.8}$	Results from this paper

TABLE II: Cross-Subject Emotion Recognition Performance on the FACED Dataset

Fig. 4: Confusion matrices for emotion classification on the SEED, SEED-IV, SEED-V, and FACED datasets.

Training Strategy	Test Repr.	SEED (Acc%)	SEED-IV (Acc%)	SEED-V (Acc%)	FACED cls2 (Acc%)	FACED cls9 (Acc%)
Single-Path Independent Training	ST-only	$88.9\pm 9.2$	$73.5\pm 9.5$	$73.6\pm 12.5$	$78.5\pm 4.5$	$63.0\pm 7.5$
Single-Path Independent Training	TS-only	$87.1\pm 9.4$	$74.3\pm 9.9$	$74.2\pm 10.8$	$78.0\pm 4.2$	$62.5\pm 7.0$
DP-STCL Dual-Path Joint Training	ST	$89.5\pm 10.5$	$74.9\pm 9.1$	$75.2\pm 12.3$	$79.8\pm 4.5$	$64.1\pm 7.4$
	TS	$89.3\pm 11.0$	$75.7\pm 9.9$	$75.3\pm 10.7$	$79.0\pm 4.5$	$64.0\pm 7.0$
	TS+ST	$\mathbf{90.3\pm 10.2}$	$\mathbf{76.8\pm 10.4}$	$\mathbf{79.0\pm 11.4}$	$\mathbf{80.5\pm 4.4}$	$\mathbf{66.5\pm 7.8}$

TABLE III: Ablation Results for Dual-Path Spatiotemporal Modeling and the Joint Training Mechanism

Fig. 5: Subject-wise accuracy distribution of dual-path spatiotemporal modeling.

Fig. 6: Performance comparison of different cross-path learning strategies across datasets.

Fig. 7: Sensitivity analysis of the cross-path semantic alignment weight $\lambda$ across datasets.

Dataset	Method	3-shot (Acc%)	5-shot (Acc%)	10-shot (Acc%)
SEED	LoRA	$91.1\pm 7.9$	$91.2\pm 7.9$	$91.3\pm 7.5$
	Head-only	$77.8\pm 10.4$	$79.8\pm 9.2$	$79.9\pm 10.7$
	Full-finetune	$91.6\pm 7.2$	$92.8\pm 6.2$	$94.3\pm 5.1$
SEED-IV	LoRA	$77.2\pm 9.6$	$77.9\pm 8.4$	$83.6\pm 5.5$
	Head-only	$63.1\pm 9.5$	$60.8\pm 10.8$	$62.7\pm 10.1$
	Full-finetune	$77.5\pm 9.3$	$78.6\pm 8.4$	$85.8\pm 5.9$

TABLE IV: Comparison of Classification Accuracy under Different Few-Shot Settings and Parameter Update Strategies

Fig. 8: t-SNE visualization of the representations extracted by the TS branch, ST branch, and fused $\text{TS}+\text{ST}$ representation on the five emotion recognition tasks.

EEG Emotion Recognition

Electroencephalography (EEG) emotion recognition uses brain signals to classify or track affective states across people and recording settings.

See all 32 papers

MGMCL: Multi-Granularity Manifold Contrastive Learning With Neural ODEs for Cross-Subject EEG Emotion Recognition
Aug 9, 2026Xiang XieCross-Subject EEG Decoding Contrastive Learning
Cross-Subject EEG Emotion Recognition Based on Temporal Asynchronous Alignment Contrastive Learning
May 21, 2026Ying Xie, Yi Zheng, Zehui Xiao +2Cross-Subject EEG Decoding Contrastive Learning
School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · School of Computer Science and Technology, Tianjin University, Tianjin 300354, China
Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition
Jul 5, 2026Huqin Weng, Jiayang Huang, Yimin Wen +3Contrastive Learning Self-Supervised Learning

Aug 9, 2026J/K move · Enter open · S save

MGMCL: Multi-Granularity Manifold Contrastive Learning With Neural ODEs for Cross-Subject EEG Emotion Recognition

Xiang Xie

Cross-Subject EEG Decoding Contrastive Learning EEG Emotion Recognition Emotion Recognition

Open paper AskPDF arXiv

Cross-subject electroencephalogram (EEG)-based emotion recognition remains challenging due to substantial inter-individual variability and discrete formulation that overlooks affective continuity. Existing methods operate in Euclidean space and focus on marginal distribution alignment, failing to preserve the semantic structure of emotions across subjects. This article proposes MGMCL, reconceptualizing emotion recognition as learning continuous representations on symmetric positive definite (SPD) Riemannian manifolds. The frame?work introduces multi-granularity manifold contrastive learning at instance, emotion, and trajectory levels while preserving semantic ordering. Neural ordinary differential equations on manifolds model continuous emotion dynamics. Cross-subject generalization employs Gromov-Wasserstein manifold alignment. Weakly-supervised learning enables continuous valence-arousal-dominance prediction from discrete labels. Extensive experiments on three public datasets demonstrate state-of-the-art performance: 91.23% accuracy on SEED, 73.82% on SEED-IV, and 76.38% on DEAP, achieving consistent improvements of 1.89%, 1.66%, and 1.28% over previous best methods, respectively.

Abstract

EEG Emotion Recognition

Explore similar work

MGMCL: Multi-Granularity Manifold Contrastive Learning With Neural ODEs for Cross-Subject EEG Emotion Recognition

Cross-Subject EEG Emotion Recognition Based on Temporal Asynchronous Alignment Contrastive Learning

Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition