Organizations: Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603 Lembah Pantai, Kuala Lumpur, Malaysia. · Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430022, China · Hubei Provincial Clinical Research Center for Brain Computer Interface, Huazhong University of Science and Technology, Wuhan, Hubei 430022, China.
EEG-eye movement emotion recognition can support mental-health monitoring and adaptive interfaces, but most fusion models treat the two signals as parallel features, classify each window alone and degrade when eye data are lost. We propose the Context-Aware Topological Distillation Graph Network (CTD-GNet), in which EEG features form the nodes of an electrode graph, and eye movements and preceding windows modulate its edges, nodes and readout, the edges through electrode-specific prompts. Bidirectional distillation of the graph, node relations and predictions couples this multimodal teacher with a single-window EEG student, improving the teacher and aligning the two graphs 35-fold. In leave-one-subject-out tests on SEED and SEED-IV against 17 baselines, CTD-GNet has the highest mean accuracy, 84.91% and 76.17%, significantly above 9 and 6 of 13 eye-based baselines after Holm correction, the smallest cross-subject spread among multimodal models and the highest worst-subject accuracy. It is the most accurate model at every rate of missing eye data on SEED-IV, keeps 66.1% on SEED without eye data, more than any EEG-only model, and stays 9.3 and 11.9 points above all single-window fusion models under eye-movement noise of one standard deviation. On film clips unseen in training, it leads on SEED with 77.1%.
Figures & tables
Fig. 1: Two ways to combine EEG and eye movements. (a) Symmetric fusion encodes both signals as feature vectors and merges them by concatenation or cross-attention; missing eye data change the fused feature, and each window is classified independently. (b) CTD-GNet: EEG gives the nodes of an electrode graph, and eye movements (green) and the preceding windows (purple) modulate its edges, nodes and readout. Bidirectional topology distillation couples the graph with that of an EEG-only student, which serves devices without an eye tracker.
Fig. 2: Architecture of CTD-GNet. Teacher (blue): a GRU context bank encodes the K preceding windows into ct ; a gated node pathway adds the eye movements to the EEG node embeddings; the generator adds the context prompt Πc to the queries, the eye prompt Πe to the keys and logAphy to the scores, and a row-wise softmax gives AT ; a GCN propagates over AT , and a gated multimodal unit fuses the graph readout with the eye movements and ct . Student (orange): the same operations with prompts from the current EEG window. Red dashed arrows: distillation losses, with gradients to both networks. Right: deployment and training objective.
Fig. 3: Dual-prompt graph generator. (a) Proposition 1 : with shared prompts γ and η , the term (ai+γ)η⊤ is constant within each row (red frame) and is removed by the row-wise softmax. (b) Electrode-specific prompts make qiπje⊤ vary with i and j , so the eye movements change the graph. (c) The physical prior acts as a soft mask: softmaxrow(S)⊙Aphy normalized per row equals softmaxrow(S+logAphy) .
Fig. 4: Bidirectional multi-level topology distillation. (a) Ltopo compares rows of the softened topologies, Lrel rows of the N×N electrode relation matrices, which need no projection between networks of different width, and Llogit the class distributions. (b) Proposition 2 : the topology difference ΔA bounds the message difference. (c) Training: ω(e) ramps to 1 over Ew epochs, and the distillation gradients reach both networks.
Fig. 5: Accuracy on each test subject for the 14 models that use eye movements. Boxes span the quartiles over the test subjects, whiskers reach the worst and the best subject, dots mark the individual subjects and diamonds the mean.
Method
SEED
SEED-IV
Negative
Neutral
Positive
Macro
Neutral
Sad
Fear
Happy
Macro
SVM
54.8
57.3
79.3
63.8
45.9
43.7
38.7
45.6
43.5
EEGNet [ 21 ]
38.1
43.1
60.6
47.3
34.2
29.1
26.5
38.2
32.0
DGCNN [ 25 ]
53.4
57.5
79.1
63.3
43.7
42.0
38.5
45.6
42.5
RGNN [ 26 ]
44.5
51.6
78.3
58.2
40.3
33.0
29.5
40.5
35.9
MLP (eye only)
71.2
80.6
73.5
75.1
76.0
63.2
78.3
67.0
71.1
TABLE II: F1 score of each class (%), mean over the test subjects. Macro: mean over the classes, as in Table I . The best value in each column is in bold .
Method
SEED
SEED-IV
Acc
F1
Acc
F1
Student without distillation
63.85
63.22
41.93
41.58
Vanilla KD [ 39 ]
64.46
63.84
41.85
41.54
FitNets [ 40 ]
64.00
63.55
41.99
41.77
RKD [ 41 ]
63.92
63.25
41.55
41.17
SP [ 42 ]
63.61
63.04
42.08
41.78
TABLE III: Prediction without eye movements (accuracy and macro-F1, %). Upper block: student trained with different distillation losses in the base configuration. Middle block: multimodal models tested with all eye movements removed. Lower block: CTD-GNet in the final configuration.
Model
Input
Params
FLOPs
CPU
GPU
DGCNN [ 25 ]
EEG
131.5k
0.79M
0.09
0.20
RGNN [ 26 ]
EEG
2.6k
0.12M
0.08
0.20
DCCA [ 6 ]
EEG+Eye
120.7k
0.24M
0.08
0.15
TFN [ 8 ]
EEG+Eye
204.6k
0.41M
0.09
0.16
LMF [ 9 ]
EEG+Eye
65.3k
0.13M
0.10
0.21
MISA [ 36 ]
EEG+Eye
125.5k
0.44M
0.16
0.29
TABLE IV: Computational cost per window on SEED-IV ( K=5 ). FLOPs count a multiply-accumulate as two operations, with recurrent layers counted analytically; latency (ms) is measured at batch size one on one core of an Intel Xeon Gold 6430 CPU and on an NVIDIA RTX 4090 GPU. Hist.: K past windows.
Method
SEED
SEED-IV
DGCNN [ 25 ]
63.4
39.8
MLP (eye only)
63.2
57.6
Feature concatenation
70.2
61.7
Bimodal-LSTM [ 14 ]
72.6
62.3
MMIN [ 16 ]
71.2
58.9
CTD-GNet student
64.2
38.9
TABLE V: Generalization to new subjects and new stimuli: accuracy (%) on film clips that no training or validation subject has seen. In each fold, one clip per emotion, on SEED-IV one per emotion and session, is held out from all training and validation subjects.
Variant
SEED
SEED-IV
Acc
F1
Acc
F1
Final (GRU, bidirectional)
84.91
84.67
76.17
76.01
TCN instead of GRU
83.92 †
83.68
75.65
75.43
One-directional distillation
83.71 †
83.52
75.90
75.74
No distillation
83.71 †
83.53
75.82
75.65
Bidirectional, logit loss only
84.57
84.35
76.25
76.08
TABLE VI: Ablation of the teacher (accuracy and macro-F1, %). Upper block: the final configuration and variants of it. Lower block: components removed from the base configuration, which uses a TCN context encoder and one-directional distillation. † : p<0.05 against the first row of the block, two-sided Wilcoxon signed-rank test on per-fold accuracies, uncorrected; with Holm correction within each block, no difference reaches the 0.05 level.
Fig. 6: Test accuracy versus the position of the window within the trial, in 20-s bins plotted at the bin centre, over all folds and seeds. Bins with fewer than 1000 test windows are omitted.
Fig. 7: Validation accuracy over the hyperparameter grid, relative to the selected configuration, which lies at zero. Each panel varies one hyperparameter and keeps the others at their selected values. The temperature and the loss weights are searched in the base configuration, whose stop-gradient leaves the teacher unaffected by them, so only the student is shown in these panels.
Fig. 8: Accuracy under missing (a, b) and noisy (c, d) eye-movement data, mean over the test subjects. Blue band: teacher ± one standard deviation over the test subjects. Grey band: range of the other seven models that use eye movements, namely the eye-only MLP, feature concatenation, DCCA-AM, TFN, LMF, MISA and ModDrop. The dual mode uses the student for windows without eye data; under noise, it differs from the teacher only on the few windows whose eye data are invalid in the recordings.
Dataset
Method
Missing rate r
Noise level σn
20%
40%
60%
80%
100%
0.25
0.5
1
2
SEED
MLP (eye only)
66.7
58.5
50.0
41.7
33.4
73.0
67.8
57.7
47.2
Feature concatenation
78.1
73.5
69.0
64.4
60.0
81.6
78.9
71.2
58.5
DCCA-AM [ 6 ]
76.4
71.8
67.3
62.8
58.3
79.6
76.5
69.4
59.9
TFN [ 8 ]
75.9
70.7
65.5
60.4
55.3
79.6
75.8
67.3
56.6
LMF [ 9 ]
76.9
71.6
66.3
61.1
55.9
80.1
76.0
67.2
56.5
TABLE VII: Accuracy (%) under missing and noisy eye-movement data, mean over the test subjects and five random draws. The results without corruption are those of Table I . The best value in each column is in bold .
Fig. 9: Learned graphs per emotion on SEED-IV. Rows: teacher topology AT , distilled student topology AS , student topology without distillation, and eye-movement attribution ΔA=Et[AT−AT(xe=0)] , where the eye features of all windows are set to their calibrated mean of zero; red edges gain and blue edges lose weight through the eye movements. Each panel shows the 5% strongest edges averaged over the test windows of the class.
Fig. 10: Learned graphs per emotion on SEED, drawn as in Fig. 9 .
Fig. 11: Band-wise topology sensitivity Ib,i of the teacher, defined in Eq. ( 20 ), for the α , β and γ bands and each emotion, averaged over the test windows of all folds and seeds. The values at the 62 electrodes (dots) are interpolated with a thin-plate spline; nose up, left hemisphere on the left, one colour scale for both datasets.
Distillation
SEED
SEED-IV
KL ↓
Jaccard ↑
KL ↓
Jaccard ↑
None
7.55 ± 1.42
0.209 ± 0.047
7.90 ± 1.47
0.093 ± 0.030
Bidirectional
0.21 ± 0.09
0.414 ± 0.131
0.23 ± 0.06
0.226 ± 0.063
TABLE VIII: Agreement between the teacher graph AT and the student graph AS on the test windows: mean row-wise KL divergence KL(AT,i∥AS,i) and Jaccard overlap of the 10% strongest edges, mean ± standard deviation over the 36 (SEED) and 45 (SEED-IV) runs of all folds and seeds.
Dataset
Emotion
Frontal
Central
Temporal
Parietal
Occipital
SEED
Negative
− 0.70
− 0.33
− 2.05
− 0.48
− 0.54
Neutral
− 0.95
− 0.14
− 2.43
− 0.54
− 0.80
Positive
− 0.49
− 0.14
− 1.30
− 0.16
− 0.51
SEED-IV
Fear
− 0.38
− 0.01
− 0.55
− 0.31
− 0.50
Happy
− 0.34
− 0.16
− 0.24
− 0.52
− 0.25
Neutral
− 0.58
− 0.00
− 0.27
− 0.27
− 0.04
TABLE IX: Change of the attention received by each scalp region through the eye movements, N100∑i∑j∈RΔAij (points), for the teacher graph averaged over the test windows of each emotion. Frontal: FP, AF and F electrodes (14); central: FC and C (14); temporal: FT7, FT8, T7, T8, TP7 and TP8 (6); parietal: CP and P (16); occipital: PO, O and CB (12). Each row sums to zero.
EEG-based emotion recognition is critical for mental health monitoring and affective brain-computer interfaces, yet existing deep learning approaches often treat emotion classes as isolated labels, ignoring their psychological interdependencies. We propose a graph-regularized learning framework that conceptualizes emotions as nodes in a graph where edges encode proximity based on dimensional emotion theories. We adapt three complementary regularization strategies--Graph Label Smoothing (intuitive soft labeling), Commuting distance on graph via Graph Laplacian (spectral graph theory), and Sliced Wasserstein Distance (optimal transport on graph)--ordered by increasing computational complexity. These strategies penalize model predictions that deviate from the established emotion topology. Our framework is evaluated across three representative backbone architectures: AudioTransformer (pure transformer), Conformer (CNN-transformer hybrid), and DCGNN (causal graph neural network), demonstrating architecture-agnostic benefits. Experiments on SEED-IV (4 classes) and SEED-V (5 classes) datasets show consistent improvements: best case up to +5.42% accuracy and 39% reduction in psychologically implausible misclassifications. Ultimately, our framework help raise the upper bound of performance achievable with standard approaches. Code will be released.
Dongyang Kuang, Zizheng Ma, Yushan Zhang +1
School of Mathematics (Zhuhai), Sun Yat-sen University, Guangzhou, China
EEG-based emotion recognition holds significant promise for objective diagnosis of mood disorders. Graph neural networks (GNNs) have emerged as the dominant paradigm for modeling inter-channel dependencies in EEG, yet existing approaches rely on symmetric adjacency matrices derived from spatial proximity or functional correlations that fundamentally capture statistical associations rather than directed causal influences, which conflicts with the inherently asymmetric, causally-driven nature of neural information flow. To bridge this gap, we propose GL-LFGNN, a Global-Local Dual-branch Causal Graph Neural Network grounded in Liang-Kleeman information flow theory. Unlike Granger causality that merely assesses temporal precedence, our approach rigorously quantifies causal strength from a dynamical systems perspective, yielding neurophysiologically interpretable directed graphs. A dual-branch architecture further integrates whole-brain connectivity with region-specific processing aligned to established functional neuroanatomy. On the MEEG dataset, GL-LFGNN achieves 86.17% (Arousal) and 86.71% (Valence) accuracy with only 37K parameters -- approximately 10% of the current state-of-the-art -- demonstrating that principled causal modeling can simultaneously enhance interpretability, generalization, and computational efficiency. Code will be released.
Mixed emotions represent a clinically relevant but still underexplored target for automatic emotion recognition. EEG provides millisecond-level access to neural activity, yet most EEG pipelines analyze the signal through a single temporal window, thereby fixing the temporal structure available to the model. This study introduces a multi-scale temporal framework for EEG-based emotion recognition. The EEG waveform is decomposed into windows of one or several durations, processed by a shared attention-based encoder, and integrated through a dynamic fusion module that assigns sample-specific weights across temporal scales. The framework is evaluated under a subject-independent protocol in binary and three-class settings, with the three-class task including the mixed affective category. The best results are 65.22% for the two-class task and 45.43% for the three-class task. Both are obtained with three-scale dynamic-fusion configurations and remain substantially above the full-signal baseline. The best-performing temporal scales differ between the two tasks. Dynamic fusion outperforms concatenation in the highest-scoring two-class configuration and slightly exceeds it in the highest-scoring three-class configuration, although these multi-scale settings require substantially more computation than the full-signal baseline.
Stefanos Gkikas, Yang Guo, Guangliang Li +3
Honda Research Institute Japan Wako City, Japan · Faculty of Information Science and Engineering Ocean University of China Qingdao, China · BioSIS (Biosensing & Intelligent Systems) Lab Centre for Intelligent Computing and Systems University of Canberra Canberra, Australia +1