Robust EEG-based Emotion Recognition from Coarse Trial-level Labels under Intra-trial Affective Variability
Organizations: School of Mathematics (Zhuhai) Sun Yat-sen University Guangdong, 519082, CHINA · School of Computer Science and Engineering Sun Yat-sen University Guangdong, 510275, CHINA
Abstract
EEG-based emotion recognition has become an active research area in brain-computer interfaces. However, fine-grained affective annotation is costly, so most datasets provide only a global label for each long EEG trial. For practical emotion recognition, trials are commonly divided into short temporal windows, with the global label propagated to all windows for training. This practice treats trial-level emotion as temporally uniform, although affective responses may fluctuate within a trial. Consequently, windows that weakly reflect or deviate from the global emotional state may receive unreliable labels, posing a key challenge to robust EEG-based emotion recognition. To address this challenge, we propose VARY, a variability-aware robust learning framework for EEG-based emotion recognition from coarse trial-level labels. VARY incorporates a Two-stage Sample Selection (TSS) strategy to identify EEG windows with reliable emotional labels, and introduces MixAug, a data augmentation method tailored for EEG to improve model robustness under limited and unreliable supervision. We comprehensively evaluate VARY under the setting of label-noise learning on two public datasets, using three network architectures and multiple label-noise settings. Our method demonstrates superior performance, achieving a maximum accuracy gain of 11.54% on the DREAMER dataset and 5.39% on the SEED dataset compared with the best-performing baseline, highlighting the effectiveness of VARY in learning from unreliable trial-level labels under intra-trial affective variability.
Figures & tables
| Method | RACNN | Transformer | LSTM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | |
| Regular | 67.79 | 48.80 | 59.83 | 61.96 | 47.75 | 56.44 | 73.19 | 56.20 | 65.82 |
| SCE | 66.59 | 45.49 | 55.65 | 70.54 | 56.65 | 63.37 | 79.22 | 71.21 | 72.48 |
| Co-teaching | 81.97 | 70.16 | 75.00 | 79.61 | 73.58 | 74.93 | 88.09 | 75.02 | 78.82 |
| SELF | 90.54 | 77.21 | 65.63 | 79.64 | 71.15 | 72.18 | 83.70 | 74.92 | 80.20 |
| DivideMix | 90.01 | 86.61 | 86.78 | 76.46 | 69.90 | 72.90 | 87.02 | 82.27 | 85.60 |
| Method | RACNN | Transformer | LSTM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | |
| Regular | 67.99 | 49.43 | 59.01 | 63.53 | 47.74 | 57.58 | 73.72 | 56.34 | 65.78 |
| SCE | 66.18 | 45.19 | 55.30 | 72.56 | 58.19 | 65.34 | 79.89 | 73.67 | 72.85 |
| Co-teaching | 82.15 | 70.99 | 72.40 | 80.40 | 75.32 | 75.33 | 88.01 | 76.43 | 79.25 |
| SELF | 91.52 | 77.77 | 67.01 | 80.51 | 71.76 | 72.32 | 84.42 | 77.37 | 80.58 |
| DivideMix | 87.49 | 82.08 | 83.46 | 72.33 | 65.21 | 67.99 | 84.39 | 80.55 | 82.92 |
| Method | EEGNet | Transformer | LSTM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | sym 0.2 | sym 0.4 | asym 0.3 | |
| Regular | 66.00 | 44.75 | 55.57 | 51.48 | 42.76 | 47.18 | 63.00 | 44.75 | 54.98 |
| SCE | 63.99 | 43.21 | 54.12 | 51.00 | 42.41 | 46.81 | 78.50 | 52.37 | 66.09 |
| Co-teaching | 71.60 | 41.88 | 56.04 | 56.62 | 43.24 | 49.95 | 83.29 | 57.08 | 73.37 |
| SELF | 60.82 | 43.91 | 55.17 | 53.25 | 44.46 | 48.50 | 68.12 | 46.94 | 59.07 |
| DivideMix | 82.94 | 48.23 | 66.45 | 62.60 | 49.27 | 56.36 | 90.85 | 62.31 | 79.23 |
| Method | Valence | Arousal | ||
|---|---|---|---|---|
| sym 0.4 | asym 0.3 | sym 0.4 | asym 0.3 | |
| Co-divide | 89.64 | 91.13 | 86.84 | 87.32 |
| EPS | 86.07 | 87.20 | 84.18 | 84.69 |
| TSS (ours) | 93.37 | 94.00 | 93.62 | 93.36 |
| Augmentation | Description | Settings |
|---|---|---|
| AddNoise* | Add random noise to time series. | scale = 0.001 |
| Drift* | Drifts the value of time series from its original values randomly and smoothly. | max_drift = 0.25, n_drift_points = 5 |
| Convolve* | Convolve time series with a kernel window. | window = ’flattop’, size = 5 |
| TimeWarp* | Randomly change the speed of timeline. | n_speed_change = 5, max_speed_ratio = 3 |
| Permutation | Randomly shuffles segments of a time series to create new variations. | max_number_of_segments = 5 |
| Method | Valence | Arousal | ||
|---|---|---|---|---|
| w/o | w/ | w/o | w/ | |
| None | 70.83 | 68.12 | 70.96 | 69.15 |
| AddNoise | 71.01 | 69.36 | 71.44 | 70.27 |
| Drift | 77.98 | 69.25 | 77.02 | 69.84 |
| Convolve | 84.66 | 93.36 | 84.65 | 93.55 |
| TimeWarp | 79.91 | 88.34 | 80.33 | 88.82 |
| Method | Valence | Arousal | ||
|---|---|---|---|---|
| sym 0.4 | asym 0.3 | sym 0.4 | asym 0.3 | |
| VARY | 93.37 | 94.00 | 93.62 | 93.36 |
| w/o NEP | 90.72 | 90.74 | 91.91 | 92.01 |
| w/o CoT | 91.65 | 92.37 | 92.50 | 92.21 |
| w/o LCSF | 91.74 | 92.81 | 92.09 | 92.49 |
| w/o TSS | 72.13 | 77.98 | 72.79 | 77.66 |
EEG Emotion Recognition
Electroencephalography (EEG) emotion recognition uses brain signals to classify or track affective states across people and recording settings.
See all 34 papers