Modeling Shared and Individual Structure for Cross-Subject Continuous Affect Regression from EEG-fNIRS

Xuan Wang; Bing Wang; Shuai Chang; Hao Yuan; Xinbo Qi; Xinyue Zhang

cs.AIOct 2, 2026

Modeling Shared and Individual Structure for Cross-Subject Continuous Affect Regression from EEG-fNIRS

Authors: Xuan Wang, Bing Wang, Shuai Chang, Hao Yuan, Xinbo Qi, Xinyue Zhang

Abstract

Continuous, second-by-second valence-arousal estimation from physiological signals is typically studied in a subject-dependent setting, where the model sees labeled data from the same person it is later evaluated on. We study the harder zero-shot cross-subject variant on a synchronized EEG-fNIRS dataset: predict raw-scale ([1, 255]) valence and arousal trajectories for subjects whose labels the model never observes, given only their unlabeled EEG/fNIRS recordings while watching the same video stimuli as a disjoint set of training subjects. We decompose the affect trajectory into a structure shared across subjects who watch the same stimuli and an individual structure estimated for each test subject from a label-free EEG marker (alpha-band cross-channel synchrony), which rescales the shared trajectory around the scale midpoint. We validate the per-subject calibration mechanism on four independent axes: leave-one-subject-out correlation between the marker and each subject's true optimal gain, a functional-form comparison against non-linear alternatives, a repeated leave-4-out component ablation isolating each part of the pipeline's contribution, and a ceiling analysis bounding the remaining headroom for per-subject scaling. On held-out subjects, the model reaches an overall MAE of 25.96 / 22.80 across two evaluation batches (valence 21.94 / 19.6, arousal 29.98 / 26.0), well below EEGNet and ASAC-Net baselines reported for the same subject-independent split (raw scale score 60.6 and 55.0 respectively). We further report a systematic negative-result search across model architectures, feature representations, and prediction targets that found no signal able to improve on the single alpha-synchrony marker.

Explore similar work

Oct 2, 2026

Low-Cost Video--Time Priors as a Strong Baseline for EEG--fNIRS Emotion Regression on Familiar Videos

Continuous emotion regression estimates moment-to-moment valence and arousal while a viewer watches a video. In familiar-video deployment, responses fron training participant-specific estimate, and prior-dominating fixed fusion tests whether physiology adds residual correction. In five-fold subject-held-out evaluation on 24was within 0.05 and 0.32 MAE of fusion in the internal and external evaluations, respectively. Source-explicit ablations showed that video identity and within-video tine accounted for most of the reduction, while EG-FNIRS gains were smaller and varied across participants and videos. These results identify the video-time prior as a strong, low-cost baseline and position EEG-fNIRS as an optional residual signal for familiar-video emotion regression.

Minghao Kong, Jiurun Chen, Ying Gao +2

School of Electronics and Communication Engineering Sun Yat-sen University, Shenzhen Campus Shenzhen, China · Department of Network Intelligence Pengcheng Laboratory Shenzhen, China

Emotion Recognition EEG Emotion Recognition Multimodal Sensor Fusion

Ask PDF

Aug 9, 2026

MGMCL: Multi-Granularity Manifold Contrastive Learning With Neural ODEs for Cross-Subject EEG Emotion Recognition

Cross-subject electroencephalogram (EEG)-based emotion recognition remains challenging due to substantial inter-individual variability and discrete formulation that overlooks affective continuity. Existing methods operate in Euclidean space and focus on marginal distribution alignment, failing to preserve the semantic structure of emotions across subjects. This article proposes MGMCL, reconceptualizing emotion recognition as learning continuous representations on symmetric positive definite (SPD) Riemannian manifolds. The frame?work introduces multi-granularity manifold contrastive learning at instance, emotion, and trajectory levels while preserving semantic ordering. Neural ordinary differential equations on manifolds model continuous emotion dynamics. Cross-subject generalization employs Gromov-Wasserstein manifold alignment. Weakly-supervised learning enables continuous valence-arousal-dominance prediction from discrete labels. Extensive experiments on three public datasets demonstrate state-of-the-art performance: 91.23% accuracy on SEED, 73.82% on SEED-IV, and 76.38% on DEAP, achieving consistent improvements of 1.89%, 1.66%, and 1.28% over previous best methods, respectively.

Xiang Xie

Cross-Subject EEG Decoding Contrastive Learning EEG Emotion Recognition Emotion Recognition

Ask PDF

Jun 15, 2026

Learning aligned EEG representations with subject-specific encoders

Cross-subject EEG decoding promises more training data, but it also exposes neural networks to strong inter-subject distribution shifts. We study whether task supervision and architecture alone can learn subject-aligned representations. We replace a shared EEG encoder with subject-specific encoders followed by a common classifier, and compare this hybrid model with standard EEGNet, AttentionBaseNet, and CTNet baselines with Euclidean Alignment (EA) on three motor-imagery datasets and one motor-execution dataset. EA improves shared encoders by recentering subject covariances, whereas the hybrid encoder reduces reliance on EA: removing EA has little effect on validation-loss dynamics or latent-space organization, and both hybrid variants consistently outperform non-aligned shared baselines. Subject-specific heads increase class distinctiveness and place each subject close to its own latent manifold while improving within-subject class separation. However, on cross-subject classification, subject-specific heads hinder direct parameter transfer to unseen subjects, motivating quantitative head selection and a brief calibration session. Although decoding gains depend on the dataset and backbone, our main findings concern that the sole use of architecture pressure promotes representation learning and alignment in a direction complementary to domain adaptation methods such as Euclidean Alignment. A per-subject low-rank adapter of only 2Cr parameters recover the full encoder's accuracy across five backbones and ranks

r=1

to 16, so the per-subject module can be compressed by two to three orders of magnitude.

Bruna J. Lopes, Gabriel Schwartz, Sylvain Chevallier +2

University of São Paulo, São Paulo, Brazil · Université Paris-Saclay, Inria TAU team, LISN-CNRS, France · Institut de neuromodulation, GHU Paris, psychiatrie et neurosciences, centre hospitalier Sainte-Anne, pôle hospitalo-universitaire 15, Université Paris Cité, Paris, France +3

Cross-Subject EEG Decoding MI Classification EEG Decoding Representation Alignment Latent Space Alignment

Ask PDF