Pseudo-Label Augmentation for Affect Sensing in Small Collaborative Groups

Meisam Jamshidi Seikavandi; Tanya Ignatenko; Fabricio Batista Narcizo; Paolo Burelli; Jesper Bünsow Boldt; Andrew Burke Dittberner

cs.LGSep 13, 2026

Pseudo-Label Augmentation for Affect Sensing in Small Collaborative Groups

Authors: Meisam Jamshidi Seikavandi, Tanya Ignatenko, Fabricio Batista Narcizo, Paolo Burelli, Jesper Bünsow Boldt, Andrew Burke Dittberner

Organizations: IT University of Copenhagen brAIn Lab Copenhagen, Denmark · GN Advanced Science, GN Hearing Ballerup, Denmark · GN Advanced Science, GN Hearing Glenview, Illinois, United States

Abstract

Physiological affect sensing in naturalistic group interaction is often limited by sparse labels rather than sensor data: wearable devices produce many time windows, while self-reports are collected only a few times per session. Using GroupAffect-4, a four-person collaborative dataset with wearable physiology, eye tracking, Big Five personality, and post-task VAD labels, we study pseudo-label augmentation for affect sensing under sparse supervision. We compare no augmentation, Gaussian Process pseudo-labelling, personality-aware trust weighting, and joint personality-plus-confidence weighting within a shared target-construction pipeline. Results show that pseudo-label augmentation improves over the labelled-only baseline in the known-team setting. However, the narrow range of Big Five cosine similarities (0.91-0.99) makes fine-grained personality weighting ineffective; personality similarity functions mainly as a same-team filter rather than a calibrated trust signal. With smoothing, augmented SVM variants are effectively tied on Valence and Arousal, while the joint personality-plus-confidence variant gives the highest Dominance score. Cross-subject LOSO transfer remains encouraging, especially for Arousal, whereas strict session-isolated LOGO removes the augmentation benefit. Given only 10 groups, LOGO should be interpreted as a conservative lower bound on unseen-group transfer. Overall, the results suggest that pseudo-label augmentation can make better use of sparsely labelled collaborative affect data, while personality information is most useful as a within-team selection mechanism.

Figures & tables

Modality	Device	Rate	Features
PPG	EmotiBit	$\sim$ 25 Hz	HR, RMSSD, SDNN, amp.
EDA	EmotiBit	$\sim$ 25 Hz	SCL/SCR, phasic, tonic
Skin temp.	EmotiBit	$\sim$ 25 Hz	Mean, slope
IMU	EmotiBit	$\sim$ 25 Hz	Accel. mag., movement
Gaze	Tobii G3	40–60 Hz	Fix. rate, saccade
Pupil	Tobii G3	40–60 Hz	Mean diam., dilation var.

Table 1. Physiology and eye-tracking inputs (GroupAffect-4).

Label pipeline	V	A	D
Fixed thresholds, no weights	0.575	0.516	0.338
Balanced tertile, no weights	0.372	0.443	0.429
Full target pipeline	0.405	0.464	0.495

Table 2. Label-pipeline ablation motivating the shared target construction. Values are T3 macro-F1 for an MLP without augmentation.

Component	Definition
Dataset	GroupAffect-4, 40 participants, 10 groups
Signals	Gaze, pupil, EDA, PPG, skin temp., IMU
Features	49 physiology/eye-tracking summary features
Labels	Balanced midpoint-tertile VAD classes
Train/val/test	T0+T1 / T2 / T3
Base model	RBF-SVM, $C=1$ , balanced class weights

Table 3. Canonical benchmark setup used throughout this paper.

Check	Why it matters
Train-fitted thresholds	Keeps hard and pseudo-labels in one class space.
Exclude test-task labels	Prevents direct temporal label leakage.
Regenerate LOGO pool	Removes held-out group from all augmentation signals.
Keep BFI out of SVM features	Preserves personality as trust, not identity.
Report V/A/D separately	Reveals that A2 mainly helps V and D.

Table 4. Checks required before claiming leakage-aware augmentation.

Variant	Pseudo-label / weight rule
A0	No augmentation; labelled T0+T1 windows only.
A1	GP class argmax for all VAD dimensions.
A2	GP class argmax, BFI cosine similarity as trust weight.
A3	GP class argmax, BFI similarity multiplied by GP confidence.

Table 5. Main augmentation variants carried through the paper. Preprocessing and window-size side ablations are reported separately in Section 5.5 .

Figure 1. SVM augmentation variants on the canonical T3 test split (A0: no augmentation; A1: GP-only; A2: BFI-only; A3: BFI

\times

GP). Green outlines mark the best variant per VAD dimension. Grouped bar chart of macro-F1 for A0–A3 on Valence, Arousal, and Dominance. A2 is highest for Valence and Dominance, while A1 is highest for Arousal.

Variant	V	A	D
A0: LOSO no aug	0.190	0.438	0.334

Table 6. LOSO cross-subject stress test (no augmentation). Aggregate over 32 held-out participants with T3 labels.

Figure 2. Session-isolated LOGO augmentation ( leakage-clean : pool, GP, and BFI maps all exclude the held-out session), mean across 10 held-out groups. The shaded band marks A0

\pm

0.03 per dimension; every augmented variant falls inside it, and no variant significantly outperforms A0 (Wilcoxon

p{>}0.2

for all). Per-dim only; no pooling. Grouped bar chart of session-isolated LOGO macro-F1 across ten held-out groups. All augmented variants remain within 0.03 of the A0 baseline for each VAD dimension.

Condition	Variant	V	A	D
15s baseline	A0	0.570	0.530	0.385
	A1	0.592	0.546	0.406
	A2 (best aug)	0.608	0.535	0.435
	A3	0.593	0.544	0.431
EDA/PPG smooth (ctr.)	A0	0.601	0.566	0.409
	A1	0.642	0.624	0.427

Table 7. Side ablation on smoothing and window size (known-team, test = T3). Per-dim only; no pooling.

Affective Computing

Affective computing develops systems that recognize, model, and respond to human emotions and affective states from behavior and physiological signals.

See all 141 papers

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction
May 19, 2026Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara +10Emotion Recognition Affective Computing
GN Advanced Science, GN Group, Ballerup, Denmark · IT University of Copenhagen, brAIn lab, Copenhagen, Denmark · Copenhagen Business School, Copenhagen, Denmark +1
Decision-Level Fusion for Robust Wearable Affect Recognition
May 14, 2026Lokesh Singh, Athina Georgara, Jayati Deshmukh +2Multimodal Robustness Frequency-Domain Feature Learning
University of Southampton · Southampton, United Kingdom
GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos
Apr 17, 2026Deepak Kumar, Abhishek Pratap Singh, Puneet Kumar +2Multimodal Emotion Recognition Emotion Recognition
Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, Roorkee, India · Indian Institute of Technology Ropar, Punjab, India · Zhejiang University, China +1

May 19, 2026J/K move · Enter open · S save

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction

Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Shan Ahmed Shaffi+9

GN Advanced Science, GN Group, Ballerup, Denmark · IT University of Copenhagen, brAIn lab, Copenhagen, Denmark · Copenhagen Business School, Copenhagen, Denmark+1

Emotion Recognition Affective Computing

Open paper AskPDF arXiv

Existing affective-computing, social-signal-processing, and meeting corpora capture important parts of human interaction, but they rarely support analysis of affect in co-located groups as a coupled individual, interpersonal, and group-level process. The required signals (per-participant physiology, eye movement, audio, self-report, task outcomes, and personality) are usually fragmented across separate dataset traditions. We introduce GroupAffect-4, a multimodal corpus of 40 participants in 10 four-person groups, each completing four ecologically varied collaborative tasks spanning information pooling, negotiation, idea generation, and a public-goods game. Each participant is instrumented with a wrist-worn physiology sensor, eye-tracking glasses, and a close-talk microphone; sessions include continuous affect self-reports, post-task questionnaires, task outcomes, and Big-Five personality scores, all time-aligned to a shared clock. The dataset covers over 91% of expected physiology windows and 98% of eye-tracking windows, with strong task validity confirmed by a clear affective manipulation check across the negotiation block. We define fifteen benchmarkable targets spanning three analysis levels -- within-person state, between-person traits, and group dynamics -- and report leave-one-group-out feasibility baselines establishing the dataset's evaluative scope. GroupAffect-4 is released with a BIDS-inspired structure, Croissant metadata, a datasheet, per-session quality reports, and open processing scripts. Code and processing scripts are available at https://github.com/meisamjam/GroupAffect-4; the dataset is publicly archived at https://zenodo.org/records/20037847.

Abstract

Affective Computing

Explore similar work

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction

Decision-Level Fusion for Robust Wearable Affect Recognition

GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos