Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection

Stefanos Gkikas; Thomas Kassiotis; Yang Guo; Guangliang Li; Eric Nichols; Houshyar Asadi; Nikolaos Smyrnis; Giorgos Giannakakis

eess.SPAug 10, 2026

Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection

Authors: Stefanos Gkikas, Thomas Kassiotis, Yang Guo, Guangliang Li, Eric Nichols, Houshyar Asadi, Nikolaos Smyrnis, Giorgos Giannakakis

Organizations: Honda Research Institute Japan Wako City, Japan · Department of Electronic Engineering Hellenic Mediterranean University Chania, Greece · Faculty of Information Science and Engineering Ocean University of China Qingdao, China · Institute for Intelligent Systems Research and Innovations (IISRI) Deakin University Geelong, Australia · University Mental Health, Neurosciences and Precision Medicine, Research Institute “COSTAS STEFANIS” 2nd Department of Psychiatry, Medical School, National and Kapodistrian University of Athens Athens, Greece

Abstract

Electrodermal activity (EDA) is widely used in automatic stress detection, yet most pipelines treat it only as a raw one-dimensional waveform. This study examines whether complementary visual representations of EDA provide useful information for stress classification and whether their fusion im- proves recognition performance. Six image-based representations are derived from each EDA recording: an unwrapped short-time Fourier transform (STFT) phase spectrogram, an instantaneous-frequency map computed from that phase, a power spectral density (PSD) spectrogram, a continuous wavelet transform scalogram, a recurrence plot, and a rendered waveform trace. The selected representations are stacked as channels of a single multichannel input, together with the raw waveform, and processed by a shared asymmetric-attention architecture. Experiments on a 58-subject stress dataset show that representation fusion improves over the raw waveform. The best configuration, which combines five representations while excluding the unwrapped phase spectrogram, reaches 70.97% test accuracy, compared with 67.36% for the raw waveform. The single PSD spectrogram achieves 69.44%, remaining close to the best-fused configuration at a lower computational cost. The results show that alternative visual forms of the same EDA signal can provide useful inductive biases for stress detection, and that a compact selection of complementary representations can be more effective than the raw waveform alone.

Figures & tables

Figure 1: Example of the six visual representations derived from a single EDA recording. Top row (left to right): Angle , Phase , PSD . Bottom row (left to right): Recurrence , Scalogram , Wave .

Figure 2: Overview of the proposed pipeline. Each EDA recording produces the raw waveform and, in parallel, six visual representations. Representations are flattened into one-dimensional channels, stacked together with the raw waveform, and processed by the shared asymmetric-attention backbone for binary stress classification.

Hyperparameter	Value
Depth	1
Number of latents	64
Latent dimension ( $d_{0}$ )	128
Cross-attention heads	8
Cross-attention head dimension	16
Self-attention heads	8

TABLE I : Architectural hyperparameters, shared by every configuration evaluated in this study.

Method / Parameter	Value
Waveform (1D) augmentations
AddNoise	$p\in[0.10,0.50]$ , SNR factor $\in[1,1000]$
Masking	$p\in[0.10,0.50]$ , mask $15$ – $30\%$ of length
Image (2D) augmentations
AugMix	$p=0.10$ , magnitude $3$
TrivialAugmentWide	$p=0.10$ , $31$ magnitude bins

TABLE II : Augmentation, regularization, and training configuration.

#	Task	Duration (sec)	State
Social Exposure
1	Neutral reference	120	N
2	Baseline description	120	N
3	Interview	120	S
Emotional Recall
4	Neutral reference	120	N

TABLE III : Experimental tasks employed in this study.

Split	Difficulty Group
	Q1 – Hard	Q2 – Med-Hard	Q3 – Med-Easy	Q4 – Easy
Training (38)	P017, P018, P022, P026, P034, P035, P042, P045, P050, P056	P001, P002, P003, P007, P012, P021, P033, P040, P048	P004, P014, P016, P032, P036, P046, P047, P052, P053, P054	P005, P010, P020, P028, P029, P037, P039, P041, P057
Validation (8)	P038, P055	P009, P023	P006, P013	P019, P030
Testing (12)	P008, P025, P044	P011, P024, P043	P015, P031, P058	P027, P051, P059

TABLE IV : Subject-level split by difficulty group. Subjects are ranked by combined z-score and assigned to four quartile-based groups (Q1 = hardest, Q4 = easiest).

Representation	Computational Cost		Validation				Testing
	Params (M)	GFLOPs	Accuracy	Precision	F1	Average	Accuracy
Waveform	2.00	0.97	61.78	61.66	61.40	61.61	67.36
Angle	1.86	0.86	66.13	66.00	66.02	66.05	57.78
Phase	1.86	0.86	71.30	72.21	71.48	71.66	62.78
PSD	1.86	0.86	68.33	70.26	68.39	68.99	69.44
Recurrence	1.86	0.86	66.42	67.13	66.52	66.69	65.69

TABLE V : Performance and computational cost using the EDA modality (400 epochs).

Representation	200 ep.	400 ep.	$\Delta$
Waveform	67.08	67.36	$+0.28$
Angle	57.78	57.78	$+0.00$
Phase	61.53	62.78	$+1.25$
PSD	69.44	69.44	$+0.00$
Recurrence	63.75	65.69	$+1.94$
Scalogram	63.75	64.86	$+1.11$

TABLE VI : Test accuracy at 200 vs. 400 training epochs.

Figure 3: Test accuracy across single representations, waveform $\oplus$ representation combinations, and the two full-fusion configurations, at the $400$ -epoch training budget. The excluded- Angle configuration reaches the highest test accuracy at $70.97\%$ ; PSD alone remains within $1.53$ points at $0.86$ vs. $1.58$ GFLOPs.

Explore similar work

Feb 20, 2026

A foundation model for electrodermal activity data

Foundation models have recently extended beyond natural language and vision to timeseries domains, including physiological signals. However, progress in electrodermal activity (EDA) modeling is hindered by the absence of large-scale, curated, and openly accessible datasets. EDA reflects sympathetic nervous system activity and is widely used to infer cognitive load, stress, and engagement. Yet very few wearable devices provide continuous, unobtrusive sensing, and the only large-scale archive to date is proprietary. To address this gap, we compile EDAMAME, a collection of EDA traces from 24 public datasets, comprising more than 25,000 hours from 634 users. Using this resource, we train UME, the first dedicated foundation model for EDA. In eight out of ten scenarios, UME outperforms baselines and matches generalist timeseries foundation models while using 20x fewer computational resources. Our findings, however, also highlight the intrinsic challenges of EDA modeling, motivating further research to unlock its full potential. All datasets, model weights, and code are released to support further research.

Leonardo Alchieri, Matteo Garzon, Lidia Alecci +4

Università della Svizzera Italiana (USI), Via la Santa 1, 6962 Lugano, Switzerland

Time-Series Foundation Models

Ask PDF

Mar 19, 2026

Stress Classification from ECG Signals Using Vision Transformer

Vision Transformers have shown tremendous success in numerous computer vision applications; however, they have not been exploited for stress assessment using physiological signals such as Electrocardiogram (ECG). In order to get the maximum benefit from the vision transformer for multilevel stress assessment, in this paper, we transform the raw ECG data into 2D spectrograms using short time Fourier transform (STFT). These spectrograms are divided into patches for feeding to the transformer encoder. We also perform experiments with 1D CNN and ResNet-18 (CNN model). We perform leave-onesubject-out cross validation (LOSOCV) experiments on WESAD and Ryerson Multimedia Lab (RML) dataset. One of the biggest challenges of LOSOCV based experiments is to tackle the problem of intersubject variability. In this research, we address the issue of intersubject variability and show our success using 2D spectrograms and the attention mechanism of transformer. Experiments show that vision transformer handles the effect of intersubject variability much better than CNN-based models and beats all previous state-of-the-art methods by a considerable margin. Moreover, our method is end-to-end, does not require handcrafted features, and can learn robust representations. The proposed method achieved 71.01% and 76.7% accuracies with RML dataset and WESAD dataset respectively for three class classification and 88.3% for binary classification on WESAD.

Zeeshan Ahmad, Naimul Khan

Electrocardiogram Classification Electrocardiography Vision Transformer

Ask PDF

Jul 1, 2026

I\textsuperscript{2}RiMA: Spectral Riemannian Representation with Temporal Attention for Mental Stress Detection based on EEG Signals

Cross-subject EEG stress detection remains challenging because discriminative stress-related patterns are both subject-dependent and frequency-specific. Conventional Riemannian methods model spatial covariance mainly in the time domain, overlooking neural oscillations that are critical for high-level cognitive state decoding, while standard temporal tokenization often fragments inter-slice temporal coherence. To address these limitations, we propose \method{}, an Intra-Inter Riemannian Manifold Attention Network for EEG-based stress detection. \method{} constructs spatial covariance matrices independently at each frequency point and maps them to the SPD tangent space, preserving channel-wise geometry together with frequency-specific discriminative cues. It further introduces frequency cluster aggregation to select informative spectral components and reduce redundancy by forming compact, data-driven frequency clusters aligned with EEG rhythms. Finally, an intra-inter slice attention module adaptively integrates local slice-level spectral dynamics and global temporal context across EEG sequences. Experiments on three datasets show that \method{} consistently outperforms five state-of-the-art baselines, achieving up to 82.78% balanced accuracy while remaining efficient with only 1.60M parameters and 31.95M FLOPs.

Cheng He, Kunyu Peng, Shangen Han +3

Laboratory of Neural Computing and Intelligent Perception, College of Information Engineering, Capital Normal University, Beijing, China · Karlsruhe Institute of Technology, Germany · School of Psychology, Capital Normal University, Beijing, China

Cross-Subject EEG Decoding Electroencephalography Frequency-Domain Feature Learning Geometric Representation Learning

Ask PDF