Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection

Stefanos Gkikas; Thomas Kassiotis; Yang Guo; Guangliang Li; Eric Nichols; Houshyar Asadi; Nikolaos Smyrnis; Giorgos Giannakakis

eess.SPAug 10, 2026

Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection

Authors: Stefanos Gkikas, Thomas Kassiotis, Yang Guo, Guangliang Li, Eric Nichols, Houshyar Asadi, Nikolaos Smyrnis, Giorgos Giannakakis

Organizations: Honda Research Institute Japan Wako City, Japan · Department of Electronic Engineering Hellenic Mediterranean University Chania, Greece · Faculty of Information Science and Engineering Ocean University of China Qingdao, China · Institute for Intelligent Systems Research and Innovations (IISRI) Deakin University Geelong, Australia · University Mental Health, Neurosciences and Precision Medicine, Research Institute “COSTAS STEFANIS” 2nd Department of Psychiatry, Medical School, National and Kapodistrian University of Athens Athens, Greece

Abstract

Electrodermal activity (EDA) is widely used in automatic stress detection, yet most pipelines treat it only as a raw one-dimensional waveform. This study examines whether complementary visual representations of EDA provide useful information for stress classification and whether their fusion im- proves recognition performance. Six image-based representations are derived from each EDA recording: an unwrapped short-time Fourier transform (STFT) phase spectrogram, an instantaneous-frequency map computed from that phase, a power spectral density (PSD) spectrogram, a continuous wavelet transform scalogram, a recurrence plot, and a rendered waveform trace. The selected representations are stacked as channels of a single multichannel input, together with the raw waveform, and processed by a shared asymmetric-attention architecture. Experiments on a 58-subject stress dataset show that representation fusion improves over the raw waveform. The best configuration, which combines five representations while excluding the unwrapped phase spectrogram, reaches 70.97% test accuracy, compared with 67.36% for the raw waveform. The single PSD spectrogram achieves 69.44%, remaining close to the best-fused configuration at a lower computational cost. The results show that alternative visual forms of the same EDA signal can provide useful inductive biases for stress detection, and that a compact selection of complementary representations can be more effective than the raw waveform alone.

Figures & tables

Figure 1: Example of the six visual representations derived from a single EDA recording. Top row (left to right): Angle , Phase , PSD . Bottom row (left to right): Recurrence , Scalogram , Wave .

Figure 2: Overview of the proposed pipeline. Each EDA recording produces the raw waveform and, in parallel, six visual representations. Representations are flattened into one-dimensional channels, stacked together with the raw waveform, and processed by the shared asymmetric-attention backbone for binary stress classification.

Hyperparameter	Value
Depth	1
Number of latents	64
Latent dimension ( $d_{0}$ )	128
Cross-attention heads	8
Cross-attention head dimension	16
Self-attention heads	8

TABLE I : Architectural hyperparameters, shared by every configuration evaluated in this study.

Method / Parameter	Value
Waveform (1D) augmentations
AddNoise	$p\in[0.10,0.50]$ , SNR factor $\in[1,1000]$
Masking	$p\in[0.10,0.50]$ , mask $15$ – $30\%$ of length
Image (2D) augmentations
AugMix	$p=0.10$ , magnitude $3$
TrivialAugmentWide	$p=0.10$ , $31$ magnitude bins

TABLE II : Augmentation, regularization, and training configuration.

#	Task	Duration (sec)	State
Social Exposure
1	Neutral reference	120	N
2	Baseline description	120	N
3	Interview	120	S
Emotional Recall
4	Neutral reference	120	N

TABLE III : Experimental tasks employed in this study.

Split	Difficulty Group
	Q1 – Hard	Q2 – Med-Hard	Q3 – Med-Easy	Q4 – Easy
Training (38)	P017, P018, P022, P026, P034, P035, P042, P045, P050, P056	P001, P002, P003, P007, P012, P021, P033, P040, P048	P004, P014, P016, P032, P036, P046, P047, P052, P053, P054	P005, P010, P020, P028, P029, P037, P039, P041, P057
Validation (8)	P038, P055	P009, P023	P006, P013	P019, P030
Testing (12)	P008, P025, P044	P011, P024, P043	P015, P031, P058	P027, P051, P059

TABLE IV : Subject-level split by difficulty group. Subjects are ranked by combined z-score and assigned to four quartile-based groups (Q1 = hardest, Q4 = easiest).

Representation	Computational Cost		Validation				Testing
	Params (M)	GFLOPs	Accuracy	Precision	F1	Average	Accuracy
Waveform	2.00	0.97	61.78	61.66	61.40	61.61	67.36
Angle	1.86	0.86	66.13	66.00	66.02	66.05	57.78
Phase	1.86	0.86	71.30	72.21	71.48	71.66	62.78
PSD	1.86	0.86	68.33	70.26	68.39	68.99	69.44
Recurrence	1.86	0.86	66.42	67.13	66.52	66.69	65.69

TABLE V : Performance and computational cost using the EDA modality (400 epochs).

Representation	200 ep.	400 ep.	$\Delta$
Waveform	67.08	67.36	$+0.28$
Angle	57.78	57.78	$+0.00$
Phase	61.53	62.78	$+1.25$
PSD	69.44	69.44	$+0.00$
Recurrence	63.75	65.69	$+1.94$
Scalogram	63.75	64.86	$+1.11$

TABLE VI : Test accuracy at 200 vs. 400 training epochs.

Figure 3: Test accuracy across single representations, waveform $\oplus$ representation combinations, and the two full-fusion configurations, at the $400$ -epoch training budget. The excluded- Angle configuration reaches the highest test accuracy at $70.97\%$ ; PSD alone remains within $1.53$ points at $0.86$ vs. $1.58$ GFLOPs.

Explore similar work

A foundation model for electrodermal activity data
Feb 20, 2026Leonardo Alchieri, Matteo Garzon, Lidia Alecci +4Time-Series Foundation Models
Università della Svizzera Italiana (USI), Via la Santa 1, 6962 Lugano, Switzerland
Stress Classification from ECG Signals Using Vision Transformer
Mar 19, 2026Zeeshan Ahmad, Naimul KhanElectrocardiogram Classification Electrocardiography
I\textsuperscript{2}RiMA: Spectral Riemannian Representation with Temporal Attention for Mental Stress Detection based on EEG Signals
Jul 1, 2026Cheng He, Kunyu Peng, Shangen Han +3Cross-Subject EEG Decoding Electroencephalography
Laboratory of Neural Computing and Intelligent Perception, College of Information Engineering, Capital Normal University, Beijing, China · Karlsruhe Institute of Technology, Germany · School of Psychology, Capital Normal University, Beijing, China

Feb 20, 2026J/K move · Enter open · S save

A foundation model for electrodermal activity data

Leonardo Alchieri, Matteo Garzon, Lidia Alecci, Francesco Bombassei De Bona+3

Università della Svizzera Italiana (USI), Via la Santa 1, 6962 Lugano, Switzerland

Time-Series Foundation Models

Open paper AskPDF arXiv

Foundation models have recently extended beyond natural language and vision to timeseries domains, including physiological signals. However, progress in electrodermal activity (EDA) modeling is hindered by the absence of large-scale, curated, and openly accessible datasets. EDA reflects sympathetic nervous system activity and is widely used to infer cognitive load, stress, and engagement. Yet very few wearable devices provide continuous, unobtrusive sensing, and the only large-scale archive to date is proprietary. To address this gap, we compile EDAMAME, a collection of EDA traces from 24 public datasets, comprising more than 25,000 hours from 634 users. Using this resource, we train UME, the first dedicated foundation model for EDA. In eight out of ten scenarios, UME outperforms baselines and matches generalist timeseries foundation models while using 20x fewer computational resources. Our findings, however, also highlight the intrinsic challenges of EDA modeling, motivating further research to unlock its full potential. All datasets, model weights, and code are released to support further research.

Abstract

Explore similar work

A foundation model for electrodermal activity data

Stress Classification from ECG Signals Using Vision Transformer

I\textsuperscript{2}RiMA: Spectral Riemannian Representation with Temporal Attention for Mental Stress Detection based on EEG Signals