cs.LGFeb 26, 2025

Cross-Modality Investigation on WESAD Stress Classification

Authors: Eric Oliver, Sagnik Dakshit

Organizations: The University of Texas at Tyler, USA

Abstract

Deep learning's growing prevalence has driven its widespread use in healthcare, where AI and sensor advancements enhance diagnosis, treatment, and monitoring. In mobile health, AI-powered tools enable early diagnosis and continuous monitoring of conditions like stress. Wearable technologies and multimodal physiological data have made stress detection increasingly viable, but model efficacy depends on data quality, quantity, and modality. This study develops transformer models for stress detection using the WESAD dataset, training on electrocardiograms (ECG), electrodermal activity (EDA), electromyography (EMG), respiration rate (RESP), temperature (TEMP), and 3-axis accelerometer (ACC) signals. The results demonstrate the effectiveness of single-modality transformers in analyzing physiological signals, achieving state-of-the-art performance with accuracy, precision and recall values in the range of 99.73%99.73\% to 99.95%99.95\% for stress detection. Furthermore, this study explores cross-modal performance and also explains the same using 2D visualization of the learned embedding space and quantitative analysis based on data variance. Despite the large body of work on stress detection and monitoring, the robustness and generalization of these models across different modalities has not been explored. This research represents one of the initial efforts to interpret embedding spaces for stress detection, providing valuable information on cross-modal performance.

Figures & tables

Explore similar work

CardsList
  1. Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signals

    Jun 12, 2026Desta Haileselassie Hagos, Saurav Keshari Aryal, Patrick Ymele-Leki +2Temporal Convolutional NetworksDeep Ensembles

  2. From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities

    Sep 17, 2026Desta Haileselassie Hagos, Saurav Keshari Aryal, Legand L. BurgeEmotion RecognitionMultimodal Emotion Recognition

  3. Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detection

    Jun 23, 2026Louis Simon, Mohamed ChetouaniWearable Health Monitoring