Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning

Ioannis Kyprakis; Vasileios Skaramagkas; Georgia Karanasiou; Vasilis Bouratzis; Andri Papakonstantinou; Dimitar Stefanovski; Kalliopi Keramida; Aristofania Simatou

cs.LGApr 8, 2026

Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning

Authors: Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou, Vasilis Bouratzis, Andri Papakonstantinou, Dimitar Stefanovski, Kalliopi Keramida, Aristofania Simatou, +5 more

Organizations: ECE Dept. HMU; FORTH-ICS; Heraklion, Greece · FORTH-BRI; Ioannina, Greece · Ioannina Univ. Hosp., UoI Med. Sch.; Ioannina, Greece · Breast Center; Karolinska Univ. Hosp.; Stockholm, Sweden · Inst. of Oncology Ljubljana; Ljubljana, Slovenia · NKUA Med. Sch.; Attikon Hosp., Agios Savvas Hosp.; Athens, Greece · Agios Savvas Hosp.; Athens, Greece · IEO Psychology Division; Univ. of Milan, Italy · BOCOC; Nicosia, Cyprus

Abstract

Psychological stress is clinically relevant in cardio-oncology, yet it is typically assessed only through patient-reported outcome measures (PROMs) and is rarely integrated into continuous cardiotoxicity surveillance. We estimate perceived stress in an elderly, multicenter breast cancer cohort (CARDIOCARE) using multimodal wearable data from a smartwatch (physical activity and sleep) and a chest-worn ECG sensor. Wearable streams are transformed into heterogeneous visual representations, yielding a weakly supervised setting in which a single Perceived Stress Scale (PSS) score corresponds to many unlabeled windows. A lightweight pretrained mixture-of-experts backbone (Tiny-BioMoE) embeds each representation into 192-dimensional vectors, which are aggregated via attention-based multiple instance learning (MIL) to predict PSS at month 3 (M3) and month 6 (M6). Under leave-one-subject-out (LOSO) evaluation, predictions showed moderate agreement with questionnaire scores (M3: R^2=0.24, Pearson r=0.42, Spearman rho=0.48; M6: R^2=0.28, Pearson r=0.49, Spearman rho=0.52), with global RMSE/MAE of 6.62/6.07 at M3 and 6.13/5.54 at M6.

Figures & tables

	Spectrograms
Modality	Representation	M3		M6
		Low	Elev.	Low	Elev.
Activity	Weekly heatmaps	2028	2721	1629	1943
Sleep	Weekly heatmaps	1162	2071	1067	1344
Sleep	Hypnograms	9464	11532	7647	9298
ECG	Recurrence plots	3102	5310	1873	2715

TABLE I: Distribution of visual representation instances per modality, stress level, and time point.

Fig. 1: Example of visual representations used. From left to right: Physical Activity Heatmap, Hypnogram, Sleep Heatmap, Scalogram, Recurence, Poincaré, Spectrogam

Fig. 2: A schematic illustration of the proposed pipeline; The first two images (human icons) were generated using Google Gemini [ 30 ] (synthetic images).

Component	Params (M)	FLOPs (G)
Tiny-BioMoE	$7.340$	$3.04$
Attention-MIL	$0.215$	$1.21$
Total	7.555	4.25

TABLE II: Model complexity (parameters and FLOPs).

Metric	M3	M6
RMSE (mean $\pm$ std)	4.17 $\pm$ 3.24	3.34 $\pm$ 3.11
Global RMSE	6.62	6.13
Global MAE	6.07	5.54
Global $R^{2}$	0.24	0.28
Global Pearson $r$	0.42	0.49
Global Spearman $\rho$	0.48	0.52

TABLE III: LOSO performance using all modalities. Global metrics are computed over pooled out-of-subject predictions.

Metric	P+S	P+E	S+E
M3 RMSE (m $\pm$ s)	6.00 $\pm$ 4.98	6.04 $\pm$ 4.95	6.10 $\pm$ 4.94
M3 Global RMSE	7.79	7.81	7.85
M6 RMSE (m $\pm$ s)	5.60 $\pm$ 4.80	5.53 $\pm$ 4.71	5.50 $\pm$ 4.72
M6 Global RMSE	7.09	7.12	7.15

TABLE IV: Ablation study (pairwise modality fusion) under LOSO, transposed for compactness. P=Physical activity, S=Sleep, E=ECG.

Explore similar work

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning
Apr 8, 2026Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou +10Clinical Prediction Clinical Risk Prediction
ECE Dept. HMU FORTH-ICS Heraklion, Greece · FORTH-BRI Ioannina, Greece · Univ. of Ioannina (Med.) Physiology Lab Ioannina, Greece +6
AdaptStress: Online Adaptive Learning for Interpretable and Personalized Stress Prediction Using Multivariate and Sparse Physiological Signals
Feb 19, 2026Xueyi Wang, Claudine J. C. Lamoth, Elisabeth WilhelmWearable Health Monitoring Explainable Time Series Forecasting
Engineering and Technology institute Groningen (ENTEG), University of Groningen, Groningen, 9742AG, the Netherlands · Department of Human Movement Sciences, University Medical Center Groningen, Groningen, Netherlands
Stress Classification from ECG Signals Using Vision Transformer
Mar 19, 2026Zeeshan Ahmad, Naimul KhanElectrocardiogram Classification Electrocardiography

Apr 8, 2026J/K move · Enter open · S save

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou, Lampros Lakkas+9

ECE Dept. HMU FORTH-ICS Heraklion, Greece · FORTH-BRI Ioannina, Greece · Univ. of Ioannina (Med.) Physiology Lab Ioannina, Greece+6

Clinical Prediction Clinical Risk Prediction Wearable Health Monitoring Oncology Multiple Instance Learning

Open paper AskPDF arXiv

Frailty and functional decline strongly influence treatment tolerance and outcomes in older patients with cancer, yet assessment is typically limited to infrequent clinic visits. We propose a multimodal wearable framework to estimate frailty-related functional change between visits in elderly breast cancer patients enrolled in the multicenter CARDIOCARE study. Free-living smartwatch physical activity and sleep features are combined with ECG-derived heart rate variability (HRV) features from a chest strap and organized into patient-horizon bags aligned to month 3 (M3) and month 6 (M6) follow-ups. Our innovation is an attention-based multiple instance learning (MIL) formulation that fuses irregular, multimodal wearable instances under real-world missingness and weak supervision. An attention-based MIL model with modality-specific multilayer perceptron (MLP) encoders with embedding dimension 128 aggregates variable-length and partially missing longitudinal instances to predict discretized change-from-baseline classes (worsened, stable, improved) for FACIT-F and handgrip strength. Under subject-independent leave-one-subject-out (LOSO) evaluation, the full multimodal model achieved balanced accuracy/F1 of 0.68 +/- 0.08/0.67 +/- 0.09 at M3 and 0.70 +/- 0.10/0.69 +/- 0.08 at M6 for handgrip, and 0.59 +/- 0.04/0.58 +/- 0.06 at M3 and 0.64 +/- 0.05/0.63 +/- 0.07 at M6 for FACIT-F. Ablation results indicated that smartwatch activity and sleep provide the strongest predictive information for frailty-related functional changes, while HRV contributes complementary information when fused with smartwatch streams.

Abstract

Explore similar work

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

AdaptStress: Online Adaptive Learning for Interpretable and Personalized Stress Prediction Using Multivariate and Sparse Physiological Signals

Stress Classification from ECG Signals Using Vision Transformer