Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

Ioannis Kyprakis; Vasileios Skaramagkas; Georgia Karanasiou; Lampros Lakkas; Andri Papakonstantinou; Domen Ribnikar; Kalliopi Keramida; Dorothea Tsekoura

cs.LGApr 8, 2026

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

Authors: Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou, Lampros Lakkas, Andri Papakonstantinou, Domen Ribnikar, Kalliopi Keramida, Dorothea Tsekoura, +5 more

Organizations: ECE Dept. HMU FORTH-ICS Heraklion, Greece · FORTH-BRI Ioannina, Greece · Univ. of Ioannina (Med.) Physiology Lab Ioannina, Greece · Breast Center Karolinska Univ. Hosp. Stockholm, Sweden · Inst. of Oncology Ljubljana Ljubljana, Slovenia · NKUA Med. Sch.; Attikon Hosp., Agios Savvas Hosp. Athens, Greece · Aretaieio Univ. Hosp., NKUA, Athens, Greece · IEO Psychology Division; Univ. of Milan, Italy · BOCOC Nicosia, Cyprus

Abstract

Frailty and functional decline strongly influence treatment tolerance and outcomes in older patients with cancer, yet assessment is typically limited to infrequent clinic visits. We propose a multimodal wearable framework to estimate frailty-related functional change between visits in elderly breast cancer patients enrolled in the multicenter CARDIOCARE study. Free-living smartwatch physical activity and sleep features are combined with ECG-derived heart rate variability (HRV) features from a chest strap and organized into patient-horizon bags aligned to month 3 (M3) and month 6 (M6) follow-ups. Our innovation is an attention-based multiple instance learning (MIL) formulation that fuses irregular, multimodal wearable instances under real-world missingness and weak supervision. An attention-based MIL model with modality-specific multilayer perceptron (MLP) encoders with embedding dimension 128 aggregates variable-length and partially missing longitudinal instances to predict discretized change-from-baseline classes (worsened, stable, improved) for FACIT-F and handgrip strength. Under subject-independent leave-one-subject-out (LOSO) evaluation, the full multimodal model achieved balanced accuracy/F1 of 0.68 +/- 0.08/0.67 +/- 0.09 at M3 and 0.70 +/- 0.10/0.69 +/- 0.08 at M6 for handgrip, and 0.59 +/- 0.04/0.58 +/- 0.06 at M3 and 0.64 +/- 0.05/0.63 +/- 0.07 at M6 for FACIT-F. Ablation results indicated that smartwatch activity and sleep provide the strongest predictive information for frailty-related functional changes, while HRV contributes complementary information when fused with smartwatch streams.

Figures & tables

Task	0	1	2	Total
FACIT $\Delta$ M3	132	252	78	462
FACIT $\Delta$ M6	120	243	99	462
Handgrip $\Delta$ M3	75	124	34	233
Handgrip $\Delta$ M6	77	98	58	233

TABLE I: Class counts (0=worsened, 1=stable, 2=improved).

Modality	Stable		Worsened		Improved
Modality	M3	M6	M3	M6	M3	M6
Garmin Handgrip	20.8k	17.6k	11.5k	15.3k	8.1k	11.2k
Polar Belt Handgrip	5.5k	3.8k	7.6k	4.1k	1.9k	3.3k
Garmin FACIT	40.4k	41.9k	21.7k	18.6k	13.9k	17.6k
Polar FACIT	16.6k	7.9k	9.7k	4.5k	2.9k	4.0k

TABLE II: Number of available instances (in thousands, “k”) by modality, endpoint, follow-up horizon, and delta class (stable/worsened/improved).

Fig. 1: A graphical illustration of the proposed pipeline; the initial two images (human icons) were generated using Google Gemini [ 30 ] (synthetic images).

Endpoint	Metric	M3	M6
Hg	Balanced Accuracy	$0.68\pm 0.08$	$\mathbf{0.70\pm 0.10}$
Hg	F1 score	$0.67\pm 0.09$	$\mathbf{0.69\pm 0.08}$
FACIT	Balanced Accuracy	$0.59\pm 0.04$	$\mathbf{0.64\pm 0.05}$
FACIT	F1 score	$0.58\pm 0.06$	$\mathbf{0.63\pm 0.07}$

TABLE III: Overall results (mean

\pm

std across LOSO iterations) for the full multimodal MIL model.

Split	Endpoint	BalAcc M3	F1 M3	BalAcc M6	F1 M6
P+S	Hg	0.65 $\pm$ 0.04	0.63 $\pm$ 0.02	0.67 $\pm$ 0.05	0.65 $\pm$ 0.03
P+S	FACIT	0.52 $\pm$ 0.07	0.50 $\pm$ 0.08	0.57 $\pm$ 0.06	0.55 $\pm$ 0.04
P+E	Hg	0.60 $\pm$ 0.04	0.61 $\pm$ 0.03	0.62 $\pm$ 0.05	0.62 $\pm$ 0.04
P+E	FACIT	0.50 $\pm$ 0.02	0.49 $\pm$ 0.01	0.51 $\pm$ 0.03	0.51 $\pm$ 0.04
S+E	Hg	0.50 $\pm$ 0.03	0.49 $\pm$ 0.05	0.54 $\pm$ 0.05	0.55 $\pm$ 0.03
S+E	FACIT	0.46 $\pm$ 0.05	0.46 $\pm$ 0.01	0.48 $\pm$ 0.06	0.49 $\pm$ 0.05

TABLE IV: Ablation results (mean

\pm

std across LOSO iterations) using modality pairs.

Explore similar work

Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning
Apr 8, 2026Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou +10Oncology Wearable Sensing
ECE Dept. HMU; FORTH-ICS; Heraklion, Greece · FORTH-BRI; Ioannina, Greece · Ioannina Univ. Hosp., UoI Med. Sch.; Ioannina, Greece +6
The Gait Signature of Frailty: Transfer Learning based Deep Gait Models for Scalable Frailty Assessment
Mar 25, 2026Laura McDaniel, Basudha Pal, Crystal Szczesny +5Gait Analysis Transfer Learning
Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA. · Division of Geriatrics and Gerontology Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA. · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. +3
Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multi-Cancer Cohort
Mar 31, 2026Franco Rugolon, Korbinian Randl, Braslav Jovanovic +2Clinical Outcome Prediction Clinical Risk Prediction
Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden

Apr 8, 2026J/K move · Enter open · S save

Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning

Ioannis Kyprakis, Vasileios Skaramagkas, Georgia Karanasiou, Vasilis Bouratzis+9

ECE Dept. HMU; FORTH-ICS; Heraklion, Greece · FORTH-BRI; Ioannina, Greece · Ioannina Univ. Hosp., UoI Med. Sch.; Ioannina, Greece+6

Oncology Wearable Sensing

Open paper AskPDF arXiv

Psychological stress is clinically relevant in cardio-oncology, yet it is typically assessed only through patient-reported outcome measures (PROMs) and is rarely integrated into continuous cardiotoxicity surveillance. We estimate perceived stress in an elderly, multicenter breast cancer cohort (CARDIOCARE) using multimodal wearable data from a smartwatch (physical activity and sleep) and a chest-worn ECG sensor. Wearable streams are transformed into heterogeneous visual representations, yielding a weakly supervised setting in which a single Perceived Stress Scale (PSS) score corresponds to many unlabeled windows. A lightweight pretrained mixture-of-experts backbone (Tiny-BioMoE) embeds each representation into 192-dimensional vectors, which are aggregated via attention-based multiple instance learning (MIL) to predict PSS at month 3 (M3) and month 6 (M6). Under leave-one-subject-out (LOSO) evaluation, predictions showed moderate agreement with questionnaire scores (M3: R^2=0.24, Pearson r=0.42, Spearman rho=0.48; M6: R^2=0.28, Pearson r=0.49, Spearman rho=0.52), with global RMSE/MAE of 6.62/6.07 at M3 and 6.13/5.54 at M6.

Abstract

Explore similar work

Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning

The Gait Signature of Frailty: Transfer Learning based Deep Gait Models for Scalable Frailty Assessment

Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multi-Cancer Cohort