cs.HCOct 2, 2026
SaveCharacterizing the Performance Gap in Human Activity Recognition for Older Adults
Organizations: University of Maryland College Park, Maryland, USA · Boston University Boston, Massachusetts, USA · University of Michigan Ann Arbor, Michigan, USA
Abstract
Human activity recognition (HAR) from wrist-worn accelerometers is increasingly used for health and behavioral tracking. Yet, most wearable HAR models are developed and evaluated on datasets dominated by younger adults, leaving it unclear whether benchmark progress generalizes across age groups. In this work, we leverage MyMove, our carefully annotated, free-living older-adult HAR dataset (mean age 71), to evaluate deep-learning architectures and training regimes under both leave-one-subject-out and cross-dataset transfer. We find that improvements on younger-adult benchmarks fail to transfer equally to data collected from older adults, resulting in a persistent and often widening performance gap. However, richer representations, particularly frozen self-supervised features pretrained on the age-diverse UK Biobank dataset, substantially improve performance on data from older adults and consistently narrow the performance gap, at modest cost to younger-adult performance, though disparities remain. These findings suggest that benchmark gains and architectural scaling alone provide an incomplete picture of progress in wearable HAR, and broader advances may require representations that better capture population diversity, alongside personalized adaptation to individual movement patterns and routines.
Figures & tables
| Dataset | Pop. | Age | #Part. | #Act. 1 | #Win |
| RealWorld (RWy) ( Sztyler and Stuckenschmidt, 2016 ) | younger | 29.7 (16–52) | 14 | 5 | 8,900 |
| RealWorld (RWo) ( Sztyler and Stuckenschmidt, 2016 ) | older | 62 | 1 | 5 | 736 |
| MyMove (MMo) | older | 71.1 (61–90) | 13 | 5 | 17,160 |
| MyMove (MMy) | younger | 25.5 (20–31) | 2 | 3 | 418 |
| WISDM ( Weiss, 2019 ) | students | – | 46 | 4 | 6,230 |
| UK Biobank ( Doherty et al., 2017 ) | broader | 62 (45–79) | 103k | – | 1B |
Table 1. Smartwatch-based human activity recognition datasets used in the study. 1 The number of activities reported here is after excluding uncommon activities from datasets.
Figure 1. Activity composition across datasets; each row sums to 100% of that population’s labeled windows. Horizontal stacked bars of activity-class proportions in each dataset. Horizontal stacked bar chart comparing activity-class proportions across five datasets, grouped into scripted (WISDM, RealWorld younger, RealWorld older) and free-living (MyMove older, MyMove younger). Each bar spans the full width and is divided into five segments for Lying, Sitting, Standing, Walking, and Running. The scripted datasets are close to balanced: WISDM is roughly a quarter each of Sitting, Standing, Walking, and Running with no Lying; RealWorld younger is near 20 percent for every class; RealWorld older is similar but 30 percent Running. The free-living datasets are heavily skewed toward stationary activity: MyMove older is 57.9 percent Sitting, 29.0 Standing, 11.0 Walking, 1.8 Lying, and 0.3 Running; MyMove younger is 86.6 percent Sitting, 4.3 Standing, and 9.1 Walking, with no Lying or Running.
| Feature | Description | Source |
| mean_{x,y,z} | Per-axis mean | ( Yuan et al., 2024 ) |
| std_{x,y,z} | Per-axis standard deviation | ( Yuan et al., 2024 ) |
| range_{x,y,z} | Per-axis range (max min) | ( Yuan et al., 2024 ) |
| mag_{mean,std,range} | Vector-magnitude mean, standard deviation, range | ( Yuan et al., 2024 ) |
| mag_mad | Median absolute deviation of magnitude | ( Yuan et al., 2024 ) |
| mag_{kurtosis,skew} | Kurtosis and skewness of magnitude | ( Yuan et al., 2024 ; Narayanswamy et al., 2024 ) |
Table 2. Thirty-two handcrafted accelerometry features typically appearing in prior HAR literature e.g. , ( Yuan et al., 2024 ; Narayanswamy et al., 2024 ) .
Figure 2. Benchmark gains on younger adults fail to transfer equally to older adults. Architectures are sorted by RWy performance; parameter counts in parentheses. Line chart of leave-one-subject-out macro F1 for six architectures, by population. Line chart with three series over six deep-learning architectures on the horizontal axis, ordered left to right by ascending RealWorld younger-adult performance: MLP (91k parameters), MobileHART (206k), DeepConvLSTM (295k), Transformer (436k), AttnTCN (130k), and LIMUBert (134k). The vertical axis is pooled macro F1 from 0.2 to 0.9. The RealWorld younger series stays highest, rising gently from about 0.76 to 0.81. The RealWorld older single-subject series is in the middle, varying between about 0.56 and 0.70. The MyMove older series stays lowest, between about 0.34 and 0.57, and does not follow the left-to-right ordering: it peaks at AttnTCN rather than at the architectures that are best for younger adults. The wide and roughly constant gap between the younger and MyMove older series is the key point: gains along the younger-adult ordering do not reach older adults.
Figure 3. Zero-shot benchmark gains on younger adults fail to transfer equally to older adults. Architectures are sorted by WISDM RWy performance. Line chart of zero-shot cross-dataset macro F1 for six architectures, by population. Line chart of zero-shot cross-dataset performance, with models trained on WISDM and evaluated without fine-tuning. Six architectures on the horizontal axis are ordered left to right by ascending WISDM-to-RealWorld younger performance: MLP (91k parameters), AttnTCN (130k), Transformer (436k), LIMUBert (134k), MobileHART (206k), and DeepConvLSTM (295k). The vertical axis is pooled macro F1 from 0.2 to 0.9, and all three series are dashed. The RealWorld younger series rises from about 0.71 to 0.83. The RealWorld older single-subject series tracks close to it, dipping to about 0.62 at Transformer. The MyMove older series stays well below, between about 0.39 and 0.60. As in the leave-one-subject-out figure, the MyMove older series remains far under the younger series across all architectures, so benchmark gains again fail to transfer to older adults under cross-dataset evaluation.
| Evaluation | Group | Lie | Sit | Stand | Walk | Run |
|---|---|---|---|---|---|---|
| LOSO | MMo | 13% | 81% | 58% | 64% | 70% |
| RWy | 70% | 69% | 80% | 97% | 88% | |
| Cross-dataset | MMo | – | 67% | 45% | 45% | 73% |
| RWy | – | 50% | 67% | 94% | 93% |
Table 3. Per-class F1 for AttnTCN under LOSO and cross-dataset (WISDM RealWorld/MyMove) evaluation.
Figure 4. Increasingly structured representations produce more aligned performance between younger and older adults, with the frozen SSL-pretrained ResNet backbone strongest. (a) LOSO mean macro F1 (95% CIs) for RealWorld younger and MyMove older adults, without vs. with SSL. (b,c) LOSO and cross-dataset (Train:WISDM) pooled macro F1 by population across the six configurations; (d) similar to (a), for cross-dataset. Four panels showing structured representations narrow the younger-older performance gap. Four panels sharing a vertical axis of macro F1 from 0.2 to 0.9. Panels a and d are narrow interaction plots; panels b and c are wider line charts. In panel a (leave-one-subject-out), two lines connect performance without self-supervised learning to performance with it: RealWorld younger rises slightly from about 0.77 to 0.80, while MyMove older rises steeply from about 0.54 to 0.68, so the lines converge; an annotation reports a significant cohort-by-representation interaction, chi-squared 19.3 and p below ten to the minus four. Panels b and c plot pooled macro F1 across six MLP-head configurations ordered by increasing representation quality (standalone MLP, frozen random ResNet, trained ResNet, handcrafted features, ResNet-SSL, frozen ResNet-SSL) for RealWorld younger, a RealWorld older subject, and MyMove older; older-adult performance climbs steadily along this ordering under both leave-one-subject-out (b) and cross-dataset training on WISDM (c), with frozen ResNet-SSL highest. Panel d repeats the panel-a interaction for cross-dataset transfer, where both cohorts improve in parallel and the interaction is not significant (chi-squared 0.55, p equals 0.46). Richer representations lift older adults most under leave-one-subject-out, narrowing the age gap, but not under cross-dataset transfer.
Figure 5. Adding the frozen SSL-pretrained backbone concentrates predictions more strongly along the diagonal, particularly for older adults, relative to the standalone MLP. Four confusion matrices comparing MLP and frozen-SSL predictions for younger and older adults. Two-by-two grid of five-by-five confusion matrices, with true class on the vertical axis and predicted class on the horizontal axis over Lying, Sitting, Standing, Walking, and Running; darker cells mean higher probability and correct predictions lie on the diagonal. The top row is RealWorld younger adults, the bottom row is MyMove older adults; the left column is the standalone MLP and the right column adds a frozen self-supervised ResNet backbone. For younger adults the diagonal is already strong under the MLP and strengthens with the SSL backbone, for example Walking recall rising from 0.82 to 0.98. For MyMove older adults the MLP is weak and off-diagonal, with Lying recall only 0.24 (mostly confused with Sitting), Standing 0.41, and Walking 0.49; adding the frozen SSL backbone concentrates predictions along the diagonal, raising Lying to 0.39, Sitting to 0.87, Standing to 0.63, Walking to 0.77, and Running to 0.89. The largest gains appear for older adults, especially standing and walking, which the MLP frequently confused.
Figure 6. UMAP ( McInnes et al., 2018 ) projections for our older-adult dataset. Limited separation, e.g. , for Standing and Walking , is shown. Three UMAP projections of older-adult windows under raw, handcrafted, and SSL representations. Three side-by-side UMAP scatter plots of the same MyMove older-adult accelerometer windows, colored by activity (Lying, Sitting, Standing, Walking, Running), under three representations: raw accelerometer signal, handcrafted features, and a frozen self-supervised ResNet. In the raw-signal panel the classes are heavily intermixed with no clear clusters. In the handcrafted-features panel a distinct Running cluster separates out, but Sitting, Standing, and Walking still overlap substantially. In the frozen self-supervised panel the structure is clearest: Running forms an isolated cluster, Lying and Sitting gather on one side and Walking on another, yet Standing remains partially overlapping with both Sitting and Walking. Richer representations increasingly separate the activities for older adults, but the stationary Standing-versus-Walking distinction stays hard to resolve even with the best representation.