The Gait Signature of Frailty: Transfer Learning based Deep Gait Models for Scalable Frailty Assessment
Authors: Laura McDaniel, Basudha Pal, Crystal Szczesny, Yuxiang Guo, Zhaoyang Wang, Ryan Roemmich, Peter Abadir, Rama Chellappa
Organizations: Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA. · Division of Geriatrics and Gerontology Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA. · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. · Center for Movement Studies, Kennedy Krieger Institute, Baltimore, MD, USA. · Department of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, USA. · Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
Frailty is a condition in aging medicine characterized by diminished physiological reserve and increased vulnerability to stressors. However, frailty assessment remains subjective, heterogeneous, and difficult to scale in clinical practice. Gait is a sensitive marker of biological aging, capturing multisystem decline before overt disability. Yet the application of modern computer vision to gait-based frailty assessment has been limited by small, imbalanced datasets and a lack of clinically representative benchmarks. In this work, we introduce a publicly available silhouette-based frailty gait dataset collected in a clinically realistic setting, spanning the full frailty spectrum and including older adults who use walking aids. Using this dataset, we evaluate how pretrained gait recognition models can be adapted for frailty classification under limited data conditions. We study both convolutional and hybrid attention-based architectures and show that predictive performance depends primarily on how pretrained representations are transferred rather than architectural complexity alone. Across models, selectively freezing low-level gait representations while allowing higher-level features to adapt yields more stable and generalizable performance than either full fine-tuning or rigid freezing. Conservative handling of class imbalance further improves training stability, and combining complementary learning objectives enhances discrimination between clinically adjacent frailty states. Interpretability analyses reveal consistent model attention to lower-limb and pelvic regions, aligning with established biomechanical correlates of frailty. Together, these findings establish gait-based representation learning as a scalable, non-invasive, and interpretable framework for frailty assessment and support the integration of modern biometric modeling approaches into aging research and clinical practice.
Figures & tables
Figure 1: The figure presents the overall framework from clinical motivation to model evaluation. Frailty is introduced as a multidimensional geriatric syndrome motivating objective gait based assessment. Clinical labels are derived using the Fried frailty phenotype defined by five criteria exhaustion, weight loss, slow gait speed, weak grip strength, and low energy, with mobility impairment particularly slow gait speed serving as a core marker of physical frailty. Gait data are collected during a ten meter self paced walk using markerless capture, followed by segmentation and silhouette frame sequencing to construct a labeled silhouette dataset. The conceptual trajectory of age related physical frailty highlights progressive gait deterioration as a primary functional manifestation. The dataset is used to train and compare SwinGait and DeepGait architectures under different finetuning strategies. The bottom panel summarizes demographic characteristics of the study cohort stratified by gender.
Experiment
Micro AUC (%)
Cohen’s Kappa
CNN
PE
Swin S1
Swin S2
SwinGait M1
0.7721 ± 0.0664
0.5923 ± 0.1128
×
×
×
×
SwinGait M2
0.7790 ± 0.0910
0.6190 ± 0.1465
✓
×
×
×
SwinGait M3
0.7513 ± 0.0838
0.5903 ± 0.1301
✓
✓
×
×
SwinGait M4
0.7250 ± 0.0865
0.5103 ± 0.2126
✓
✓
✓
×
SwinGait M5
0.6316 ± 0.0713
0.2110 ± 0.1447
✓
✓
✓
✓
Table 1: Performance summary under different freezing strategies. Patch Embedding (PE) is treated as a learnable projection layer. Here ✓denotes freezing while × denotes trainable.
Experiment
Frozen Layers
Micro AUC (%)
Cohen’s Kappa
DeepGaitV2 D0
None
0.7348 ± 0.0548
0.5787 ± 0.0335
DeepGaitV2 D1
Layer 0
0.7788 ± 0.0312
0.5228 ± 0.0889
DeepGaitV2 D2
Layers 0–1
0.7612 ± 0.0594
0.5657 ± 0.0764
DeepGaitV2 D3
Layers 0–2
0.7407 ± 0.0306
0.4947 ± 0.1241
DeepGaitV2 D4
Layers 0–3
0.7458 ± 0.0623
0.5308 ± 0.0584
DeepGaitV2 D5
Layers 0–4
0.6961 ± 0.0567
0.4412 ± 0.0811
Table 2: Performance summary for DeepGaitV2 under different backbone freezing strategies. Convolutional layers are indexed from shallow (Layer 0) to deep (Layer 4).
Backbone
Configuration
Frozen Components
Class Weights
Micro AUC
Cohen’s Kappa
SwinGait
Frozen CNN (M2)
CNN frozen; PE, S1, S2 trainable
No
0.7790 ± 0.0910
0.6190 ± 0.1465
SwinGait
Frozen CNN (M2)
CNN frozen; PE, S1, S2 trainable
Yes
0.7683 ± 0.0858
0.5658 ± 0.0824
SwinGait
Unfrozen CNN
All backbone components trainable
No
0.7721 ± 0.0664
0.5923 ± 0.1128
SwinGait
Unfrozen CNN
All backbone components trainable
Yes
0.7631 ± 0.0685
0.6091 ± 0.0981
DeepGaitV2
Partially Frozen (D1)
Layer 0 frozen
No
0.7788 ± 0.0312
0.5228 ± 0.0889
DeepGaitV2
Partially Frozen (D1)
Layer 0 frozen
Yes
0.7852 ± 0.0558
0.5248 ± 0.0765
Table 3: Transfer-learning comparison between SwinGait and DeepGaitV2 under selected best freezing strategies with and without inverse square-root class weighting. For SwinGait, the “Frozen CNN” configuration corresponds to M2 (CNN frozen, patch embedding and both Swin stages trainable). For DeepGaitV2, the partially frozen configuration corresponds to freezing convolutional Layer 0.
Figure 2: Exemplar of gradient activation maps for best performing models. Rows 1 and 2 : SwinGait M2 and its corresponding GradCAM. Rows 3 and 4 : DeepGait D1 and its corresponding GradCAM.
Figure 3: Overview of the proposed frailty assessment framework. (a) In-clinic RGB videos are converted to privacy-preserving silhouette sequences using Detectron2. (b) Two transfer-learning backbones, DeepGaitV2 and SwinGait, extract gait representations for frailty prediction. In DeepGaitV2, the Basic Block 2D consists of two spatial convolution layers with batch normalization and ReLU activation, followed by an identity shortcut, while the Basic Block P3D replaces full 3D convolution with a pseudo-3D design composed of spatial and temporal convolutions, optional downsampling, synchronized batch normalization, and a 3D residual shortcut. (c) Clinician-facing interpretation of the predicted frailty status and potential downstream applications, including screening, longitudinal monitoring, and care planning.