GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment

Yize Cai; Zhiqing Hong

cs.CVNov 1, 2026

GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment

Authors: Yize Cai, Zhiqing Hong

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · The Hong Kong University of Science and Technology, Hong Kong, China.

Abstract

Gait assessment supports the diagnosis, monitoring, and prognosis of diverse movement-related conditions, such as neurological and musculoskeletal disorders. Recent advances in deep learning using everyday wearable inertial measurement units (IMUs) offer an accessible and low-cost approach to automated gait assessment. However, IMU measurements vary substantially across individuals, body locations, and device configurations, while existing deep learning models are typically developed using small, homogeneous datasets for specific clinical tasks, limiting their robustness and generalizability in diverse real-world contexts. Here we present GaitFM, a cross-modal foundation model for gait assessment using IMU sensors. GaitFM improves the generalizability of gait assessment by transferring knowledge from large vision models and pretraining on unlabeled IMU sensor data. We pretrain GaitFM on a large curated dataset comprising about 4,000 participants and 1.23 million gait windows across diverse clinical and daily conditions, sensor modalities, and body locations. GaitFM demonstrates strong performance across 18 gait-related tasks, including 3 gait parameter estimation tasks, 14 disease and gait abnormality detection tasks, and 1 clinical outcome prediction task. Furthermore, GaitFM generalizes across different sensor configurations and body locations, including those commonly used in clinical settings and everyday wearable devices. These findings demonstrate the effectiveness of transferring visual knowledge in improving sensor-based gait assessment performance, offering a new promising pathway for addressing data scarcity in the sensor and medical domains.

Figures & tables

Figure 1: Overview of the GaitFM framework. a, GaitFM adopts a masked autoencoder (MAE) architecture initialized with weights pretrained on ImageNet-1K, comprising approximately 1.28 million natural images across 1,000 classes. b, A sensor-to-image embedder enables continued MAE pretraining on a curated gait dataset comprising approximately 4,000 participants and 1.23 million IMU gait windows from seven heterogeneous datasets. c, Fine-tuning of GaitFM for downstream gait assessment tasks. GaitFM accommodates varying sensor placements and modality combinations, supporting flexible gait assessment across clinical and daily settings. The downstream tasks encompass gait parameter estimation, disease and abnormality diagnosis, and clinical outcome prediction. d, Distribution of samples across sensor placements.

Figure 2: GaitFM achieves strong performance across diverse gait assessment tasks and benefits from pretrained visual knowledge. a,b, Comparison of gait parameter estimation tasks on the internal Mobilise-D dataset and the external GaitPrint dataset. a, Coefficient of determination ( $R^{2}$ ) for identical model architectures with and without visual pretraining, demonstrating the benefits of pretrained visual knowledge on gait parameter estimation tasks. b, Mean absolute error (MAE) comparison across three gait parameter estimation tasks. c,d, Area under the receiver operating characteristic curve (AUROC) for disease and gait abnormality diagnosis tasks on six internal datasets (ClinicalGait, FoGContest, Mobilise-D, mPower, COMPWALK-ACL and WearGait-PD) and two external datasets (FoG-STAR and StrokeRehabilitation). c, Performance comparison across disease and gait abnormality diagnosis tasks. d, Comparison of identical model architectures with and without visual pretraining, demonstrating the benefits of pretrained visual knowledge on disease and gait abnormality diagnosis tasks. e, t-SNE visualization of learned representations for frailty diagnosis in the GSTRIDE dataset. GaitFM achieves a higher silhouette score than end-to-end models and its counterpart without visual pretraining, indicating improved separation between classes. f,g, AUROC for clinical outcome prediction task on the internal GSTRIDE dataset and the external LTMM dataset. f, Performance comparison on clinical outcome prediction task. g, Comparison of identical model architectures with and without visual pretraining, demonstrating the benefits of pretrained visual knowledge on clinical outcome prediction task. Error bars denote 95% confidence intervals (CIs) obtained from 1,000 bootstrap resamples. $P$ values are calculated using paired bootstrapping ( b ) and two-sided Wilcoxon signed-rank tests ( c and f ).

Figure 3: GaitFM accommodates dynamic sensor-location configurations without retraining. a, Performance on disease and abnormality diagnosis tasks across dynamic combinations of dedicated sensor locations in ClinicalGait, COMPWALK-ACL, FoG-STAR and StrokeRehabilitation. b, Contribution of vision-pretrained knowledge under dynamic combinations of dedicated sensor locations. c, PD diagnosis on WearGait-PD and gait-parameter estimation on GaitPrint across different combinations of simulated sensor placements representative of everyday wearable devices (left and right trouser pockets, left and right wrists, and head). d, Contribution of vision-pretrained knowledge to PD diagnosis and gait parameter estimation, under dynamic combinations available in everyday wearable devices.

Figure 4: Visual knowledge transfer enhances IMU-based gait representation learning and data efficiency. a, Performance comparison of GaitFM with foundation models pretrained on other modalities: general time-series data (Chronos 3 ), wearable physiological data (NormWear 33 ), and daily-life IMU data (BioPM 53 ). Each model was initialized with its original pretrained weights and then further pretrained on the same gait training set used for GaitFM. GaitFM achieves performance comparable to or better than that of these models, demonstrating the transferability of visual knowledge to sensor-data modeling. b, Representative examples of masked sensor image reconstruction with accelerometer and gyroscope data (acc + gyro, left), accelerometer data only (acc only, middle), and gyroscope data only (gyro only, right). c, Reconstruction loss of GaitFM with and without visual knowledge (Scratch). Visual knowledge reduces reconstruction loss across all three input settings, suggesting that it helps GaitFM capture local sensor patterns and cross-modality correlations. d,e, Effect of model scale on classification and regression tasks, evaluated using the Tiny, Base, Large, and Huge variants of the LVM. Validation loss is shown as a function of model size for FoG episode detection (FoGContest) and Parkinson’s disease diagnosis (mPower, d ), as well as gait-speed, step-length, and step-time estimation (Mobilise-D, e ). f, Data efficiency of GaitFM and GaitFM (Scratch) when fine-tuned with 5%, 10%, 20%, 50%, and 100% of the downstream training data. Across FoG episode detection (FoGContest), Parkinson’s disease diagnosis (mPower), fall-risk prediction (GSTRIDE), and step-length estimation (Mobilise-D), GaitFM with visual knowledge consistently achieves better performance, demonstrating improved data efficiency through visual knowledge transfer.

Gait Analysis

Gait analysis uses movement and biomechanical signals to characterize walking, recognize people, assess health, and guide rehabilitation or locomotion.

See all 81 papers

Inertia-1: An Open Exploration of Wearable Motion Foundation Models
Jul 7, 2026Zongzhe Xu, Aakarsh Anand, Sarah Jiang +4Human Activity Recognition Wearable Sensing
University of California, Los Angeles · Duke University · John Hopkins University
Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach
Sep 9, 2026Chandan Biswas, Aryan Singh, Anabik PalCross-Modal Knowledge Distillation Gait Analysis
NeuroAI Fusion Labs Kolkata, India · Indian Institute of Science Education and Research Berhampur, Odisha
Foundation models for movement data: Are they ready for prime-time?
Aug 13, 2026Alexander Bräuer, Benjamin Cauchi, Nils StrodthoffHuman Activity Recognition Healthcare
Carl von Ossietzky Universität Oldenburg, Oldenburg, Lower Saxony, 26129, Germany

Jul 7, 2026J/K move · Enter open · S save

Inertia-1: An Open Exploration of Wearable Motion Foundation Models

Zongzhe Xu, Aakarsh Anand, Sarah Jiang, Chuntung Zhuang+3

University of California, Los Angeles · Duke University · John Hopkins University

Human Activity Recognition Wearable Sensing Motion Representation Learning Time-Series Foundation Models

Open paper AskPDF arXiv

Wearable motion sensing provides a continuous and scalable window into human behavior and health, making it a natural fit for foundation models, yet its pretraining and scaling principles remain poorly understood. Prior work studies isolated design choices, such as sensor placement or sampling frequency, often under fixed settings and narrow downstream tasks that fail to capture real-world sensing diversity. We introduce Inertia-1, a fully open exploration of wearable motion foundation models. Using massive corpora of accelerometer data from global sources spanning more than 18.2M hours, we build a controlled framework for studying the full lifecycle of wearable motion foundation models, covering data choices such as sensor modality, device placement, sampling rate, window length; model choices such as architectures and model size; and training choices such as pretraining objective and data scale. Extensive evaluations across 15 datasets spanning human activity recognition, freezing-of-gait detection, and disease prediction reveal intriguing findings for building motion foundation models that generalize across tasks and sensing conditions. Collectively, Inertia-1 not only presents state-of-the-art recipes for diverse downstream tasks, but also serves as a comprehensive, practical, and open cookbook for wearable motion representation learning.

Abstract

Gait Analysis

Explore similar work

Inertia-1: An Open Exploration of Wearable Motion Foundation Models

Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach

Foundation models for movement data: Are they ready for prime-time?