GaitVista: Reliability-Aware AI Measurement toward Accessible Longitudinal Gait Assessment
Authors: Nethmi Jayasinghe, Mihir Parashar, Amit Ranjan Trivedi
Organizations: Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL, USA. · Illinois Mathematics and Science Academy, Aurora, IL, USA.
Tracking recovery of walking function requires detecting meaningful gait change across rehabilitation sessions, yet objective 3D measurement remains confined to specialized motion-capture laboratories. Small camera sets and body-worn inertial sensors broaden access, but reliability varies across joints and time, allowing sensing failures to masquerade as patient change. We present GaitVista, a reliability-aware measurement layer whose lightweight gate assigns joint- and frame-specific visual contributions using camera coverage, local visual quality, cross-modal disagreement, and root-motion continuity, and exposes them for inspection. Across seven clean and degraded sensing conditions on TotalCapture, GaitVista reduces average full-body and lower-body error by 27.7% and 27.8%, attains the lowest worst-condition error among fusion methods, and reduces the gap to a joint-frame oracle from 2.76--5.33~cm for condition-blind baselines to 1.11~cm. On MoVi with image-derived keypoints, it is the only deployable fusion method to improve over both unimodal streams, reducing marker-supported error by 6.4% relative to the strongest learned fusion baseline. On TotalCapture, it improves bilateral knee-flexion waveform accuracy by 18.9%. Raw inertial measurements from five TotalCapture participants show location- and time-varying magnetic disturbance, supporting the design's reliability premise. Both benchmarks contain neurologically healthy participants in controlled settings and retain participant-specific IMU calibration; we therefore report progress toward accessible gait assessment, not validated clinical deployment.
Figures & tables
Figure 1: Overview of GaitVista. A small camera set and body-worn IMUs produce complementary visual and inertial pose estimates. Joint- and frame-specific reliability cues drive an inspectable fusion gate, yielding fused motion, assigned modality contributions, standardized avatar views, and gait measures for longitudinal review.
Figure 2: Location- and time-varying inertial disturbance in TotalCapture. (a) Magnetic dip-angle deviation across sensor locations for five walking2 participants. (b) Sensor with the largest magnetic-norm deviation over time; dashed lines separate participants, and marker size indicates the margin over the second-largest deviation.
MPJPE (cm) ↓
Gbest
Selector gap
Method
Mechanism
Avg.
Lower
Worst
(cm) ↑
(cm) ↓
Vision-only
Unimodal
6.88 ± 0.85
10.49 ± 2.22
19.21 ± 5.18
–
–
IMU-only
Unimodal
6.21 ± 1.50
9.06 ± 0.81
8.26 ± 1.41
–
–
Equal-weight fusion
Fixed blend
6.92 ± 1.58
9.72 ± 1.79
15.36 ± 6.47
−2.98
3.72
Kalman-Condition
Condition-tuned filter
4.98 ± 1.77
6.35 ± 0.62
11.02 ± 5.64
−1.04
1.78
Concat+MLP
Pose-input MLP
6.82 ± 1.36
8.75 ± 1.23
14.92 ± 5.29
−2.88
3.62
Table 1: Subject-disjoint TotalCapture results under 5-pixel keypoint perturbations. Values are mean ± population s.d. across five held-out participants after averaging five seeds and seven conditions. Worst is the maximum condition error per participant; lower is better except Gbest . Additional baselines are reported in the supplement.
Figure 3: TotalCapture under perturbed 2D observations. (a) Camera setup; (b) to (d) one-camera pose comparisons; (e) full-body MPJPE across camera counts; (f) lower-body MPJPE under sensing failures; (g) gain over the stronger modality; and (h) fixed-avatar rendering. Curves show mean ± population s.d. across five held-out participants after seed averaging.
Method
Avg. RMSE
Combined RMSE
ROM error
( ∘ ) ↓
( ∘ ) ↓
( ∘ ) ↓
Vision
12.04 ± 1.09
27.46 ± 2.18
27.84 ± 4.84
IMU
11.08 ± 1.03
25.17 ± 1.46
27.16 ± 4.44
Equal
10.58 ± 0.94
25.74 ± 2.45
25.54 ± 4.54
Concat+MLP
8.57 ± 0.81
26.36 ± 1.96
15.67 ± 2.63
GaitVista
6.95 ± 0.68
25.22 ± 1.53
14.97 ± 2.66
Table 2: Bilateral knee-flexion agreement on TotalCapture under five-subject LOSO. Combined denotes lower-body occlusion with IMU dropout. Values are mean ± population s.d. after seed averaging.
Method
MPJPE (cm) ↓
GbestM5
Marker-5
Ankle-2
Full-24
(cm) ↑
Vision
12.00 ± 0.81
12.61 ± 1.54
11.39 ± 1.09
–
IMU
49.70 ± 3.66
26.78 ± 1.36
32.99 ± 3.21
–
Equal
31.64 ± 1.58
23.50 ± 2.48
22.09 ± 1.62
−19.64
Concat+MLP
12.43 ± 1.05
12.61 ± 1.56
11.59 ± 1.27
−0.43
MoME
12.56 ± 0.92
13.55 ± 1.55
11.60 ± 1.23
−0.56
Table 3: Subject-disjoint MoVi results using MediaPipe keypoints from two synchronized views. Marker-5 and Ankle-2 are directly marker-supported; Full-24 partly relies on sparse-marker fitting. Values are mean ± population s.d. over five held-out participants after seed averaging.
Figure 4: MoVi evaluation with image-derived keypoints. (a) Synchronized views; (b) to (d) held-out pose comparisons; and (e) participant-wise Gbest over marker-supported joints. Positive values indicate improvement over both unimodal streams; error bars show variation across five seeds.
Variant
MPJPE (cm) ↓
Δ Avg.
Avg.
Lower
Combined
(cm) ↓
Static joint
7.05 ± 0.99
10.33 ± 2.01
9.53 ± 0.84
+2.74
No disagreement
4.69 ± 0.80
6.47 ± 1.15
9.18 ± 2.23
+0.38
No root continuity
5.03 ± 1.06
6.96 ± 2.00
9.43 ± 1.88
+0.72
No visual quality
6.60 ± 1.06
9.24 ± 1.90
9.39 ± 0.86
+2.29
GaitVista
4.31 ± 0.93
5.91 ± 0.44
9.11 ± 2.28
0.00
Table 4: TotalCapture ablations with 5-pixel keypoint perturbations. Values are mean ± population s.d. across five held-out participants after seed averaging. Avg. and Lower average seven conditions; Combined denotes lower-body occlusion with IMU dropout. Δ Avg. is relative to GaitVista, and Static joint uses only joint-specific preference.
Figure 5: Inspection prototype for one held-out sequence with (a) eight-camera and (b) one-camera sensing. Views show reconstructed pose, regional visual contributions, and bilateral knee-flexion trajectories. Contributions reflect modality use, not calibrated correctness.