Confidence Filtering in Video-Based Gait Assessment: Measurement Error versus Selection Shift
Organizations: Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat Gan, Israel.
Abstract
Video-based gait assessment often keeps only high-confidence pose observations, which can lower measurement error but change the movement assessed. We compared confidence selection with budget-matched random selection in a Parkinson-related cohort (n = 96) and an NF2-related replication cohort (n = 18). In the Parkinson-related cohort, confidence selection produced a small but consistent shift in the observed movement summary (+0.05 SD, 95% CI 0.03 to 0.07), while held-out Mini-BESTest error stayed flat (-0.04 points, -0.13 to 0.05); a policy preserving motion-group proportions reduced the shift, while NF2 showed no shift and its predictor did not beat a constant baseline. Because this shift mixes measurement error with selection shift, we decompose and simulate both on the cohort's window structure. Filtering lowers local error when error depends on confidence, but biases the endpoint once confidence tracks movement more than error. In the Parkinson-related data, confidence was only weakly tied to movement (rho = 0.14). There, filtering is harmless if well-tracked windows are unbiased. If instead systematic error only redistributes within participants, filtering can add up to 0.58 degrees. Unchanged clinical error from such small-cohort predictors therefore cannot show that quality filtering preserves the assessed movement.
Figures & tables
| Source | Analyzed inventory | Evaluation role |
|---|---|---|
| Parkinson-related [ 15 , 17 ] | 96 of 145 participants; 763 windows, ; 3 dual-task strata | Observed shift; primary clinical prediction |
| Simulation | Parkinson-related window structure; synthetic knee-excursion truth | , , , known |
| NF2 [ 15 , 18 ] | 18 of 19 participants; 175 windows, ; outbound/return strata | Clinical replication |
| Method | Parkinson-related MAE | NF2 MAE |
|---|---|---|
| Training mean | 3.02 | 5.98 |
| Training median | 3.04 | 4.97 |
| Full record (100%) | 2.83 | 5.32 |
| 50% nominal coverage | ||
| Random | 2.87 | 5.29 |
| Confidence | 2.84 | 5.29 |
Selection Bias
Selection bias studies how nonrandom sampling or observation distorts statistical inference, model training, and evaluation relative to a target population.
See all 44 papers