OGA-AID: Clinician-in-the-loop AI Report Drafting Assistant for Multimodal Observational Gait Analysis in Post-Stroke Rehabilitation
Authors: Khoi T. N. Nguyen, Nghia D. Nguyen, Hui Yu Koh, Patrick W. H. Kwong, Karen Sui Geok Chua, Ananda Sidarta, Baosheng Yu
Organizations: Rehabilitation Research Institute of Singapore, Nanyang Technological University, Singapore · Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore · The Grainger College of Engineering, University of Illinois Urbana-Champaign, United States · VinUni-Illinois Smart Health Center, VinUniversity, Vietnam · Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong · Institute of Rehabilitation Excellence, Tan Tock Seng Hospital, NHG Health, Singapore
Gait analysis is essential in post-stroke rehabilitation but remains time-intensive and cognitively demanding, especially when clinicians must integrate gait videos and motion-capture data into structured reports. We present OGA-AID, a clinician-in-the-loop multi-agent large language model system for multimodal report drafting. The system coordinates 3 specialized agents to synthesize patient movement recordings, kinematic trajectories, and clinical profiles into structured assessments. Evaluated with expert physiotherapists on real patient data, OGA-AID consistently outperforms single-pass multimodal baselines with low error. In clinician-in-the-loop settings, brief expert preliminary notes further reduce error compared to reference assessments. Our findings demonstrate the feasibility of multimodal agentic systems for structured clinical gait assessment and highlight the complementary relationship between AI-assisted analysis and human clinical judgment in rehabilitation workflows.
Figures & tables
Figure 1: The overall architecture of OGA-AID. The system is designed to integrate multi-modal gait evidence with contextual patient information and clinician preliminary observations for post-stroke gait assessment through coordinated, agent-based analysis.
Figure 2: Processed multi-modal inputs for LLM inference in OGA-AID. (1) Anonymized gait recording that blurred the faces of people, (2) Joint trajectory plots after processing the MoCap trajectory input, (3) Patient clinical profile with demography and hemiparesis information, (4) Clinician’s initial observation.
Recording-based Factors
Factor
Description
Use of hand gait aid
Dependence on assistance during walking reflects impaired balance, weight transfer, or lower limb control.
Unaffected side step length
Shortened forward step distance reflects inadequate weight transfer, weakness, or instability of the affected limb.
Affected side weight shift
Insufficient body-weight shift toward paretic limb suggests impaired balance or weakness in the anti-gravity muscles of the lower limb.
Stance width
Mediolateral distance between feet during walking. Widened distance suggests balance deficits.
Guardedness
Hesitant gestures like stiffness, reduced arm swing, or slow movement due to fear of falling or difficult movement coordination.
Table 1: Assessment factors for post-stroke gait defined in the WGS. Each factor is evaluated using an ordinal scale reflecting increasing severity. OGA-AID utilizes these factors to generate structured gait analysis reports.
Figure 3: Statistics of 15 patients, including gender, stroke type, hemiparetic side, age, and duration since the stroke event (in months).
LLM
MAE ↓
Max AE ↓
Bias
Below MCID
Single pass
Qwen-3 VL
9.18
20.00
9.18
✗
GPT-5 Mini
3.15
7.70
1.61
✗
GPT-5.1
2.35
4.75
0.45
✗
Gemini-3 Flash
2.41
5.58
0.95
✗
Ours
Qwen-3 VL
6.28
11.00
6.28
✗
GPT-5 Mini
3.17
8.00
3.04
✗
Table 2: Performance comparison between single-pass LLMs and OGA-AID for WGS score reports for N=45 samples, without clinician-in-the-loop. ↓ indicates lower values are better. “Below MCID” denotes if the MAE falls under the 2.25 MCID threshold.
Figure 4: Per-factor MAE of WGS assessments by GPT-5.1 and Gemini-3 Flash across 45 samples. Each bar represents the MAE between predicted and reference ordinal scores for a given WGS factor. Error bars indicate standard deviation across runs.
Inputs
GPT-5.1
Gemini-3 Flash
MAE ↓
Max AE ↓
Bias
MAE ↓
Max AE ↓
Bias
R
3.80
9.50
-3.34
5.44
10.75
-5.32
R + D
2.33
6.75
-1.53
6.24
12.75
-6.15
R + T
2.85
8.50
-1.27
2.06
5.50
0.48
R + T + D
2.05
6.75
0.38
1.94
4.25
0.23
Table 3: Performance under different input combinations for OGA-AID. R denotes movement recordings, T denotes trajectory, D denotes patient clinical profile, orange and blue colors denote negative and positive Bias respectively.
Obs
GPT-5.1
Gemini-3 Flash
MAE ↓
Max AE ↓
Bias
MAE ↓
Max AE ↓
Bias
-
2.05
6.75
0.38
1.94
4.25
0.23
Short
2.07
6.00
1.01
1.90
3.25
-0.03
Med.
1.96
8.00
0.89
1.76
6.00
0.38
Long
1.67
4.00
1.00
1.58
4.00
-0.20
Table 4: Performance with clinician preliminary observations. “Obs” indicates the length of observations supplied to OGA-AID, with ”-” referring to the base case without any initial observation.
Current automated gait analysis for clinical applications relies on uninterpretable black-box classifiers. Although Vision-Language Models (VLMs) offer strong reasoning capabilities, applying them directly to gait videos often leads to hallucinations, because they struggle to measure subtle geometric deviations from raw visual contexts. To address this, we introduce DrGait, a training-free agentic framework that shifts the VLM's role from a direct visual reasoner to a clinical planner. DrGait decouples semantic reasoning from geometric perception through a structured Triage-Verification-Synthesis (TVS) workflow. Given an input video and a set of basic spatiotemporal metrics, the DrGait agent first performs a heuristic triage to propose diagnostic hypotheses, which are then verified by autonomously calling deterministic biomechanical tools that operate on reconstructed 3D mesh trajectories, segmented 2D pose tracks, and event-centered video evidence. Finally, a closed-loop mechanism recursively updates the agent's reasoning context based on the feedback. By anchoring VLM's reasoning in verifiable geometric and temporal measurements, DrGait reduces hallucinations, achieving competitive diagnostic accuracy while generating transparent and audit-ready clinical reports.
Tailoring stroke rehabilitation requires assessing how movements are organized, not merely if they succeed. Currently, this assessment is a rate-limiting bottleneck. Instruments like the Action Research Arm Test (ARAT) compress rich behavioral observations into single ordinal endpoints, discarding the movement-quality details that distinguish recovery from compensation. Automated alternatives typically chase accuracy on noisy, single-observer labels to output opaque scores - a technology-centric approach that rarely reaches clinical practice. To address this, we present xAARA: an engine designed to augment rather than replace clinical judgment. From multi-view video, xAARA returns ARAT assessments with calibrated uncertainty and explanations across task, movement-phase, and movement-quality levels. Treating clinical scoring as an ill-posed inference problem, xAARA composes 692 calibrated multimodal models via a Dynamic Bayesian Network with entropy-based gating. It qualifies results against clinical validity rules and defers low-confidence cases. In 105 stroke survivors (788 exercises), xAARA achieved 94.2% task accuracy (Cohen's kappa=0.934) and 81.3% movement-phase accuracy (kappa=0.727), reducing predictive uncertainty by 96.1% compared to single-clinician scoring. For subjective cases, it matched at least one rater 100% of the time and never returned out-of-range scores. Four independent clinicians validated the assessments and indicated willingness to adopt the system. We argue that principled uncertainty quantification and clinician-aligned explainability are the critical bridges moving automated assessment from technical demonstration to a deployable clinical tool.
Tamim Ahmed, Thanassis Rikakis
Bioemdical Engineering, University of Soutehrn California,, Los Angeles, 90007, CA, USA.
Pathological gait datasets remain scarce due to privacy, recruitment, cost, and movement variability. Our work presents a multimodal LLM-guided framework for pathology-aware 3D gait data synthesis from structured textual descriptions. The proposed method generates fixed-length synthetic skeleton-based gait sequences for pathological gait classification tasks. The framework combines motion tokenisation, pathology-aware language conditioning, LLM-based semantic augmentation, and language-to-gait generation. A key contribution is the proposed pathological tokeniser, which is designed to preserve pathology-specific motion characteristics during discrete representation learning. Experiments suggest that the proposed synthetic sequences improve downstream classification for recurrent classifiers when combined with real data. The best result is obtained using a GRU classifier trained with real and synthetic samples, achieving 92.77% accuracy under a leave-one-subject-out protocol.