See, Learn, Assist: Safe and Self-Paced Robotic Rehabilitation via Video-Based Learning from Demonstration
Authors: Ali Alabbas, Camillo Murgia, Joanne Regan, Philip Long
Organizations: Faculty of Mechanical Engineering, Atlantic Technological University, Galway, Ireland. · Department of Health and Nutritional Science, Atlantic Technological University, Sligo, Ireland.
In this paper, we propose a novel framework that allows therapists to teach robot-assisted rehabilitation exercises remotely via RGB-D video. Our system encodes demonstrations as 6-DoF body-centric trajectories using Cartesian Dynamic Movement Primitives (DMPs), ensuring accurate posture-independent spatial generalisation across diverse patient anatomies. Crucially, we execute these trajectories through a decoupled hybrid control architecture that constructs a spatially compliant virtual tunnel, paired with an effort-based temporal dilation mechanism. This architecture is applied to three distinct rehabilitation modalities: Passive, Active-Assisted, and Active-Resistive, by dynamically linking the exercise's execution phase to the patient's tangential force contribution. To guarantee safety, a Gaussian Mixture Regression (GMR) model is learned on-the-fly from the patient's own limb. This allows the detection of abnormal interaction forces and, if necessary, reverses the trajectory to prevent injury. Experimental validation demonstrates the system's precision, achieving an average trajectory reproduction error of 3.7cm and a range of motion (ROM) error of 5.5 degrees. Furthermore, dynamic interaction trials confirm that the controller successfully enforces effort-based progression while maintaining strict spatial path adherence against human disturbances.
Figures & tables
Fig. 1 : Hardware architecture of the system (left) and body-centric frames constructed from skeleton keypoints (right).
Fig. 2 : A GMR model learned from a subject’s limb during elbow flexion-extension exercise (left). The safety corridor is defined as a ±5 standard deviation around the mean expected force (in red). (Right) Exercise trajectories during shoulder abduction-adduction executed at varying trunk rotation angles. The trajectories are aligned at their starting points to demonstrate the high fidelity of the body-centric encoding.
Exercise
Reproduction Error (cm)
Elbow flexion–extension
1.8
Shoulder internal–external rotation
4.5
Shoulder abduction–adduction
4.9
TABLE I : Reproduction RMSE for the three exercises.
Limb Length (m)
Path Distance (m)
Normalized Ratio (R)
0.61
0.47
0.77
0.55
0.43
0.78
0.53
0.41
0.77
0.50
0.38
0.76
0.50 (Unscaled)
0.47
0.94
TABLE II : Normalized reach ratio ( R ) across different limb lengths: scaled vs. unscaled baseline.
Fig. 3 : Comparison of time-normalized joint angle trajectories for the three upper-limb exercises. Orange lines show the demonstration (subject 1), while green and purple lines show the robot reproductions for subjects 1 and 2.
Fig. 4 : Validation of the virtual tunnel and effort-based temporal dilation. (Left) The subject intentionally applies significant orthogonal forces (in red) while varying tangential effort (in blue). Exercise progress (green) is driven only by ft , slowing when ft<0 . (Right) 3D Cartesian paths confirm that the stiff orthogonal admittance maintains spatial adherence to the reference trajectory despite the significant force applied.
Fig. 5 : Exercise progress vs patient’s delivered force in the active-assisted and active-resistive modes.
Fig. 6 : Force monitoring and adaptive reversibility during the three rehabilitation exercises. The green line indicates exercise progress as a percentage of the demonstration. When the measured interaction force (blue line) exceeds the learned safety limit (dashed red) predicted by the GMR model, the system triggers smooth trajectory reversal to maintain patient comfort.
The decline of human balance control due to aging and pathological conditions increases fall risk, a major concern in geriatric care and rehabilitation. Gait training is essential for balance recovery, enhancing walking ability and postural control. However, existing overground robotic gait trainers have limitations: body weight support systems are bulky and impractical for daily use, while end-effector-based systems often compromise transparency, altering natural gait dynamics. This paper presents the Dynamic Robotic Balance Assistant (DRBA), a novel gait trainer providing assist-as-needed body weight and balance support for various training scenarios. DRBA integrates a 3-degree-of-freedom (3-DoF) robotic arm for pelvic support with flexible motion, a compact sit-to-stand assistance module, and user-following and fall detection algorithms to ensure minimal interference and responsive support. Experimental results demonstrated high transparency, with minimal impact on natural gait dynamics. A patient trial with nine elderly patients with varying medical conditions and balance impairments (ranging from severe to mild) further validated DRBA's effectiveness. The results showed that DRBA-assisted training increased step length and walking speed compared to therapist-assisted gait training. Additionally, DRBA enabled users to perform tasks beyond their unaided ability, expanding rehabilitation possibilities. These findings highlight DRBA's potential to enhance rehabilitation outcomes by facilitating higher training intensity and enabling task-oriented exercises.
Yifan Wang, Li Li, Youlong Wang +9
School of Mechanical and Aerospace Engineering, Nanyang Technological University, 639798 Singapore
Assistive robots offer agency to humans with severe motor impairments. Often, these users control high-DoF robots through low-dimensional interfaces, such as using a 1-D sip-and-puff interface to operate a 6-DoF robotic arm. This mismatch results in having access to only a subset of control dimensions at a given time, imposing unintended and artificial constraints on robot motion. As a result, interface-limited demonstrations embed suboptimal motions that reflect interface restrictions rather than user intent. To address this, we present a trajectory reconstruction algorithm that reasons about task, environment, and interface constraints to lift demonstrations into the robot's full control space. We evaluate our approach using real-world demonstrations of ADL-inspired tasks performed via a 2-D joystick and 1-D sip-and-puff control interface, teleoperating two distinct 7-DoF robotic arms. Analyses of the reconstructed demonstrations and derived control policies show that lifted trajectories are faster and more efficient than their interface-constrained counterparts while respecting user preferences.
Demiana R. Barsoum, Mahdieh Nejati Javaremi, Larisa Y. C. Loke +1
Northwestern University & Shirley Ryan AbilityLab, Chicago, Illinois, USA
Upper extremity motor function recovery is positively linked to Task-Specific Training (TST) and sufficient therapy dosage. Rehabilitation robots can increase TST dosage via controlled, repetitive treatment and free therapists to simultaneously manage other patients, but it has yet to demonstrate significant benefits over conventional treatment. This is potentially linked to inaccurate robotic representation of personalised physical therapist-patient interaction and lack of practice variability during TST. Hence, we advocate for robotic interventions that preserve the personalised physical therapist-patient interactions when delivering TST for patients across varying practise conditions. We propose a Learning-from-Demonstration framework using Task-Parameterised Gaussian Mixture Models (TPGMM) to learn personalised physical therapist-patient interaction in Task-Specific exercises, mapping patient joint kinematics to therapist-applied torques using few demonstrations. The model is generalised to reconstruct therapist torques in new task variations. The framework was evaluated on physical interactions from 14 mock "therapist-patient" pairs over three tasks of increasing complexity, each with six variations. A benchmark comparison against a Look-Up Table was conducted. The results show both methods reproducing interactions in unseen task variations that deviate slightly from the actual interaction, with TPGMM slightly outperforming LUT. Both methods reproduced interactions that gets increasingly closer to the actual interaction as task complexity increases.
Jia Quan Loh, Vincent Crocher, Marlena Klaic +2
Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne · Melbourne School of Health Sciences, The University of Melbourne