Learning Varying Physical Therapist-Patient Interactions for Robot-mediated Upper Limb Task-Specific Training
Authors: Jia Quan Loh, Vincent Crocher, Marlena Klaic, Denny Oetomo, Ying Tan
Organizations: Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne · Melbourne School of Health Sciences, The University of Melbourne
Upper extremity motor function recovery is positively linked to Task-Specific Training (TST) and sufficient therapy dosage. Rehabilitation robots can increase TST dosage via controlled, repetitive treatment and free therapists to simultaneously manage other patients, but it has yet to demonstrate significant benefits over conventional treatment. This is potentially linked to inaccurate robotic representation of personalised physical therapist-patient interaction and lack of practice variability during TST. Hence, we advocate for robotic interventions that preserve the personalised physical therapist-patient interactions when delivering TST for patients across varying practise conditions. We propose a Learning-from-Demonstration framework using Task-Parameterised Gaussian Mixture Models (TPGMM) to learn personalised physical therapist-patient interaction in Task-Specific exercises, mapping patient joint kinematics to therapist-applied torques using few demonstrations. The model is generalised to reconstruct therapist torques in new task variations. The framework was evaluated on physical interactions from 14 mock "therapist-patient" pairs over three tasks of increasing complexity, each with six variations. A benchmark comparison against a Look-Up Table was conducted. The results show both methods reproducing interactions in unseen task variations that deviate slightly from the actual interaction, with TPGMM slightly outperforming LUT. Both methods reproduced interactions that gets increasingly closer to the actual interaction as task complexity increases.
Figures & tables
Ref.
RHHI Configuration
Input
Learning Method
Desired Output
Variation
[ 14 ]
T-R-P
Measured end-effector position
GMM
End-effector force
✗
[ 17 ]
T-R-R-P
Measured end-effector force
GMM-based non-linear system
End-effector force derivative
✗
[ 18 ]
T-R-P
Measured end-effector position
GMM-based Impedance model
End-effector force
✗
[ 19 ]
T-R-R-P
Measured patient end-effector kinematics
GMM
Patient end-effector position
✗
[ 20 ]
T-R-P
Measured end-effector kinematics
Potential Field Function
End-effector force
✗
[ 13 ]
T-R-P
Measured joint positions
Look-Up Table
Joint torques
✗
TABLE I: Learning Therapist Interaction Profiles from Demonstration— Summary of literature.
Fig. 1: Task-specific exercises performed in the study. TP refers to task points where the joint kinematics are extracted from for the task frames. 1. , 2. and 3. refers to each patient persona.
Patient History
Task Description
Sub-tasks
P1
FMA-UE
8/66
Task
Reaching towards a cup and return
1-1
Move wrist from home (TP0) to cup (TP1)
MMSE
28/30
Variations
Cup Distance
Close
1-2
Move wrist from cup (TP1) to home (TP2)
Diagnosis
Dense right hemiparesis
Far
Left middle cerebral artery stroke (6 weeks)
Shoulder Rotation
External rotation
Presentation
Severe weakness in right upper extremity
No rotation
Cannot initiate active grasp and voluntary movement against gravity
Internal rotation
TABLE II: Patient personas (P # ) with associated task descriptions.
Fig. 2: 1. Construction of TPU braces - PETG cuffs sensorised to measure force-torque interactions and placement of sensors on the wearer’s body. Sensors holders house the IMU trackers on the right upper and forearm, with the force-torque sensors fixed on top. 2. Tight attachments on the wearer’s body using buckle-velcro straps and kinesiology tapes. Foam padding is layered underneath the brace to increase surface contact with the skin. 3. Common interactions “therapist” apply on the “patient” when constrained to only interact with the sensorised cuffs (highlighted in red), identified during the experiment.
RM-ANOVA
Persona
Method
Persona * Method
Measure
Case
SS
df1
df2
MS
F
P-val
SS
df1
df2
MS
F
P-val
SS
df1
df2
MS
F
P-val
ε
Val
0.348
2
26
0.174
15.59
0.0***
0.133
1
13
0.133
7.623
0.016*
0.003
2
26
0.002
0.162
0.766
Gen
0.612
2
26
0.306
7.965
0.002**
0.164
1
13
0.164
8.134
0.014*
0.013
2
26
0.006
0.609
0.551
Δτpeak
Val
1.032
2
26
0.516
15.069
0.000***
5.694
1
13
5.694
74.026
0.000***
0.667
2
26
0.333
11.752
0.000***
Gen
0.265
2
26
0.133
1.95
0.163
2.01
1
13
2.01
24.514
0.000***
0.592
2
26
0.296
8.785
0.005**
TABLE III: Top: Repeated-measures ANOVA for Effects of Persona, Method and Interaction. Bottom: Holm-corrected pairwise comparisons between independent within-subject factor levels. (Val: Seen Variations, Gen: Unseen Variations)
Fig. 3: A: Variability of therapist behaviour, τζ={18,2,1} in the trunk, shoulder and elbow. The dotted line represents the mid-range profile mζ={18,2,1} and the shaded region represents the behaviour boundaries. B: Reconstructed torques from TPGMM (TPGMM) and vector search (LUT) for τn={18,2,1,3} .
Fig. 4: Distribution of outcome measures for both candidate methods between the validation (white background) and generalisation (grey background) cases across patient personas. The therapist’s own variability of behaviour is shown as reference.
Post-stroke rehabilitation is often necessary for patients to regain proper walking gait. However, the typical therapy process can be exhausting and physically demanding for therapists, potentially reducing therapy intensity, duration, and consistency over time. We propose a Patient-Therapist Force Field (PTFF) to visualize therapist responses to patient kinematics and a Synthetic Therapist (ST) machine learning model to support the therapist in dyadic robot-mediated physical interaction therapy. The first encodes patient and therapist stride kinematics into a shared low-dimensional latent manifold using a Variational Autoencoder (VAE) and models their interaction through a Gaussian Mixture Model (GMM), which learns a probabilistic vector field mapping patient latent states to therapist responses. This representation visualizes patient-therapist interaction dynamics to inform therapy strategies and robot controller design. The latter is implemented as a Long Short-Term Memory (LSTM) network trained on patient-therapist interaction data to predict therapist-applied joint torques from patient kinematics. Trained and validated using leave-one-out cross-validation across eight post-stroke patients, the model was integrated into a ROS-based exoskeleton controller to generate real-time torque assistance based on predicted therapist responses. Offline results and preliminary testing indicate the potential of their use as an alternative approach to post-stroke exoskeleton therapy. The PTFF provides understanding of the therapist's actions while the ST frees the human therapist from the exoskeleton, allowing them to continuously monitor the patient's nuanced condition.
Grayson Snyder, Lorenzo Vianello, Levi Hargrove +2
Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA. · Legs and Walking Lab, Shirley Ryan AbilityLab, Chicago, IL, USA.
Robotic systems can enhance the amount and repeatability of physically guided motor training. Yet their real-world adoption is limited, partly due to non-intuitive trainer/therapist-trainee/patient interactions. To address this gap, we present a haptic teleoperation system for trainers to remotely guide and monitor the movements of a trainee wearing an arm exoskeleton. The trainer can physically interact with the exoskeleton through a commercial handheld haptic device via virtual contact points at the exoskeleton's elbow and wrist, allowing intuitive guidance. Thirty-two participants tested the system in a trainer-trainee paradigm, comparing our haptic demonstration system with conventional visual demonstration in guiding trainees in executing arm poses. Quantitative analyses showed that haptic demonstration significantly reduced movement completion time and improved smoothness, while speech analysis using large language models for automated transcription and categorization of verbal commands revealed fewer verbal instructions. The haptic demonstration did not result in higher reported mental and physical effort by trainers compared to the visual demonstration, while trainers reported greater competence and trainees lower physical demand. These findings support the feasibility of our proposed interface for effective remote human-robot physical interaction. Future work should assess its usability and efficacy for clinical populations in restoring clinicians' sense of agency during robot-assisted therapy.
Beatrice Luciani, Alex van den Berg, Matti Lang +2
Department of Cognitive Robotics, Delft University of Technology, Delft, The Netherlands · Department of Rehabilitation Medicine, Erasmus Medical Center, Rotterdam, The Netherlands.
In this paper, we propose a novel framework that allows therapists to teach robot-assisted rehabilitation exercises remotely via RGB-D video. Our system encodes demonstrations as 6-DoF body-centric trajectories using Cartesian Dynamic Movement Primitives (DMPs), ensuring accurate posture-independent spatial generalisation across diverse patient anatomies. Crucially, we execute these trajectories through a decoupled hybrid control architecture that constructs a spatially compliant virtual tunnel, paired with an effort-based temporal dilation mechanism. This architecture is applied to three distinct rehabilitation modalities: Passive, Active-Assisted, and Active-Resistive, by dynamically linking the exercise's execution phase to the patient's tangential force contribution. To guarantee safety, a Gaussian Mixture Regression (GMR) model is learned on-the-fly from the patient's own limb. This allows the detection of abnormal interaction forces and, if necessary, reverses the trajectory to prevent injury. Experimental validation demonstrates the system's precision, achieving an average trajectory reproduction error of 3.7cm and a range of motion (ROM) error of 5.5 degrees. Furthermore, dynamic interaction trials confirm that the controller successfully enforces effort-based progression while maintaining strict spatial path adherence against human disturbances.
Ali Alabbas, Camillo Murgia, Joanne Regan +1
Faculty of Mechanical Engineering, Atlantic Technological University, Galway, Ireland. · Department of Health and Nutritional Science, Atlantic Technological University, Sligo, Ireland.