Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.
Figures & tables
Fig. 1: Overview of the musculoskeletal simulation and learning framework. (a) Reference kinematics and joint moments are extracted from a biomechanics dataset [ 21 ] containing synchronized motion capture and force plate data using OpenSim [ 22 ] . The policy observes future reference kinematics and outputs muscle activations to imitate able-bodied human locomotion. The reward function is designed to track reference kinematics and moments while simultaneously minimizing energy expenditure via a metabolic cost probe. (b) Fine-tuning phase for the human-exoskeleton system. The action space is extended to include assistive torque motors alongside muscle activations to simulate exoskeleton assistance. (c) Fine-tuning phase for the impaired gait policy. An activation mask reduces the maximum excitation of targeted muscles to generate compensatory impaired gait patterns. Both conditions can be applied simultaneously to generate impaired-gait-specific exoskeleton assistance. For both downstream fine-tuning tasks, the framework uses lower kinematic tracking weights and a higher metabolic energy penalty to accommodate the induced physical changes.
Pelvis
Thigh
Shank
Foot
Total
(kg)
(kg, each)
(kg, each)
(kg, each)
(kg)
Hip Exo
1.5
0.7
–
–
2.9
Ankle Exo
1.5
–
1.0
0.2
3.9
TABLE I: Mass Distribution of Simulated Exoskeleton Devices
Base Model
Fine-Tuning
Component
k
w
k
w
Position (pos)
-2.0
0.25
-0.4
0.25
Velocity (vel)
-0.05
0.1
-0.01
0.1
Root position (root)
-500.0
0.15
-500.0
0.15
End-effector (ee)
-80.0
0.25
-16.0
0.25
Torque (torq)
-2.0
0.25
-0.4
0.25
TABLE II: Gain Constants ( k ) and Reward Weights ( w )
Fig. 2: Kinematic and kinetic tracking performance of the baseline policy across five locomotor speeds (walking and running). Blue and gray lines represent the simulated agent and the ground-truth data, respectively. Joint angles and moments for the hip, knee, and ankle are compared against the ground truth; ground reaction force is not included as a reference signal during training. All values are segmented into gait cycles using heel contact events, interpolated to 100 data points, and averaged over 10 cycles. A fourth-order zero-lag Butterworth low-pass filter with a 4 Hz cutoff frequency is applied to both the ground-truth and simulated joint moment signals to reduce noise.
Parameter
Value
Network Size (Actor / Critic)
[512, 512, 256]
Batch Size
256
Learning Rate
Linear decay from 3×10−4
Soft Update Coefficient
0.02
Entropy Coefficient
auto
Discount Factor
0.95
TABLE III: SAC Hyperparameters
Fig. 3: Generated exoskeleton assistive torque profiles (Nm/kg, normalized by model mass) for the hip and ankle joints during walking and running. The assistance profiles from the fine-tuned policy (blue gradients, ranging from 0.6 to 2.2 m/s) are compared against experimentally optimized torque profiles from prior work [ 13 , 39 , 40 ] . Reference profiles for running with a hip exoskeleton consist of optimized extension-only splines, while reference ankle profiles represent optimized plantarflexion-only splines.
Fig. 4: Gross metabolic cost (W/kg) of the generated exoskeleton control policies for the able-bodied model. (a) Metabolic cost across locomotion speeds for the no-exoskeleton baseline (gray), hip assistance (blue), and ankle assistance (green) conditions. (b) Average metabolic cost during walking and running. All values are computed using the metabolic cost probe.
Fig. 5: Kinematic and kinetic deviations of simulated impaired gaits (orange) relative to the able-bodied baseline (blue). Top row: joint angles and moments for the affected side of the model with simulated ankle plantarflexor weakness, characterized by excessive dorsiflexion and a reduced plantarflexion push-off moment. Bottom row: results for the model with simulated hip flexor weakness, exhibiting a compensatory hip-hiking strategy and a corresponding reduction in the biological hip flexion moment.
Fig. 6: Exoskeleton assistive torque profiles for the affected and non-affected sides of the impaired-gait models at a walking speed of 1.2 m/s. In the model with ankle plantarflexor weakness (left), peak plantarflexion assistance on the affected side (orange) is 59% greater than on the non-affected side (dark blue). In the model with hip flexor weakness (right), peak hip flexion assistance on the affected side is 49% greater.
Fig. 7: Effect of impaired-gait-specific exoskeleton assistance on kinematics and energy expenditure at a walking speed of 1.2 m/s. (a) Joint kinematics of the affected (orange) and non-affected (blue) sides for models with ankle plantarflexor weakness (top) and hip flexor weakness (bottom). (b) Gross metabolic cost for simulated impaired gait with exoskeleton assistance (green) compared to the no-exoskeleton baseline (gray).
Developing exoskeleton controllers that generalize across diverse locomotor conditions typically requires extensive motion-capture data and biomechanical labeling, limiting scalability beyond instrumented laboratory settings. Here, we present a physics-based neuromusculoskeletal learning framework that trains a hip-exoskeleton control policy entirely in simulation, without motion-capture demonstrations, and deploys it on hardware via policy distillation. A reinforcement learning teacher policy is trained using a muscle-synergy action prior over a wide range of walking speeds and slopes through a two-stage curriculum, enabling direct comparison between assisted and no-exoskeleton conditions. In simulation, exoskeleton assistance reduces mean muscle activation by up to 3.4% and mean positive joint power by up to 7.0% on level ground and ramp ascent, with benefits increasing systematically with walking speed. On hardware, the assistance profiles learned in simulation are preserved across matched speed-slope conditions (r: 0.82, RMSE: 0.03 Nm/kg), providing quantitative evidence of sim-to-real transfer without additional hardware tuning. These results demonstrate that physics-based neuromusculoskeletal simulation can serve as a practical and scalable foundation for exoskeleton controller development, substantially reducing experimental burden during the design phase.
Ilseung Park, Changseob Song, Inseung Kang
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Providing personalized exoskeleton assistance across varying walking speeds remains challenging. Existing online optimization methods are sample-inefficient, requiring extensive human-in-the-loop (HIL) evaluations to optimize the entire assistive torque profile. Sim-to-real reinforcement learning (RL) offers a promising alternative but cannot directly account for individual user preferences. We propose a framework integrating sim-to-real RL with online preference learning for personalized exoskeleton assistance. Specifically, assistance timing is learned in simulation by training RL policies with human musculoskeletal models across varying walking speeds. The learned policies are then distilled and deployed on a physical hip exoskeleton using onboard sensory observations. Gaussian-process-based preference learning further personalizes the assistance magnitude through pairwise user comparisons. By decoupling assistance timing learning in simulation from magnitude optimization in real-world experiments, our framework substantially reduces the online optimization space. Human-subject experiments demonstrate efficient identification of personalized assistive torque profiles across varying walking speeds with fewer real-world evaluations.
Bin Li, Zhimin Hou, Jiacheng Hou +4
Division of Industrial Data Science (DIDS), School of Data Science, Lingnan University, Hong Kong · Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
Data-driven joint-moment predictors offer a scalable alternative to laboratory-based inverse-dynamics pipelines for biomechanics estimation and exoskeleton control. Meanwhile, physics-based reinforcement learning (RL) enables simulation-trained controllers to learn dynamics-aware assistance strategies without extensive human experimentation. However, quantitative verification of simulation-trained exoskeleton torque predictors, and their impact on human joint power injection, remains limited. This paper presents (1) an RL framework to learn exoskeleton assistance policies that reduce biological joint moments, and (2) a validation pipeline that verifies the trained control networks using an open-source gait dataset through inference and comparison with biological joint moments. Simulation-trained multilayer perceptron (MLP) controllers are developed for level-ground and ramp walking, mapping short-horizon histories of bilateral hip and knee kinematics to normalized assistance torques. Results show that predicted assistance preserves task-intensity trends across speeds and inclines. Agreement is particularly strong at the hip, with cross-correlation coefficients reaching 0.94 at 1.8 m/s and 0.98 during 5° decline walking, demonstrating near-matched temporal structure. Discrepancies increase at higher speeds and steeper inclines, especially at the knee, and are more pronounced in joint power comparisons. Delay tuning biases assistance toward greater positive power injection; modest timing shifts increase positive power and improve agreement in specific gait intervals. Together, these results establish a quantitative validation framework for simulation-trained exoskeleton controllers, demonstrate strong sim-to-data consistency at the torque level, and highlight both the promise and the remaining challenges for sim-to-real transfer.
Zihang You, Xianlian Zhou
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA