Data-driven joint-moment predictors offer a scalable alternative to laboratory-based inverse-dynamics pipelines for biomechanics estimation and exoskeleton control. Meanwhile, physics-based reinforcement learning (RL) enables simulation-trained controllers to learn dynamics-aware assistance strategies without extensive human experimentation. However, quantitative verification of simulation-trained exoskeleton torque predictors, and their impact on human joint power injection, remains limited. This paper presents (1) an RL framework to learn exoskeleton assistance policies that reduce biological joint moments, and (2) a validation pipeline that verifies the trained control networks using an open-source gait dataset through inference and comparison with biological joint moments. Simulation-trained multilayer perceptron (MLP) controllers are developed for level-ground and ramp walking, mapping short-horizon histories of bilateral hip and knee kinematics to normalized assistance torques. Results show that predicted assistance preserves task-intensity trends across speeds and inclines. Agreement is particularly strong at the hip, with cross-correlation coefficients reaching 0.94 at 1.8 m/s and 0.98 during 5° decline walking, demonstrating near-matched temporal structure. Discrepancies increase at higher speeds and steeper inclines, especially at the knee, and are more pronounced in joint power comparisons. Delay tuning biases assistance toward greater positive power injection; modest timing shifts increase positive power and improve agreement in specific gait intervals. Together, these results establish a quantitative validation framework for simulation-trained exoskeleton controllers, demonstrate strong sim-to-data consistency at the torque level, and highlight both the promise and the remaining challenges for sim-to-real transfer.
Figures & tables
Fig. 1: The RL learning framework with Exoskeleton Control Network (ECN) and Muscle Coordination Network (MCN). Idealized torque assistance (blue curved arrows) is used instead of modeling any specific physical devices. The Human Control Network (HCN) predicted action is transferred to the desired torques (i.e. human biological joint moments), which is affected by exoskeleton assistance and corresponds to the full total joint torque minus the assistance. The desired biological torques are fed to MCN to predict muscle activations. The ECN predicts normalized assistance torques and is trained in supervised learning to reduce the biological joint moments via the loss in Eq. 2 . The trained ECN and MCN work together to control the human model for walking on level ground and ramps.
Fig. 2: Simulation results of walking gait on (a)level ground, (b)inclined (10°) and (c)declined (10°) grade surfaces with idealized hip and knee assistance. The purple lines on the foot are ground reaction forces.
Fig. 3: Inference hip and knee assistance torque and power (Pred) vs. ground truth (GT) biological moment and power under level-ground walking. The sign of hip and knee torques: hip flexion is negative and knee flexion is positive.
Fig. 4: Inference hip and knee assistance torque and power vs. GT biological moment and power under ramp walking. Positive degree correspond to incline walking, negative degree denotes decline walking, and 1.2 m/s represents the reference level-ground walking.
Fig. 5: Inference hip and knee assistance torque and power generated from the level-ground walking model vs. GT biological moment and power under ramp walking. This is a mismatched model-task condition to examine the generalization capability of the level-ground walking model.
TABLE I: Cross-correlation results and environment-wise averages for joint torque and power
Fig. 6: The means of inferred joint power of hip and knee, including MP, MPP and MNP. Top: walking under different speeds. Bottom: ramp walking at different incline/decline angles.
Fig. 7: Inference hip and knee assistance power under different delay time vs. GT power under level-ground walking, inclined walking, and decline walking. Darker shades of green indicate longer delay time.
Developing exoskeleton controllers that generalize across diverse locomotor conditions typically requires extensive motion-capture data and biomechanical labeling, limiting scalability beyond instrumented laboratory settings. Here, we present a physics-based neuromusculoskeletal learning framework that trains a hip-exoskeleton control policy entirely in simulation, without motion-capture demonstrations, and deploys it on hardware via policy distillation. A reinforcement learning teacher policy is trained using a muscle-synergy action prior over a wide range of walking speeds and slopes through a two-stage curriculum, enabling direct comparison between assisted and no-exoskeleton conditions. In simulation, exoskeleton assistance reduces mean muscle activation by up to 3.4% and mean positive joint power by up to 7.0% on level ground and ramp ascent, with benefits increasing systematically with walking speed. On hardware, the assistance profiles learned in simulation are preserved across matched speed-slope conditions (r: 0.82, RMSE: 0.03 Nm/kg), providing quantitative evidence of sim-to-real transfer without additional hardware tuning. These results demonstrate that physics-based neuromusculoskeletal simulation can serve as a practical and scalable foundation for exoskeleton controller development, substantially reducing experimental burden during the design phase.
Ilseung Park, Changseob Song, Inseung Kang
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Human locomotion emerges from high-dimensional neuromuscular control, making predictive musculoskeletal simulation challenging. We present a physiology-informed reinforcement-learning framework that constrains control using muscle synergies. We extracted a low-dimensional synergy basis from inverse musculoskeletal analyses of a small set of overground walking trials and used it as the action space for a muscle-driven three-dimensional model trained across variable speeds, slopes and uneven terrain. The resulting controller generated stable gait from 0.7-1.8 m/s and on ± 6∘ grades and reproduced condition-dependent modulation of joint angles, joint moments and ground reaction forces. Compared with an unconstrained controller, synergy-constrained control reduced non-physiological knee kinematics and kept knee moment profiles within the experimental envelope. Across conditions, simulated vertical ground reaction forces correlated strongly with human measurements, and muscle-activation timing largely fell within inter-subject variability. These results show that embedding neurophysiological structure into reinforcement learning can improve biomechanical fidelity and generalization in predictive human locomotion simulation with limited experimental data.
Ilseung Park, Eunsik Choi, Jangwhan Ahn +1
Carnegie Mellon University · Department of Physical Education, Seoul National University, Seoul, Republic of Korea · Seoul National University +2
Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.
Itak Choi, Ilseung Park, Eni Halilaj +1
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.