Authors: Ilseung Park, Eunsik Choi, Jangwhan Ahn, Jooeun Ahn
Organizations: Carnegie Mellon University · Department of Physical Education, Seoul National University, Seoul, Republic of Korea · Seoul National University · UNC-Chapel Hill and NC State University · Lampe Joint Department of Biomedical Engineering, UNC-Chapel Hill and NC State University, Raleigh, NC 27695, USA
Human locomotion emerges from high-dimensional neuromuscular control, making predictive musculoskeletal simulation challenging. We present a physiology-informed reinforcement-learning framework that constrains control using muscle synergies. We extracted a low-dimensional synergy basis from inverse musculoskeletal analyses of a small set of overground walking trials and used it as the action space for a muscle-driven three-dimensional model trained across variable speeds, slopes and uneven terrain. The resulting controller generated stable gait from 0.7-1.8 m/s and on ± 6∘ grades and reproduced condition-dependent modulation of joint angles, joint moments and ground reaction forces. Compared with an unconstrained controller, synergy-constrained control reduced non-physiological knee kinematics and kept knee moment profiles within the experimental envelope. Across conditions, simulated vertical ground reaction forces correlated strongly with human measurements, and muscle-activation timing largely fell within inter-subject variability. These results show that embedding neurophysiological structure into reinforcement learning can improve biomechanical fidelity and generalization in predictive human locomotion simulation with limited experimental data.
Figures & tables
Figure 1: Comparison of reinforcement learning performance for independent and synergistic control strategies. Mean episode reward is shown as a function of total training timesteps (×10 7 ), averaged over five training runs with different random seeds. Solid lines indicate a 10-point moving average applied to the mean rewards, and shaded regions represent the standard deviation across runs. Both control strategies exhibit stable learning behavior throughout training. The independent controller (orange) shows an earlier rise in reward and converges to a higher final performance level than the synergistic controller (blue).
Figure 2: Similarity between predicted muscle activations and experimental EMG envelopes across eight lower-limb muscles. Top box plots show Pearson correlation coefficients between predicted muscle activations and experimental EMG envelopes during level treadmill walking at 1.2 m/s for soleus (SOL), gastrocnemius medialis (GAS), tibialis anterior (TA), semitendinosus (ST), biceps femoris (BF), rectus femoris (RF), vastus medialis (VM), and vastus lateralis (VL). Gray box plots represent the cross-human experimental benchmark derived from 22 participants of the open-source dataset, and the surrounding gray shaded regions indicate the corresponding envelope of cross-human variability. Orange and blue box plots correspond to independent and synergistic control conditions, respectively. Round dots denote individual correlation coefficients (n = 231 for cross-human pairwise comparisons; n = 22 per controller for correlations between simulated and experimental activations). Bottom line plots show representative activation profiles over the normalized gait cycle, where gray curves indicate experimental EMG envelopes across participants, and colored curves represent predicted activations from the independent (orange) and synergistic (blue) controllers. Muscle activations were normalized to reference voluntary contraction (RVC): for each muscle, predicted activations were scaled to the maximum amplitude observed within each controller, whereas experimental EMG envelopes were normalized to the subject-specific maximum amplitude.
Figure 3: Comparison of experimental and simulated kinematics, kinetics, and ground reaction forces across walking speeds. Sagittal-plane joint angles and joint moments are shown for the hip, knee, and ankle over the gait cycle, alongside anterior–posterior (AP) and vertical GRFs. Joint moments and GRFs were normalized by body mass. Experimental measurements are shown in grayscale, independent control in orange, and synergistic control in blue. For each condition, lighter colors indicate slower walking speeds (0.7 m/s) and darker colors indicate faster walking speeds (1.8 m/s) in increments of 0.1 m/s. Arrows highlight gait phases exhibiting clear speed-dependent magnitude modulation. Red arrows denote trends observed in the experimental data and similarly captured by the synergistic controller, whereas green arrows indicate regions where the independent controller deviates from the experimentally observed pattern.
Figure 4: Comparison of experimental and simulated kinematics, kinetics, and ground reaction forces across terrain slope conditions. Sagittal-plane joint angles and joint moments are shown for the hip, knee, and ankle over the gait cycle, alongside AP and vertical GRFs. Joint moments and GRFs were normalized by body mass. Experimental measurements are shown in grayscale, independent control in orange, and synergistic control in blue. For each condition, darker colors represent downhill walking (−5°), intermediate colors represent level walking (0°), and lighter colors represent uphill walking (+5°).
Figure 5: Phase-dependent root-mean-square error (RMSE) ratios of simulated kinematics, kinetics, and GRFs relative to experimental data. Heatmaps summarize the averaged RMSE ratios across gait phases (loading response, mid-stance, pre-swing, and swing) for independent control (orange) and synergistic control (blue) under both speed (left) and slope (right) conditions. RMSE values for joint angles, joint moments, and GRFs were normalized by the corresponding cross-human RMSE; ratios close to 1.0 indicate the errors comparable to cross-human variability.