MSK-Bench: Benchmarking Full-Body Musculoskeletal Motor Control Across Tasks, Control Paradigms, and Physiological Metrics

Mengtao Ou; Zongzheng Zhang; Zhenghao Xiao; Yixuan Pan; Ziwen Zhuang; Hang Zhao; Hongyang Li; Yanan Sui

cs.ROSep 22, 2026

MSK-Bench: Benchmarking Full-Body Musculoskeletal Motor Control Across Tasks, Control Paradigms, and Physiological Metrics

Authors: Mengtao Ou, Zongzheng Zhang, Zhenghao Xiao, Yixuan Pan, Ziwen Zhuang, Hang Zhao, Hongyang Li, Yanan Sui, +2 more

Organizations: Institute for AI Industry Research (AIR), Tsinghua University · The University of Hong Kong (HKU) · Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University · School of Aerospace Engineering, Tsinghua University · Peking University (PKU)

Abstract

Musculoskeletal (MSK) humanoids provide a physiologically grounded embodiment for studying full-body motor control, but their high-dimensional muscle actuation, delayed activation dynamics, and redundant muscle--tendon structures make learning substantially harder than torque-driven humanoid control. Existing MSK benchmarks remain fragmented across gait, prosthetics, dexterous hands, or challenge-specific tracks, leaving full-body muscle-actuated control insufficiently evaluated under standardized tasks, methods, and metrics. We introduce MSK-Bench, a benchmark of 22 full-body motor-control tasks organized into three progressively challenging categories: postural stabilization, common locomotor behaviors, and contact-rich environmental interaction. Under unified task protocols and robustness perturbations, MSK-Bench evaluates 5 representative control paradigms, including reward-based RL, agentic reward tuning, latent-action RL, imitation-prior control, and residual adaptation over imitation priors. Beyond task success and reward, MSK-Bench further reports robustness analysis and physiology-oriented diagnostics, including activation cost, joint smoothness, and EMG-envelope similarity. Our empirical study shows that embodiment-aware exploration and structured action representations improve task coverage in high-dimensional muscle spaces, imitation priors enhance reference-compatible stabilization and locomotion but degrade under contact-rich terrain mismatch, and residual adaptation can recover successful behaviors when fixed references fail. We further find that improved task success does not necessarily imply improved physiological agreement, highlighting the importance of evaluating task performance, robustness, and physiological behavior jointly. MSK-Bench provides a task--method--metric testbed for full-body muscle-actuated humanoid control.

Figures & tables

Fig. 1: Overview of the Musculoskeletal Benchmark (MSK-Bench). (a) Benchmark Components: An open-source suite featuring 22 tasks across stabilization, locomotion, and interaction, evaluated under 5 control paradigms and 7 metric families. (b) Method Evaluation: Radar charts compare control methods across the 3 task families using 8 reported axes derived from the benchmark metrics. Each value represents the average performance across all applicable tasks within the respective category. EMG-envelope similarity is evaluated only where matched human reference data are available, while cumulative reward is reported separately in Fig. 2 .

Benchmark / Framework	Embodiment		Environment		Task Design		Evaluation		Main Focus
Benchmark / Framework	Actuation (Dim.)	Full Body	3D Scene / Terrain	Contact Rich	Task Hierarchy	Tasks / Data Scale	Physio. Eval. Metrics	Control Families	Main Focus
Torque-Driven Humanoid Benchmarks
HumanoidBench [ 88 ]	Torque (19/61)	✓	✓	✓	✓	27 tasks	✗	1	Loco-manipulation
SkillBench / SkillBlender [ 50 ]	Torque (19)	✓	$\bullet$	✓	✓	4 skills / 8 tasks	✗	1	Skill Blending
Mimicking-Bench [ 61 ]	Torque (19)	✓	✓	✓	✗	6 tasks	✗	1	Scene Interaction
Musculoskeletal Benchmarks

TABLE I: Benchmark comparison. ✓ /

\bullet

/ ✗ denote full/partial/no support in the cited evaluations. We compare actuation, full-body embodiment, scene/terrain support, contact-rich interaction, task hierarchy, benchmark scale, physiological evaluation, and evaluated control families.

Fig. 2: Task-specific reward-learning curves for 22 tasks. Four reward-based algorithms show task-dependent advantages. Shading shows across-run variation; Table II separately reports success, robustness, and nonzero-success coverage over all 22 tasks.

Group	Metric	DynSyn-SAC	DepRL	SAC	PPO
SR (%) $\uparrow$	Stabilization (6)	57.33	50.33	42.33	0.00
	Locomotion (6)	81.33	41.33	0.00	0.00
	Interaction (10)	31.20	51.40	2.80	7.40
	Average (22)	52.00	48.36	12.82	3.36
Robust. $\uparrow$	Action	46.66	53.06	15.36	3.10
	Observation	43.37	38.10	4.26	3.40

TABLE II: Full-suite point estimates. SR retains training noise. Means weight tasks equally. Robustness areas are normalized to 0–100; Avg. weights the three perturbation types equally. Coverage: SR

>0

. Bold: row best.

Fig. 3: Reward adaptation and robustness. (a) Coefficient updates; (b–d) survival under perturbations. Agentic-DepRL and fixed-reward DepRL share learner, architecture, and training budget. Each point averages 50 held-out seeds; survival is capped at 1000 steps.

Fig. 4: Action compression. Learning and perturbation tests for 416 actions and 128-, 64-, 32-, and 16-dimensional representations. The focused study improves at 128 dimensions but degrades under stronger compression; it is not a full-suite comparison.

Setting	Survival (steps) $\uparrow$	Tracking RMSE $\downarrow$	Activation $\downarrow$
Fixed reward	417	0.4446	0.4769
DeepSeek	246.6	0.4355	0.3999
GPT-6 Astra	360	0.4324	0.3879

TABLE III: Matched reasoner comparison .

Task	Duration (s) $\uparrow$	Div. (s) $\uparrow$	Tracking RMSE $\downarrow$	Outcome
Walk	0.75	0.53	0.158	Fall
Run	2.02	0.94	0.653	Fall
Stairs	0.80	N/A	0.173	Fall

TABLE IV: GPT-6 direct muscle-control diagnostic. Div.: first sustained tracking divergence; N/A indicates that the fixed divergence criterion was not met.

Fig. 5: Task-dependent performance under dynamics perturbations and stair-tracking errors. (a–e) Matched-protocol success rates of MuscleMimic/MM (PPO) and DepRL. Within each task, both methods use identical success conditions and perturbation settings. This focused cross-paradigm protocol is distinct from the full-suite benchmark protocol reported in the robustness tables; its values are not directly comparable with those tables and are excluded from aggregate robustness scores. MuscleMimic has no observed stair successes at any tested scale, including zero dynamics perturbation. (f) Reported tracking-error components on stairs characterize deviation from the reference, without identifying a timing mismatch or the cause of task failure.

Fig. 6: Walking EMG-envelope comparison. Human recordings [ 8 ] versus simulated activations. Normalized, phase-optimized correlations ( $r_{\mathrm{R}},r_{\mathrm{L}},r_{\mathrm{A}},r_{\mathrm{I}}$ ) correspond to reward-based RL, latent-action RL, agentic tuning, and imitation-prior control.

Fig. 7: Task-wise joint smoothness. Bars report $\log_{10}$ mean squared angular jerk. Stabilization shows the lowest measured jerk, locomotion the highest, and interaction intermediate values. These characterize motion fluctuations, not success or human similarity.

Fig. 8: Load-dependent recruitment in powerlift. Muscle-family activation is computed by summing the time-averaged activations of its constituent actuators. Bars report this activation mass under different barbell weights; the metric does not represent instantaneous activation or agreement with human recordings.

Fig. 9: Anatomical resolution and powerlift recruitment. Activation mass for the 416- and 700-muscle models. The latter adds neck, deep-trunk, and deep-scapular groups, expanding available recruitment pathways; this does not establish greater human similarity.

Fig. 10: Residual architecture and motions. Frozen prior plus trainable correction; blue skeletons are references. The original lower-branch label “ref action” denotes the residual correction in our equation. The 354d labels exclude hand actuation. Rewards and reference timing also change.

	SR (%) $\uparrow$		EMG $r$ $\uparrow$
Task	MM	Residual	MM	DepRL	Residual
Walking	100	100	0.71/0.69/0.63	–	0.75/0.80/0.67
Running	90	100	0.43	0.70	0.77
Stairs	0	100	0.51	0.56	0.52

TABLE V: Residual adaptation results. MM: MuscleMimic. SR retains training noise. EMG

r

reports SoleusMed./Bic.Fem./Semitend. for walking, an 11-channel phase-aligned mean for running, and a 12-muscle phase-aligned mean for stairs. Bold: best in each comparison.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 11: Illustration of six representative whole-body stabilization tasks. (a) squat : The control policy must accurately track the dynamic vertical trajectories of the hip and knee based on a phase clock $\phi$ , while maintaining a strict center-of-mass balance lock between the two feet. A specially designed “landmine penalty” is introduced to immediately eliminate abnormal strategies that attempt to exploit the height-tracking reward by kneeling on both knees. (b) stand : The robot is required to maintain an absolutely static upright posture from an initial neutral pose. The key challenge is that the system must use high muscle stiffness to lock the joints, while even slight center-of-mass drift or foot sliding is severely penalized. The reward function is strongly biased toward absolute stillness and posture maintenance. (c) powerlift : To sustain the extreme load, the reward function explicitly encourages a biomechanically plausible forward-leaning hip-hinge posture through the $R_{\mathrm{hinge}}$ reward, thereby effectively recruiting the powerful posterior-chain muscle groups. Once dangerous lumbar collapse is detected, the episode is immediately terminated to ensure structural safety. (d) Singlelegstand : The robot must lift one leg into a “flamingo” posture and strictly align the center-of-mass projection with the support foot. The task severely penalizes lateral trunk lean, and after an initial physical tolerance period of $10$ steps, the episode is immediately terminated once the lifted foot touches the ground. (e) sit : The robot is required to approach a chair and execute a precise sitting transition. To prevent cheating behaviors such as stiff-legged falling, the task enforces active knee flexion, requires the feet to be retracted beneath the support area, and introduces a “soft-landing” reward to limit pelvic descent velocity, ensuring a smooth motion. (f) balance : The robot is required to maintain dynamic balance on an unconstrained wobble board placed on a sphere. To overcome the common failure mode of falling backward, the task explicitly enforces a forward-leaning “fighting defensive stance”, with a target pitch angle of $-0.15\,\mathrm{rad}$ , and requires precise center-of-mass control to keep the balance board level.

Fig. 12: Visualization of representative locomotion behaviors generated by the humanoid robot. Each subfigure shows a sequence of motion frames, with red and blue skeletons indicating the two legs and body segments over time. (a) walk turn : The robot moves while interacting with spherical obstacles. (b) walk forward : The robot demonstrates a stable periodic gait with alternating leg support. (c) sidestep : The robot demonstrates sustained rhythmic whole-body coordination. (d) run : The robot runs forward with alternating leg motion while maintaining dynamic balance. (e) crawl : The robot performs low-posture locomotion with the body close to the floor. (f) Jump : The robot performs a dynamic aerial transition while maintaining body balance.

Fig. 13: Illustration of ten representative terrain and object interaction tasks. (a) walkandsit : The robot is required to approach a chair backward and sit down. Instead of using a conventional collision-avoidance penalty, the task rewards the robot for confidently utilizing downward inertia to complete the sitting motion and for comfortably leaning back against the chair after sitting, thereby reproducing human-like sitting biomechanics. (b) stepstones : A magnetic attraction reward is used to guide the foot toward the target landing positions, while strict penalties on trunk lateral bending, yaw deviation, and excessive knee flexion encourage an upright and stable traversal posture. (c) slide : The robot is required to adaptively lean forward to counteract inertia during slope transitions. A split-prevention penalty is introduced to terminate the episode once the lateral separation of the legs exceeds the biomechanical limit. (d) stairs : The robot is required to climb stairs using look-ahead terrain perception from a terrain radar. When upcoming steps are detected, an active leg-lifting reward is activated to encourage anticipatory step planning. (e) reach : The robot is required to rapidly and explosively reach randomly reset spatial targets with its dominant hand while maintaining a slightly crouched defensive posture. Since the target height varies, the task strictly constrains knee height to prevent cheating strategies such as kneeling. (f) hurdle : The task enforces a complete aerial flight phase and introduces a strict anti-flamingo-jump penalty to prevent abnormal strategies in which the robot crosses the hurdle by curling and dragging a single leg. (g) open door : The robot is required to approach, push open, and pass through a hinged door. A global attraction field guides the hand toward the door handle, while fall-prevention penalties discourage shortcut behaviors such as headbutting the door or diving through it. (h) chinup : The robot is required to pull its head above the horizontal bar from a hanging posture. The task rewards the activation of major pulling muscle groups such as the latissimus dorsi and biceps, while imposing strict penalties on body swinging and leg-kipping, forcing the motion to be completed through pure muscular strength. (i) catch : The robot is required to coordinate both arms to catch a flying cube thrown toward the chest while maintaining an upright posture. The key objective is to track the distance between both hands and the target, and a large sparse reward is given when the object is successfully caught and stabilized in mid-air. (j) polewalk : The robot uses terrain radar to navigate through a field of poles while avoiding obstacles. In accordance with the task specification and collision penalty, head or torso contact with a pole terminates the episode.

Task	Term	Formula	Weight
1. Stand	Pose preservation	$\exp(-\\|q_{\text{joints}}-q_{0,\text{joints}}\\|)$	$10$
	Absolute stillness	$\exp(-10.0\\|\mathbf{v}_{\text{CoM}}\\|)\cdot\exp(-0.1\\|\dot{q}\\|)$	$15$
	Feet planted	$\exp(-10.0(\\|\mathbf{v}_{\text{foot,L}}\\|+\\|\mathbf{v}_{\text{foot,R}}\\|))$	$10$
	Torso balance	$\exp(-5.0\max(0,1-\mathbf{u}_{\text{torso}}\cdot\mathbf{\hat{z}}))$	$10$
	Survival	$\mathbb{I}_{\text{alive}}$	$5$
2. Powerlift	Hold stability	$\mathbb{I}(z_{\text{bar}}>1.6\text{m}\land\|\dot{z}_{\text{bar}}\|<0.1\text{m/s})$	$100$

Appendix

TABLE VI: Reward Functions for All 22 Musculoskeletal Benchmark Tasks (Part 1: Stabilization Series). Note: Action regularization (

R_{\text{efficiency}}

) and minor limit penalties (

\sum\text{penalties}

) are applied broadly across tasks and abbreviated where standard.

Task	Term	Formula	Weight
1. Walk	Velocity tracking	$\exp(-2.0\\|\mathbf{v}_{\text{CoM}}-\mathbf{v}_{\text{target}}\\|^{2})$	$10$
	Kinematic imitation	$\exp(-2.0\\|q_{\text{joints}}-q_{\text{ref,joints}}\\|^{2})$	$3$
	Upright / No hunch	$\exp(-10.0(1-\mathbf{u}_{\text{torso}}\cdot\mathbf{\hat{z}}))$ , $\exp(-20.0(\mathbf{u}_{\text{chest}}\cdot\mathbf{\hat{z}}+0.1)^{2})$	$6/8$
	Forward face	$\exp(-5.0(1-\mathbf{u}_{\text{chest}}\cdot\mathbf{\hat{x}}))$	$8$
2. Crawl	Backwards velocity	$\text{clip}(-v_{x,\text{CoM}}/v_{\text{target}},-0.2,1.0)$	$500$
	Prone posture	$\exp(-5(\mathbf{u}_{\text{chest}}\cdot\mathbf{\hat{z}}+1)^{2})\cdot\dots$	$100$

Appendix

TABLE VII: Reward Functions for Musculoskeletal Benchmark Tasks (Part 2: Locomotion Series).

Task	Term	Formula	Weight
1. Stairs	Fwd trajectory	$\text{clip}(v_{x,\text{CoM}},-0.2,2.0)$	$50$
	Upward trajectory	$\text{clip}(v_{z,\text{CoM}},-0.5,3.0)$	$30$
	Vertical stable	$\exp(-5.0(0.98-\mathbf{u}_{\text{torso}}\cdot\mathbf{\hat{z}})^{2})$	$20$
	Lateral stable	$\exp(-5.0y_{\text{pelvis}}^{2})$	$20$
	Anticipatory lift	$\mathbb{I}(\text{radar})\cdot\max(\dot{z}_{\text{swing}},0)$	$40$
2. Hurdle	Forward velocity	$\text{clip}(v_{x,\text{CoM}},-0.1,4.0)$	$300$

Appendix

TABLE VIII: Reward Functions for Musculoskeletal Benchmark Tasks (Part 3: Interaction with Environment Series).

Task	Specific Obs ( $O_{\text{task}}$ )	Initialization (Brief)	Termination Criteria
1. Stand	Torso quat, feet height	Default neutral, zero vel.	$z_{\text{CoM}}<0.8$ or tilt $>0.8$
2. Powerlift	Dumbbell $\mathbf{p}_{\text{bar}}$ , $\Delta\mathbf{p}_{\text{bar}}$ , tilt	High CoM, pre-loaded lumbar	$z_{\text{pelvis}}<0.7$ or $q_{\text{lumbar}}>0.5$
3. Singleleg	Torso quat, feet/CoM $z$	Flamingo pose, noise	$z_{\text{pelvis}}<0.6$ , dropped foot
4. Sit	Chair relative $\Delta\mathbf{p}_{\text{chair}}$	In front of chair, hips pre-flexed	$z_{\text{pelvis}}<0.25$ , invalid contact
5. Balance	Board $\mathbf{p},\mathbf{q}$ , CoM/Ball $\Delta\mathbf{p}$	Drop $z=1.28$ m, guard stance	$z_{\text{pelvis}}<0.70$ , pitch fall
6. Squat	Phase $\phi$ , target $z_{\text{target}}$	Squat start pose, barbell hold	$z_{\text{pelvis}}<0.35$ , kneeling

Appendix

TABLE IX: Task Settings, Specific Observations, and Termination Criteria (Part 1: Stabilization Series). Base Observation (

O_{\text{base}}

): All tasks fundamentally include proprioceptive states

(q,\dot{q},\mathbf{v}_{\text{CoM}})

and muscle states (length, velocity, force, activation). Only task-specific exteroceptive or goal variables are listed below.

Task	Specific Obs ( $O_{\text{task}}$ )	Initialization (Brief)	Termination Criteria
1. Walk	Phase $\phi$ , error $\Delta q=q-q_{\text{ref}}$	Sampled from synthetic walk	$z_{\text{CoM}}<0.5$ or tilt $>0.8$
2. Crawl	Torso quat, feet height	Prone quadruped, ground $z=0.25$	Fwd move, twist, flip
3. Run	Torso angle, feet height	Standard standing	$z_{\text{pelvis}}<0.45$ , fall backward
4. Jump	Torso angle, $d_{xy,\text{feet}}$	Micro-squat, pre-flexed	$z_{\text{pelvis}}<0.45$ , wide split
5. Walk turn	Target $\Delta\mathbf{p}$ , ID, facing $a$	Stand, 20-step novice protection	$z_{\text{pelvis}}<0.55$ , $z_{\text{head}}<0.8$
6. Sidestep	Phase $\phi$ , lat dist $\Delta y_{\text{feet}}$	Crab guard stance, hips wide	$z_{\text{pelvis}}<0.45$ , split $>0.5$ m

Appendix

TABLE X: Task Settings, Specific Observations, and Termination Criteria (Part 2: Locomotion Series).

Task	Specific Obs ( $O_{\text{task}}$ )	Initialization (Brief)	Termination Criteria
1. Stairs	Phase $\phi$ , radar $\mathbf{z}_{\text{terrain}}\in\mathbb{R}^{9}$	Walk posture, initial fwd vel	$z_{\text{pelvis}}<0.75$ , lat drift
2. Hurdle	Phase $\phi$ , nearest $\Delta\mathbf{h}$	Standing, scan dynamic hurdles	$z_{\text{pelvis}}<0.55$ , heavy yaw
3. Step stones	Phase $\phi$ , stones $\Delta\mathbf{p}_{\text{stones}}$	Elevated platform $z=3.05$ m	Fall from height, $z_{\text{pelvis}}<2.5$
4. Slide	Phase $\phi$ , normal $\mathbf{n}_{\text{slope}}$	Elevated over wave terrain	Backwards fall, split $>1.2$ m
5. Open door	Handle $\Delta\mathbf{p}$ , door $\theta$ , target	Stand front of closed door	$z_{\text{pelvis}}<0.65$ , head $<1.1$
6. Reach	Phase $\phi$ , reach $\Delta\mathbf{p}_{\text{reach}}$	Quarter-squat, hands raised	Knee drop $<0.15$ m

Appendix

TABLE XI: Task Settings, Specific Observations, and Termination Criteria (Part 3: Interaction with Environment Series).

Task	Success criterion	Task	Success criterion
Stand	Reach 1,000 evaluation steps without earlier termination.	Sidestep	Maximum absolute lateral pelvis displacement from the start is at least $9.0$ m.
Powerlift	Reach 900 steps and attain a maximum dumbbell height of at least $1.6$ m.	Stairs	Maximum pelvis height is at least $1.6$ m.
Single-leg stand	Reach 1,000 evaluation steps without earlier termination.	Hurdle	The recorded number of cleared hurdles reaches the environment’s total (five if unavailable).
Sit	The environment’s has_sat_down flag becomes true.	Step stones	current_stone_idx reaches the final stone index.
Balance	Run for more than 350 steps and attain a board–ball horizontal separation below $0.5$ m.	Slide	Maximum forward pelvis displacement from the start is at least $18.0$ m.
Squat	Pelvis height first falls below $0.75$ m and subsequently rises above $0.90$ m.	Door open	Forward displacement is at least $2.0$ m, the door-open flag is true, or maximum door angle exceeds $1.0$ rad.

Appendix

TABLE XII: Task-specific success criteria used by the shared full-suite evaluator for PPO, SAC, DepRL, and DynSyn-SAC. These criteria are distinct from the environment termination rules. A positive native solved signal also counts as success for every task.

Task	Algorithm	Activation	Robustness (%)															Max	Train-env.	$\log_{10}$ smooth
		Cost	Action					Observation					Dynamic					Steps ( $\times 10^{7}$ )	SR (%)	(rad 2 /s 6 )
			0	0.05	0.1	0.15	0.2	0	0.02	0.05	0.08	0.10	0	0.05	0.1	0.15	0.2
stand	DepRL	0.4488	94	94	90	96	98	100	12	0	0	0	98	48	28	12	4	3.74	86	3.152
	SAC	0.0484	100	100	100	100	96	100	0	0	0	0	100	100	100	100	100	7.13	100	1.875
	PPO	0.3691	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1.97	0	5.479
	DynSyn-SAC	0.2859	52	42	56	46	36	46	46	22	30	14	40	32	40	30	30	5.44	52	2.104

Appendix

TABLE XIII: Benchmark Performance and Robustness Evaluation: Stabilization. Train-env. SR retains native training noise; robustness columns report separate perturbation sweeps (App. A-B ).

Task	Algorithm	Activation	Robustness(%)															Max	Train-env.	$\log_{10}$ smooth
		Cost	Action					Observation					Dynamic					Steps ( $\times 10^{7}$ )	SR (%)	(rad 2 /s 6 )
			0	0.05	0.1	0.15	0.2	0	0.02	0.05	0.08	0.10	0	0.05	0.1	0.15	0.2
walk forward	DepRL	0.4206	28	28	32	32	44	34	32	46	24	30	40	20	12	8	6	3.32	64	3.151
	SAC	0.1427	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9.88	0	3.262
	PPO	0.3940	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	2.42	0	5.558
	DynSyn-SAC	0.2771	82	88	68	60	48	82	82	80	76	76	82	76	76	64	8	5.00	76	3.417

Appendix

TABLE XIV: Benchmark Performance and Robustness Evaluation: Locomotion. Train-env. SR retains native training noise; robustness columns report separate perturbation sweeps (App. A-B ).

Task	Algorithm	Activation	Robustness (%)															Max	Train-env.	$\log_{10}$ smooth
		Cost	Action					Observation					Dynamic					Steps ( $\times 10^{7}$ )	SR (%)	(rad 2 /s 6 )
			0	0.05	0.1	0.15	0.2	0	0.02	0.05	0.08	0.10	0	0.05	0.1	0.15	0.2
stair	DepRL	0.3499	52	54	38	42	40	34	38	2	0	0	32	40	24	14	4	1.42	36	2.982
	SAC	0.1993	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	6.98	0	2.459
	PPO	0.3854	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1.57	0	5.303
	DynSyn-SAC	0.2759	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	5.92	0	2.854

Appendix

TABLE XV: Benchmark Performance and Robustness Evaluation: Interaction with Environment. Train-env. SR retains native training noise; robustness columns report separate perturbation sweeps (App. A-B ).

Method	Actor / critic MLP	Learning rate	Batch	Collection, rollout, and update/data cadence	Budget	Checkpoint used for reporting
PPO	$2\times 1024$ / $2\times 1024$	$3\times 10^{-4}$	256	2,048 steps per environment and eight environments ( $16{,}384$ transitions per rollout); 10 epochs, i.e., 10 passes over each rollout.	$2\times 10^{8}$ steps	Highest mean evaluation return; 10 det. episodes every $5\times 10^{5}$ training steps.
SAC	$2\times 1024$ / twin $2\times 1024$ Q networks	$1\times 10^{-4}$	256	Continuous off-policy collection; one vectorized step (eight new transitions) followed by one gradient step. The configured gradient_steps / train_freq ratio is 1.	$2\times 10^{8}$ steps	Highest mean evaluation return; 10 det. episodes every $2\times 10^{5}$ training steps.
DepRL	$2\times 1024$ / $2\times 1024$	Actor $5\times 10^{-5}$ ; critic $8\times 10^{-5}$ ; dual $2\times 10^{-3}$	256	Continuous replay with 3-step returns; after $10^{5}$ warm-up transitions, 30 minibatch updates per 1,000 new transitions (0.03 update iterations per transition).	$1\times 10^{8}$ steps	Last step-numbered checkpoint (the completed run ends at $10^{8}$ ); saved every $2\times 10^{6}$ steps. A best-return checkpoint is also written, but is not the evaluator default.
DynSyn-SAC	$3\times 1024$ / twin $3\times 1024$ Q networks, plus DynSyn layer	Linear $5\times 10^{-4}$ to $1.25\times 10^{-4}$	256 *	Continuous off-policy collection; one vectorized step (64 new transitions) followed by one gradient step. The configured gradient_steps / train_freq ratio is 1.	$5\times 10^{7}$ steps	Highest mean evaluation return; 3 det. episodes every $5\times 10^{6}$ training steps.

Appendix

TABLE XVI: Released training and checkpoint protocol for the four reward-based baselines. MLP widths list hidden layers for the actor and value/critic networks. “Det.” denotes deterministic evaluation episodes.

Fig. 14: Upper limb muscle activation heatmap for the 416-muscle model (Powerlift). The heatmap displays the intense, sustained activations of the available arm and shoulder musculature required to stabilize the barbell overhead. Note that this model lacks the detailed deep axial stabilizers of the neck and spine.

Fig. 15: Expanded muscle activation heatmap for the 700-muscle model (Powerlift). The top section includes the upper limb muscles, where the distinct ”blank gaps” represent the fine-grained scapular and arm muscles exclusive to the 700-model. The bottom section explicitly details the heavy recruitment of the newly available neck and spine axial stabilizers (e.g., multifidus , splenius ).

Observation block	Dimension
Root pose without global $x$ – $y$	5
Joint positions	82
Root velocity	6
Joint velocities	82
Muscle state ( $354\times$ length, velocity, force, excitation, activation)	1,770
Foot/toe touch sensors	4

Appendix

TABLE XVII: Controller-visible observation supplied at each GPT-6 decision. Dimensions are identical for Walk, Run, and Stairs.

Body Region	Muscle Category	Actuator Name Keywords
Core and Back	Paraspinals (Deep)	Ps_
	Iliocostalis	IL_
	Longissimus (Erector Spinae)	LTpT , LTpL
	Abdominals	rect_abd , EO , IO
	Other Trunk/Back	LAT , LD , QL , MF_m , TRAP , RHOM
	Pectorals	PEC

Appendix

TABLE XVIII: Anatomical muscle-family categories used for whole-body recruitment analysis. Each actuator is assigned to a category according to whether its name contains one of the corresponding keywords.

Explore similar work

Mar 31, 2026

Scaling Whole-Body Human Musculoskeletal Behavior Emulation for Specificity and Diversity

The embodied learning of human motor control requires whole-body neuro-actuated musculoskeletal dynamics, while the internal muscle-driven processes underlying movement remain inaccessible to direct measurement. Computational modeling offers an alternative, but inverse dynamics methods struggled to resolve redundant control from observed kinematics in the high-dimensional, over-actuated system. Forward imitation approaches based on deep reinforcement learning exhibited inadequate tracking performance due to the curse of dimensionality in both control and reward design. Here we introduce a large-scale parallel musculoskeletal computation framework for biomechanically grounded whole-body motion reproduction. By integrating large-scale parallel GPU simulation with adversarial reward aggregation and value-guided flow exploration, the MS-Emulator framework overcomes key optimization bottlenecks in high-dimensional reinforcement learning for musculoskeletal control, which accurately reproduces a broad repertoire of motions in a whole-body human musculoskeletal system actuated by approximately 700 muscles. It achieved high joint angle accuracy and body position alignment for highly dynamic tasks such as dance, cartwheel, and backflip. The framework was also used to explore the musculoskeletal control solution space, identifying distinct musculoskeletal control policies that converge to nearly identical external kinematic and mechanical measurements. This work establishes a tractable computational route to analyzing the specificity and diversity underlying human embodied control of movement. Project page: https://lnsgroup.cc/research/MS-Emulator.

Yunyue Wei, Chenhui Zuo, Shanning Zhuang +3

Tsinghua University.

Whole-Body Control Human Motion Generation RL Control Imitation Learning

Ask PDF

Mar 26, 2026

Towards Embodied AI with MuscleMimic: Unlocking full-body musculoskeletal motor learning at scale

Learning motor control for muscle-driven musculoskeletal models is hindered by the computational cost of biomechanically accurate simulation and the scarcity of validated, open full-body models. Here we present MuscleMimic, an open-source framework for scalable motion imitation learning with physiologically realistic, muscle-actuated humanoids. MuscleMimic provides two validated musculoskeletal embodiments - a fixed-root upper-body model (126 muscles) for bimanual manipulation and a full-body model (416 muscles) for locomotion - together with a retargeting pipeline that maps SMPL-format motion capture data onto musculoskeletal structures while preserving kinematic and dynamic consistency. Leveraging massively parallel GPU simulation, the framework achieves order-of-magnitude training speedups over prior CPU-based approaches while maintaining comprehensive collision handling, enabling a single generalist policy to be trained on hundreds of diverse motions within days. The resulting policy faithfully reproduces a broad repertoire of human movements under full muscular control and can be fine-tuned to novel motions within hours. Biomechanical validation against experimental walking and running data demonstrates strong agreement in joint kinematics (mean correlation r = 0.90), while muscle activation analysis reveals both the promise and fundamental challenges of achieving physiological fidelity through kinematic imitation alone. By lowering the computational and data barriers to musculoskeletal simulation, MuscleMimic enables systematic model validation across diverse dynamic movements and broader participation in neuromuscular control research. Code, models, checkpoints, and retargeted datasets are available at: https://github.com/amathislab/musclemimic

Chengkun Li, Cheryl Wang, Bianca Ziliotto +4

EPFL, Switzerland · McGill University, Canada · Feil & Oberfeld CRIR-Jewish Rehabilitation Hospital Research Center, Canada +1

Robotics Simulation Robot Imitation Learning Humanoid Robotics

Ask PDF

Jul 7, 2026

ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations

Humanoid robots are increasingly expected to perform contact-rich tasks that require not only accurate whole-body motion but also robust physical interaction with surrounding objects and humans. Although recent advances in humanoid motion imitation and whole-body control have achieved remarkable tracking performance, existing datasets and benchmarks primarily focus on kinematic motion while largely overlooking synchronized interaction forces. As a result, current evaluations fail to capture how external interaction forces affect tracking accuracy, stability, and control robustness. In this paper, we present ThorArena, a benchmark for evaluating force-aware humanoid interaction based on human demonstrations with synchronized motion and force measurements. We collect a real-world interaction dataset that simultaneously captures whole-body human motion and forces exerted by both hands across six representative physical interaction tasks. Based on these demonstrations, we propose force-aware evaluation metrics that jointly assess whole-body tracking accuracy, robustness under different force levels, control effort, and episode survival through the Force-Aware Tracking Score (FATS) and complementary diagnostic metrics. We further establish a unified benchmark protocol that replays recorded interaction forces in simulation and provides a standardized evaluation interface for different humanoid control policies. Experiments on representative whole-body control policies demonstrate that force-aware evaluation reveals substantial performance differences that remain largely hidden under conventional no-force evaluation. ThorArena provides a practical and reproducible framework for studying force-aware humanoid interaction and offers a new benchmark for evaluating contact-rich humanoid behaviors.

Chenhao Yu, Hongwu Wang, Weitao Zhang +5

Beijing Academy of Artificial Intelligence · Technical University of Munich

Humanoid Robot Control Humanoid Robotics

Ask PDF