TERRA: Terrain-Aware Reconstruction, Retargeting and Control for Musculoskeletal Locomotion

Merkourios Simos; Chengkun Li; Bianca Ziliotto; Alexander Mathis

cs.ROSep 29, 2026

TERRA: Terrain-Aware Reconstruction, Retargeting and Control for Musculoskeletal Locomotion

Authors: Merkourios Simos, Chengkun Li, Bianca Ziliotto, Alexander Mathis

Organizations: EPFL

Abstract

Recent advances in musculoskeletal modeling and reinforcement learning have enabled muscle-actuated agents to reproduce increasingly complex human motions. Yet these capabilities remain largely confined to flat ground, in part because motion datasets rarely include aligned terrain geometry and because retargeting terrain interactions to complex musculoskeletal bodies is challenging. We present TERRA, an end-to-end pipeline for terrain-aware retargeting and control of musculoskeletal locomotion. From kinematic trajectories alone, TERRA combines terrain priors, estimated contacts, and negative free-space evidence to recover task-relevant support geometry. TERRA further considers anatomical, tendon-continuity, and contact constraints during retargeting. Using the resulting motion-terrain pairs from five datasets, we successfully train a single muscle-actuated control policy on 9.4 hours of diverse locomotion. Across reconstruction, retargeting, and held-out tracking benchmarks, TERRA improves terrain accuracy, sharply reduces anatomical and interaction violations, and achieves the highest observed completion rate over supported terrain families. Overall, TERRA provides a practical route from scene-less motion data to muscle-actuated locomotion over diverse non-flat terrain. Project website: https://cnai.epfl.ch/terra/

Figures & tables

Method	Kinematic constraints	Terrain reconstruction	Terrain retargeting	MSK
GMR [ 24 ]	Joint bounds	✗	✗	✗
OmniRetarget [ 27 ]	Hard interaction	✗	✓	✗
SceneBot [ 21 ]	Joint bounds	✓	✓	✗
MuscleMimic [ 7 ]	Joint/equality	✗	✗	✓
TERRA	Hard + anatomy	✓	✓	✓

TABLE I: Capabilities of representative motion-processing methods. MSK denotes support for musculoskeletal target models.

Fig. 2: TERRA method overview. Heterogeneous motion data are converted to SMPL-H, used to reconstruct compact collision terrain, and retargeted to the musculoskeletal model with terrain-aware constraints. The resulting paired trajectory–terrain scenes train a muscle-actuated tracking policy, whose biomechanical outputs are compared with subject-matched force-plate and EMG data on held-out motions.

Fig. 3: Terrain-reconstruction. Method comparison on a Vielemeyer

10^{\circ}

descent ramp (top) and PRISM stepping boxes (bottom). Black crosses denote the same reference toe contacts in every panel, and dotted segments connect each contact to the reconstructed height at the same horizontal location. Vertical height is exaggerated for visibility.

(a) Known terrain
	All known terrain	Gait120			Darmstadt	Vielemeyer
Method	Family acc. (%) $\uparrow$ ( $N=5{,}474$ )	Grade MAE ( ∘ ) $\downarrow$ ( $N=1190$ )	Riser MAE (mm) $\downarrow$ ( $N=1184$ )	Stool MAE (mm) $\downarrow$ ( $N=1102$ )	Riser MAE (mm) $\downarrow$ ( $N=1282$ )	Grade MAE ( ∘ ) $\downarrow$ ( $N=568$ )
Contact least squares	N/A	1.683 $\pm$ 3.533	23.42 $\pm$ 8.71	490.00 $\pm$ 0.00	150.49 $\pm$ 57.41	1.199 $\pm$ 1.088
Voronoi	N/A	N/A	42.24 $\pm$ 39.54	80.90 $\pm$ 121.19	24.81 $\pm$ 36.39	N/A
TERRA (without physical cues)	73.5	0.220 $\bm{\pm}$ 0.235	12.25 $\pm$ 5.35	45.62 $\bm{\pm}$ 13.57	7.12 $\pm$ 6.10	0.536 $\bm{\pm}$ 0.333 (N=566)
TERRA	99.9	0.223 $\pm$ 0.244	8.08 $\bm{\pm}$ 6.99	45.62 $\bm{\pm}$ 13.57	5.17 $\bm{\pm}$ 4.36	0.536 $\bm{\pm}$ 0.353

TABLE II: Terrain classification, reconstruction, and motion–terrain consistency across datasets. Entries are per-motion mean

\pm

population standard deviation.

	Retargeting							Policy tracking
Method	Success	$D_{\mathrm{ten}}$ (%) $\downarrow$	$D_{\mathrm{col}}$ (%) $\downarrow$	$D_{\mathrm{pen}}$ (%) $\downarrow$	$D_{\mathrm{skat}}$ (%) $\downarrow$	$D_{\mathrm{float}}$ (%) $\downarrow$	Contact F1 (%) $\uparrow$	Success (%) $\uparrow$	MPJPE (mm) $\downarrow$
MM-MoCap-Body	6478/6498	2.208 $\pm$ 3.852	12.09 $\pm$ 19.90	54.56 $\pm$ 35.53	4.86 $\pm$ 5.10	22.80 $\pm$ 36.78	92.86 $\pm$ 7.47	47.34 $\pm$ 0.22	90.26 $\pm$ 0.60
GMR	6498/6498	0.556 $\pm$ 1.087	21.82 $\pm$ 39.14	19.98 $\pm$ 24.19	9.25 $\pm$ 14.62	11.82 $\pm$ 11.27	74.19 $\pm$ 21.01	56.43 $\pm$ 0.64	91.97 $\pm$ 1.73
OmniRetarget	6392/6498	0.466 $\pm$ 1.243	4.98 $\pm$ 10.82	31.97 $\pm$ 29.22	12.70 $\pm$ 10.54	3.61 $\pm$ 9.79	86.05 $\pm$ 11.79	80.29 $\pm$ 1.39	87.23 $\pm$ 2.92
TERRA (no residuals)	6498/6498	0.134 $\pm$ 0.407	3.93 $\pm$ 8.45	33.45 $\pm$ 33.63	8.88 $\pm$ 7.03	4.18 $\pm$ 11.00	88.49 $\pm$ 9.51	86.52 $\pm$ 0.25	80.58 $\pm$ 2.31
TERRA+Voronoi	6470/6498	0.081 $\pm$ 0.285	0.54 $\pm$ 1.35	2.99 $\pm$ 14.97	2.63 $\pm$ 2.82	9.97 $\pm$ 18.73	92.96 $\pm$ 5.96	82.16 $\pm$ 0.41	71.65 $\bm{\pm}$ 1.00

TABLE III: Retargeting performance across datasets and held-out policy-tracking performance.

Parameter	Value
Stationary contacts	$v_{s}=0.30$ m/s; $n_{\min}=5$ ; $W_{e}=\pm 0.30$ s; $\epsilon_{\rm loc}=0.05$ m.
Geometry margins	Contact expansion $0.10/0.05$ m; maximum extension $0.00/0.15$ m; free-space clearance $0.03/0.02$ m (vertical/horizontal).
TERRA weights	Orientation $0.5$ ; anchor $100$ ; velocity/displacement $150/200$ ; height $400$ ; separation $20{,}000$ ; clearance/route $1{,}500$ ; seat $2{,}000$ .
Constraint thresholds	$\epsilon_{\rm pen}=0.0009$ m; recovery cap $\rho=0.010$ m/iteration; leg separation $0.002$ m; collision repair $0.005$ m for $\leq 3$ frames.
$D_{\rm ten}$	Fraction with $\Delta l/l_{0}>\max(10\,\mathrm{EMA},0.001)$ and $>0.05$ .
Actor/critic width	$2048,512,512,512,512,512,512$ .

TABLE IV: Key numerical parameters for reconstruction, retargeting, and policy training.

Fig. 4: A policy trained on TERRA motions can imitate held-out motion on various terrains, such as stairs, ramps, individual platforms, and chairs.

Fig. 5: Qualitative comparison of population-averaged physiological profiles on held-out motions. Every human and policy trace was independently min–max normalized, yielding a waveform-shape comparison. Shading denotes between-subject SEM. VL: vastus lateralis; VM: vastus medialis; SOL: soleus; GAS: gastrocnemius; BF: biceps femoris; TA: tibialis anterior.

	Level	Ramp	Stairs	Stool	All
	(148)	(1,758)	(2,466)	(1,102)	(5,474)
Clean	100.00	100.00	99.84	100.00	99.93
Noise 2 mm	100.00	99.98	99.81	100.00	99.91
Noise 5 mm	100.00	99.75	99.10	100.00	99.51
Noise 10 mm	99.59	95.78	96.52	99.96	97.06
Support $-10\%$	100.00	98.76	99.61	89.46	97.30

TABLE V: Terrain reconstruction accuracy (%) across perturbations.

Success rate (%)
Terrain ( $N$ )	OmniRetarget	TERRA+Voronoi	TERRA
Flat (262)	88.07 $\pm$ 0.62	91.09 $\pm$ 0.75	91.70 $\bm{\pm}$ 0.46
Ramp (286)	98.09 $\pm$ 0.08	89.18 $\pm$ 1.06	99.49 $\bm{\pm}$ 0.18
Stairs (422)	74.14 $\pm$ 3.49	85.01 $\bm{\pm}$ 0.24	81.90 $\pm$ 1.73
Boxes (61)	38.91 $\pm$ 4.64	39.45 $\pm$ 0.19	46.45 $\bm{\pm}$ 4.45
Seats (246)	72.11 $\pm$ 0.61	70.19 $\pm$ 0.38	89.32 $\bm{\pm}$ 0.82

TABLE VI: Held-out policy tracking by terrain family.

Explore similar work

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal
Jul 15, 2026Alex Brandes, Haig Conti Georges Sajelian, Manthan Patel +11Training-Set Long-Tail Motions Synthesis
ETH Zurich · TU Munich · Stanford +1
ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting
May 7, 2026David Müller, Agon Serifi, Sammy Christen +3Cross-Embodiment Retargeting Motion Imitation
Disney Research, Switzerland
SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction
Jun 25, 2026Sirui Chen, Shibo Zhao, Zhen Wu +3Full-Body Humanoid Control Human Motion
Stanford, Amazon FAR United States · Amazon FAR United States · CMU, Amazon FAR United States

Jul 15, 2026, cs.ROJ/K move · Enter open · S save

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

Alex Brandes, Haig Conti Georges Sajelian, Manthan Patel, Dominik Hollidt+10

ETH Zurich · TU Munich · Stanford+1

Training-Set Long-Tail Motions Synthesis Wearable Devices

Open paper AskPDF arXiv

Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.

Abstract

Explore similar work

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting

SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction