Beyond Human-Derived Motion: Exploring Robot-Specific Execution for Construction Humanoids
Authors: MinJun Chang, Taehyung Kim, Yong K. Cho, Francis Baek
Organizations: School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Mobility Systems Engineering, Yonsei University, Seoul, South Korea
Construction productivity has remained a per- sistent challenge, motivating growing interest in humanoid robots that can operate in workspaces and with tools designed for human workers. Realizing this potential requires effective motion control, for which human-derived motion transfer has emerged as a promising approach by capturing and retargeting human demonstrations to humanoid robots. While promising, however, human-derived motion transfer may not fully define how a humanoid should ultimately execute the transferred motion. Once task-relevant motion information has been con- veyed, the humanoid may also exploit its own embodiment through motions that depart from the human demonstration. This study examines whether such Robot-Specific Motion offers an advantage in motion execution efficiency over Human- Derived Motion. On an adult-sized Unitree H1-2, Human- Derived Motion was generated through vision-based capture and retargeting, whereas Robot-Specific Motion used inverse kinematics and minimum-jerk transfers to reach the same targets while departing from the demonstrated posture, as verified using Rapid Upper Limb Assessment (RULA). Across a pick-and-stack sequence representing construction material handling, with four matched motion targets and three trials per condition, Robot-Specific Motion achieved 12.87 s versus 25.03 s in execution time, 0.92 m versus 4.33 m in trajectory length, and 58.7 J versus 247.3 J in two-arm mechanical work. These findings suggest that allowing humanoids to depart from demonstrated human motion may offer additional opportunities for more efficient motion execution, while broader construction tasks are needed to determine whether such differences trans- late into productivity gains.
Figures & tables
Fig. 1: Matched phases of the pick-and-stack sequence. Human-Derived Motion (blue) lifts high and holds its hands wider than the block, while Robot-Specific Motion (red) works from a sustained knee flexion.
Fig. 2: RULA grand score over the motion sequence. Human-Derived Motion (orange) tracks the human demonstration (red), while Robot-Specific Motion (teal) holds a lower and more constant score, confirming the intended postural distinction.
measure
Human-Derived
Robot-Specific
ratio
execution time (s)
25.03 ± 0.06
12.87 ± 0.15
1.9x
trajectory length (m)
4.33 ± 0.08
0.92 ± 0.03
4.7x
two-arm mech. work (J)
247.3 ± 3.0
58.7 ± 5.6
≈ 4.2x
four targets reached
3/3
3/3
N/A
TABLE I: Motion execution efficiency over the pick-and-stack sequence, three trials per condition (mean ± sd)
Fig. 3: Block-center trajectories in side view. Human-Derived Motion (blue) loops above the work zone, while Robot-Specific Motion (red) transfers directly between targets.
Learning executable motions from human videos offers a scalable solution for humanoid robots to acquire demonstration motions. However, existing pipelines typically first construct an explicit human motion representation and then convert it into robot motions via motion retargeting. Although such methods can effectively leverage large volumes of existing human data for training, the substantial differences between humans and humanoid robots in locomotion mechanisms and joint degree-of-freedom configurations make motions generated by this human-representation-centric approach difficult to execute on robots. Furthermore, errors introduced during human motion estimation inevitably propagate to the retargeting stage and cannot be eliminated via joint optimization. We propose BeyondRetarget, an end-to-end framework that directly maps monocular RGB videos to robot motions. Discarding the explicit human representation, this framework learns robot-oriented implicit representations directly from visual observations, enabling the model to capture cross-morphology motion structures. To generate motions more suitable for robot execution, we further design a contact-aware motion optimization mechanism to improve temporal consistency and physical plausibility. Experiments show that BeyondRetarget significantly improves the accuracy and robustness of generated robot motions, while achieving higher execution success rates and lower latency in both simulation environments and real humanoid robots.
Tianyu Xiong, Yi Lu, Jinrui Wang +6
School of Electronic Science and Engineering, Nanjing University, Nanjing, China · Jiangsu Mobile Information System Integration Co., Ltd., Nanjing, China · China Mobile Zijin (Jiangsu) Innovation Research Institute Co., Ltd., Nanjing, China +1
Humanoid robots, with their human-like shape and multi-tasking capabilities, are well-aligned with human-dominated workplaces, like those in civil and construction engineering, where they could collaborate with human workers or autonomously perform physically demanding and hazardous tasks. Despite this promise, limited research has explored how to endow these robots with the practical capabilities needed to perform construction tasks. To this end, this study proposes a novel perception-and-action system that enables humanoid robots to learn and perform construction tasks from worker demonstrations. This system contains two deep networks: Humanoid-PoseNet, which extracts human postures and translates them into mechanically feasible poses for a humanoid robot; and Humanoid-ActionNet, which learns robot-executable actions based on these translated poses. Experimental results demonstrate that the humanoid robot reliably executed eight construction-related actions, achieving an average motion-tracking error of 82.45 mm MPJPE (Mean Per Joint Position Error). This work provides an early step toward deploying humanoid collaborators in construction.
Yanxi Liu, Yizhi Liu
Department of Civil and Environmental Engineering, Syracuse University, 158 Link Hall, Syracuse, NY 13244, USA · Department of Civil and Environmental Engineering, Syracuse University, 151J Link Hall, Syracuse, NY 13244, USA
We present a teleoperation system that enables a single operator to perform construction tasks on a Unitree G1 humanoid, combining extended reality (XR) based upper body control with pedal-based locomotion to enable simultaneous manipulation and locomotion. Motivated by persistent labor shortages, hazardous working conditions, and challenges in humanoid autonomy, we investigate teleoperation as a practical near-term approach for reducing physical strain on workers while generating high quality demonstration data. We evaluate the system on two representative construction tasks drawn from O*NET occupational database, and report task success and completion time relative to a manual baseline. The system achieved 100% success on tool transport and 80% success on surface painting, with teleoperation requiring substantially more time compared to manual execution.
Parastoo Ali Pour, David R. Martin, Chang Min Hur +6
Department of Mechanical and Aerospace Engineering, University of California at Irvine, Irvine, CA 92697 USA · Department of Electrical Engineering and Computer Science, University of California at Irvine, Irvine, CA 92697 USA · Professional Master of Embedded and Cyber-physical Systems Program, University of California at Irvine, Irvine, CA 92697 USA