cs.RONov 10, 2026

A human-in-the-loop Control Paradigm for Construction Robotics: From VR-Guided Teleoperation to Semi-Autonomous Dual-Arm Operation

Authors: Jinhua Chen, Xiaoji Xu, Zhongyuan Liao, Huang Zhenmin, Zhang Ye, Changying Xiang

Organizations: Division of Integrative Systems and Design, Hong Kong University of Science and Technology, Hong Kong · Hong Kong University of Science and Technology, Hong Kong, China

Abstract

Construction tasks in hazardous and unstructured environments require both human adaptability and robotic precision, yet existing approaches often treat teleoperation and autonomy as separate control modes. This paper presents a human-in-the-loop control paradigm that enables structured transitions among VR-guided teleoperation, semi-autonomous shared control, and rule-based autonomous execution for contact-rich dual-arm construction tasks. The proposed system combines immersive VR guidance, a manually defined library of reusable skill primitives, and a shared-autonomy framework with explicit human-robot role allocation and command-priority rules. This integration allows operators to retain supervisory and corrective authority while delegating repetitive and precision-critical actions to the robot. The paradigm is validated on a dual-arm platform through representative facade-installation tasks. Results show that autonomous execution provides the best performance under well-calibrated nominal conditions, whereas semi-autonomous control improves resilience when environmental disturbances exceed the predefined autonomous pipeline's correction capability. Semi-autonomous control also substantially reduces subjective operator workload compared with full teleoperation. These findings demonstrate that effective construction-robot control should adapt human involvement to task and environmental uncertainty, using autonomy for repeatable operations and shared control for disturbance recovery.

Figures & tables

Explore similar work

CardsList
  1. KUKAloha: A General, Low-Cost, and Shared-Control based Teleoperation Framework for Construction Robot Arm

    Mar 20, 2026Yifan Xu, Qizhang Shen, Vineet Kamat +1Robot TeleoperationRobotic Control

  2. Toward Humanoid Robots in Construction: A Teleoperation Feasibility Study

    Sep 30, 2026Parastoo Ali Pour, David R. Martin, Chang Min Hur +6Robotic ConstructionHumanoid Loco-Manipulation

  3. Perception-and-action system for humanoid robot task execution in construction

    Aug 3, 2026Yanxi Liu, Yizhi LiuRobotic ConstructionMotion Retargeting