cs.ROOct 20, 2026

LocoPilot: Single-Operator Bilateral Teleoperation for Loco-Manipulation Policy Learning

Authors: Antonio Langella, Giovanni Minelli, Giovanni B. Dessy, Lorenzo Amatucci, Giulio Turrisi, Victor Barasuol, Claudio Semini

Organizations: LocoPilot

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Learning loco-manipulation policies requires demonstrations that capture coordinated base and arm motion on the target embodiment. Robot-free interfaces allow demonstrations to be collected independently of the target robot, increasing flexibility but without directly capturing platform-specific commands, proprioception, and body-mounted observations. On-robot teleoperation allows to capture these signals directly, but requires an interface capable of commanding the complete system concurrently. This paper presents LocoPilot, a single-operator bilateral teleoperation and data-collection framework implemented on a 12-DoF Unitree Go2 quadruped equipped with a 6-DoF AgileX Piper L manipulator, an Intel RealSense D435 wrist camera, and an Intel RealSense D405 head camera. A second 6-DoF Piper L arm, used as the leader and two handheld controllers provide simultaneous control of the base, body posture, manipulator, and gripper. The follower arm interaction forces are estimated without a dedicated force/torque sensor and rendered to the operator through a passivity-constrained haptic channel. A ROS~2 pipeline records synchronized multimodal observations together with a unified 11D action shared by teleoperation and policy deployment. Compared to prior work, LocoPilot uniquely unifies robot-in-the-loop legged data collection, force feedback without motion-capture instrumentation, and the same action interface at deployment. We evaluate the interface across multiple loco-manipulation tasks and validate the complete collection-to-deployment pipeline by fine-tuning MolmoAct2 \cite{MOLMO} on pick-and-place demonstrations and deploying on the physical robot without action retargeting or additional inverse kinematics. Code available at: https://github.com/iit-DLSLab/LocoPilot

Figures & tables

 M. ALOHATeLeManUMI-LegsOurs
 [ 11 ][ 15 ][ 20 ] 
Legged embodiment–✓✓✓
Robot in the loop✓✓–✓
Base action + proprioception✓✓–✓
Body-mounted view✓✓–✓
Robot force feedback to operator–––✓
TABLE I: Comparison with the closest systems.
Fig. 2: The two sides of the manipulation system setup. Left: the leader station, with the end-effector handle, and the handheld controller mounted. Right: the follower platform, a Piper L arm and a gripper-mounted RGB-D camera.
Fig. 3: Software modules and their placement. The robot-side modules are identical during demonstration collection and autonomous deployment; only the operator-side stack changes, with the leader arm, teleoperation loop, and recorder replaced by a single policy-inference node.
Fig. 4: Leader–follower tracking and interaction during a tabletop pick-and-place episode with a payload of known weight. (a) Norm of the end-effector position error ∥e∥\|\mathbf{e}\| . (b) Vertical component ∣Fz∣|F_{z}| of the estimated interaction force, with the payload weight drawn as a reference line. Blue and red shaded regions mark the free-motion and holding phases respectively.
ParameterValue
Control rate5 Hz5\text{\,}\mathrm{Hz}
Action dimension11
Prediction horizon10 actions ( 2 s2\text{\,}\mathrm{s} )
Adapter rank512
GPURTX 6000 Pro
TABLE II: Fine-tuning configuration.
Fig. 5: Snapshot of representative successful autonomous rollout of our fine-tuned vision-language-action model. The policy receives the two camera streams, the proprioceptive state, and the language instruction “Pick the red cube and place it on the blue base”, and commands the base and the arm through the same interface used during teleoperation, presented in Sec. III-B . In time order: the base approaches the cube (a, b), the robot pitches its body and the gripper closes (c), the cube is lifted and carried toward the target while the base repositions (d, e), and it is released on the target (f).
More in this topic

Loco-Manipulation

See all 33 papers

Explore similar work

CardsList
Sep 16, 2026

ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots

Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable robot trajectories. It then leverages these trajectories within a teacher-student distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: viloman-anonymous.pages.dev.

Zejie Tian, Ruibing Hou, Bingpeng Ma +2

State Key Laboratory of AI Safety, Institute of Computing Technology, CAS, China.

Visuomotor Policy LearningWhole-Body Robotic ManipulationRobot Imitation LearningHumanoid Robot ControlHumanoid Loco-ManipulationRobotic Loco-Manipulation+1
AskPDF
Apr 29, 2026

Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies

Quadrupedal loco-manipulation is commonly built on visual perception and proprioception. Yet reliable contact-rich manipulation remains difficult: vision and proprioception alone cannot resolve uncertain, evolving interactions with the environment. Tactile sensing offers direct contact observability, but scalable tactile-aware learning framework for quadrupedal loco-manipulation is still underexplored. In this paper, we present a tactile-aware loco-manipulation policy learning pipeline with a hierarchical structure. Our approach has two key components. First, we leverage real-world human demonstrations to train a tactile-conditioned visuotactile high-level policy. This policy predicts not only end-effector trajectories for manipulation, but also the evolving tactile interaction cues that characterize how contact should develop over time. Second, we perform large-scale reinforcement learning in simulation to learn a tactile-aware whole-body control policy that tracks diverse commanded trajectories and tactile interaction cues, and transfers zero-shot to the real world. Together, these components enable coordinated locomotion and manipulation under contact-rich scenarios. We evaluate the system on real-world contact-rich tasks, including in-hand reorientation with insertion, valve tightening, and delicate object manipulation. Compared to vision-only and visuotactile baselines, our method improves performance by 28.54% on average across these tasks.

Pokuang Zhou, Yuhao Zhou, Quan Khanh Luu +7

Purdue University, West Lafayette, IN, USA. · Istituto Italiano di Tecnologia, Genoa, Italy. · Columbia University, New York, NY, USA.

Legged RoboticsRL for RoboticsContact-Rich Robotic ManipulationTactile Sensing for Robotic ManipulationRobotic Loco-Manipulation
AskPDF
Aug 17, 2026

FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences

Visual loco-manipulation policies that can generalize to novel scenes and objects have long been a goal of robotics research. However, today's data-hungry algorithms make collecting sufficient demonstrations a struggle for tabletop manipulation, and even more so for humanoids that must also walk and balance. Learning from simulated data and transferring that behavior to the real world, as is commonly done in locomotion, sidesteps this struggle, so we replicate that recipe for loco-manipulation. In doing so, we find that cloning synthetic demonstrations results in a low performance ceiling no matter the amount of training data. Reinforcement learning breaks through it, and refining the cloned policy with Flow-GRPO on a single sparse reward yields performance that synthetic behavior cloning cannot match. Together, these stages form our end-to-end sim-to-real pipeline spanning more than 150,000 scenes, which we use to train FetchMan. We evaluate it on FetchMan-Bench, a simulation benchmark we release, and deploy it zero-shot on a real Unitree G1, where our single-object reach-and-pick policy walks to and grasps a target across unseen scenes at 73.3% success. Finally, we extend this recipe to multi-object training, a first step toward loco-manipulation generalist policies at this data scale.

Omar Rayyan, Zhi Li, Max Argus +4

University of California, Los Angeles · Allen Institute for AI · University of Washington

Robotic Loco-ManipulationLoco-ManipulationHumanoid Loco-ManipulationSim-to-Real TransferRL for RoboticsZero-Shot Robotic Manipulation+1
AskPDF