LETS: Low-Overhead External-Torque Sensing for Force-Feedback Bilateral Teleoperation
Authors: Zonghao Guo, Dongchen Guan, Yubo Wang, Peng Yang
Organizations: Institute of Automation, Chinese Academy of Sciences, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, China. · Beijing Amigos Robot Technology Co., Ltd., China.
Bilateral teleoperation with force feedback is becoming increasingly important, and the quality of external-torque estimation is critical to this process. However, manufacturer-provided disturbance-torque estimates on commercial robotic arms are often inaccurate, while the use of end-effector force/torque sensors adds cost and payload. We propose LETS, a physics-guided learning approach that assigns rigid-body dynamics to recursive Newton-Euler inverse dynamics and reserves learning for only the robot-specific residual. LETS uses only the robot's URDF model and joint feedback, deliberately excluding control commands and thereby decoupling external-torque estimation from control-command inputs. It trains on 12 minutes of free-motion data in under one minute, without requiring a dedicated end-effector six-axis force/torque sensor. In bilateral teleoperation, LETS achieves real-time external-torque estimation that enables stable force feedback. Its free-space error is 69% lower than the manufacturer-provided estimate, substantially reducing coupled oscillations between leader and follower; its contact error is 28% lower, yielding more accurate haptic feedback.
Figures & tables
Fig. 1: System overview: (a) bilateral platform, (b) bimanual box manipulation, and (c) single-arm contact interaction. Red arrows denote force feedback; blue arrows denote motion commands.
Fig. 2: LETS online system architecture. Fixed RNEA uses the URDF and current joint state, while the frozen GRU uses joint-state history. Their predictions are subtracted from measured joint torque to estimate external torque at 200 Hz. The estimate is mapped and scaled for leader-side force feedback; leader motion commands drive the follower, using joint impedance control in our experiments. Control commands do not enter the LETS estimator.
Protocol
Runs
Reference
Role
Train/validation/test
8/1/1
τext=0
fitting/selection/locked test
Subsequent free space
9
τext=0
frozen primary evaluation
F/T-referenced contact
9
τFT
frozen primary evaluation
Second physical unit
7/1/1
τext=0
fixed-protocol replication
Compound shift
10 poses
τFT
robustness stress test
TABLE I: Experimental protocols and their roles.
Fig. 3: Feedback-sensitive conditions: (a) free-space bilateral operation and (b) rigid contact with an independently measured end-effector wrench.
Estimator
Free space
Contact ( τFT )
Manufacturer estimate
0.54
0.57
Nominal RNEA
0.53
0.55
Nominal GMO
0.51
0.69
Calibrated GMO
0.17
0.57
Pure learning
0.35
0.54
LETS
0.17
0.41
TABLE II: Run-equal joint-torque RMSE ( N⋅m ) over nine frozen runs.
Fig. 4: Representative Joint 1 time-domain estimates from the frozen evaluation runs. (a) Free-space motion, where the external-torque reference is zero. (b) Rigid contact, where the reference is obtained independently from the six-axis F/T sensor as τFT .
Fig. 5: Controlled factorization study. (a) Mean per-joint test RMSE under a fixed NEXT architecture. (b) Uniform within-run subsampling demonstrates that 10% of physics-guided training windows outperform all pure-learning windows. (c) Run-equal contact RMSE for validation-selected models over nine runs using the independent joint-space reference τFT .
Method
RMSE
MAE
( N⋅m )
( N⋅m )
Manufacturer raw output
0.64
0.53
Nominal RNEA
0.56
0.48
NEXT direct torque
0.46
0.34
RNEA + NEXT residual
0.24
0.17
TABLE III: Fixed-architecture NEXT torque-estimation results ( N⋅m ).
Condition
LETS
Command-aware LETS
Free space
0.17
0.21
Contact
0.41
0.42
Distribution shift
0.52
0.59
TABLE IV: LETS and command-aware input ablation at 200 Hz (RMSE in N⋅m ).
Fig. 6: Compound-shift RMSE over ten held-out poses. Each cell reports the mean per-joint RMSE for one pose; lower is better. LETS is lowest in every pose-wise comparison.
Fig. 7: Real-robot closed-loop stress test with intermediate trajectory smoothing bypassed. Successive frames progress from left to right. Red denotes observed unloaded oscillation; green denotes smooth free motion or stable rigid contact. Continuous trajectories are provided in the supplementary video.
Robot teleoperation studies how people remotely control robots through interfaces, sensing, feedback, and autonomy, despite distance, delay, and embodiment gaps.
Contact-rich manipulation requires force sensitivity, but many robot arms lack dedicated force sensors due to their high cost. We present Neural External Torque Estimation (NEXT), a data-driven method that estimates external joint torques without needing any dedicated force sensors. NEXT trains in 1 minute from only 10 minutes of free-motion data, yet achieves estimates comparable to dedicated joint-torque sensors. NEXT enables force-feedback teleoperation on low-cost arms and improves policy learning through Force-Informed Re-Sampling Training (FIRST), which up-samples pre-contact and contact segments during behavior cloning. Across five long-horizon tasks, FIRST outperforms prior force-aware policies by over 17% in task progress. Together, NEXT and FIRST bring force-aware teleoperation and policy learning to off-the-shelf robots without additional sensing hardware. Video results and code are available at https://jasonjzliu.com/factr2
For robots to work safely in household environments, they need to be compliant and react to torque and force feedback during contact. However, the majority of existing data collection pipelines still lack the ability to capture force and torque data for learning active compliant policies. In this paper, we present Universal Manipulation Exoskeleton (UME), an upper-limb exoskeleton that provides real-time haptic torque feedback while recording whole-arm configurations and joint torque signals for teleoperation. With transparent torque feedback, human operators can even unsheathe kinematically constrained objects while blindfolded. UME is low-cost, lightweight, and portable. Equipped with an embedded IMU, it enables teleoperation for mobile manipulation. With our proposed universal retargeting algorithm, UME can teleoperate a range of robots, including the 7DoF OpenArm, 7DoF Franka, and 6DoF X-ARM. We demonstrate that this combination of capabilities enables learning bimanual, whole-body, and active compliant policies that operate effectively in highly constrained spaces. The learned robust autonomous policies achieve high success rates across a variety of tasks, including long-horizon mobile manipulation, force-mediated box flipping, visually occluded box pushing, and space-constrained tabletop manipulation. Videos, code, and additional information can be found at https://ume-exo.github.io.
Existing bilateral teleoperation platforms still rely on costly rigid six-axis force/torque sensors, tightly coupled leader-follower hardware, and kilohertz control loops. We present a Cartesian bilateral framework built on the hardware-agnostic WinGs Operating Studio (WOS) middleware, in which a low-cost compliant 6-DOF pose-and-force sensing end-effector, Delta6, is mounted on both sides so that each manipulator behaves as an end-effector 6-DOF series elastic actuator (SEA). The leader runs a damping-only admittance loop with a 6-D biquad notch filter; the follower realizes a stiffness-damping impedance through a position-based outer loop with a PID wrench-to-pose mapping. Three time scales (hardware I/O, mid-rate impedance/admittance, low-rate teleoperation messages) are explicitly decoupled, enabling the same application to drive heterogeneous arms. On a Lite6/FR3 testbed at 150 Hz, the system tracks stably under delays up to 120±40 ms and 1% packet loss, matches the prescribed virtual stiffness in contact, and shows a favorable cumulative energy signature in passivity-style tests.
Yue Feng, Weicheng Huang, I-Ming Chen
Robotics Research Centre Nanyang Technological University Singapore · WinGs Robotic USA