TouchBridge: Tactile-Conditioned Human-to-Robot Transfer for Dexterous Manipulation

Haozhe Zhang; Gu Zhang; Zirui Zou; Yuzheng Chen; Jinlong Wu; Zixuan Hu; Jianke Zhang; Yucheng Hu

cs.ROOct 29, 2026

TouchBridge: Tactile-Conditioned Human-to-Robot Transfer for Dexterous Manipulation

Authors: Haozhe Zhang, Gu Zhang, Zirui Zou, Yuzheng Chen, Jinlong Wu, Zixuan Hu, Jianke Zhang, Yucheng Hu,

Organizations: Tsinghua University · Zhejiang University · Nanyang Technological University · Hunan University ·

Abstract

Human demonstrations offer an efficient and scalable source of data for dexterous robot manipulation, but differences in sensing and hand morphology complicate the transfer of contact-rich interactions. We present TouchBridge, a hardware--software co-designed framework for human-to-robot data transformation. A wearable human demonstration interface and a corresponding robot interface establish cross-embodiment correspondence in visual and fingertip tactile sensing. We introduce Tac-Retarget, a tactile-conditioned retargeting method that uses measured contact to align fingertip contact patches and preserve their spatial relations within and across hands. Unlike object-aware or simulation-based transfer pipelines, Tac-Retarget requires no object mesh, object-pose estimate, or physics simulation for retargeting, enabling direct and scalable conversion of human demonstrations into robot-domain training data. Combined with visual synthesis and camera-anchored action conversion, TouchBridge produces robot-domain demonstrations for fine-tuning a pretrained vision-language-action (VLA) model. Across four real-world contact-rich tasks, policies fine-tuned solely on TouchBridge-converted human demonstrations achieve a 72.5% average success rate, compared with 77.5% using an equal number of teleoperated robot demonstrations, while human demonstration collection is 7.0x faster. Under the same policy and training setup, Tac-Retarget outperforms DexPilot and UniDex by 25.0 and 42.5 percentage points, respectively.

Figures & tables

Method	High-DoF Hands	Sim.- Free	No Object Pose/Geom.	Robot-Free Fine-Tuning
TouchBridge	✓	✓	✓	✓
AINA [ 4 ]	✗	✓	✗	✓
HumanEgo [ 3 ]	✗	✓	✗	✓
DexUMI [ 15 ]	✗	✓	✓	✓
ManipTrans [ 11 ]	✓	✗	✗	✓
TopoRetarget [ 9 ]	✓	✗	✗	✓

TABLE I: Comparison of human-to-robot transfer frameworks. High-DoF denotes hands with at least 20 DoF. Robot-Free denotes fine-tuning without robot-collected demonstrations. ✗ ✓ indicates that the capability has not been experimentally validated.

Fig. 2: Human–Robot Hardware Co-design. (a,b) Human demonstration and robot deployment interfaces. (c,d) Corresponding fingertip tactile-sensor placements and wrist-camera configurations reduce mismatch across embodiments.

Fig. 3: Tac-Retarget . (a) Fingertip pose alignment tracks the demonstrated fingertip positions and orientations in the camera frame. (b) Contact alignment matches the patch center and active taxels between the human and robot fingertips. (c) A tactile-activated contact graph preserves the demonstrated spatial relations among active contacts within and across the two hands. Weighting terms and regularization are omitted for clarity.

Fig. 4: Human-to-robot data generation. From left to right: raw human demonstrations, removal masks, inpainted scenes, and synthesized robot-domain observations.

Fig. 5: Four contact-rich tasks evaluated in our experiments.

Fig. 6: (a) Action replay success rates across four tasks, with 20 converted human demonstrations replayed per task under matched initial scene conditions. (b) Real-robot success rates of GR00T N1.7 [ 21 ] policies fine-tuned on 60 demonstrations per task from each data source. (c) Average collection time per successful demonstration, including failed attempts and scene resets. Multipliers indicate collection speedups. Error bars show standard deviation over the four tasks.

Fig. 7: Observation comparison across data domains. (a) Multi-view observations from real-robot and human-derived data. (b) VLM feature distributions before the action expert.

Fig. 8: Qualitative comparison of Tac-Retarget, its kinematic-only variant, and retargeting baselines.

Method	Contact error (mm)
Tac-Retarget (Ours)	4.667
DexPilot	6.354
UniDex	7.715
Tac-Retarget (w/o contact graph)	6.659
Tac-Retarget (w/o contact alignment)	7.071
Tac-Retarget (kinematic only)	7.540

TABLE II: Contact error of retargeting methods and tactile ablations.

Fig. 9: Real-robot policy success rates comparing Tac-Retarget, its tactile ablations and existing retargeting methods. Error bars show standard deviation over the four tasks.

Fig. 10: Real-robot policy success rates under different policy and pretraining settings. Error bars show standard deviation over the four tasks.

Data stream	Collecting rate
Egocentric RGB	30 Hz
Left/right wrist-view RGB	30 Hz
Bilateral Manus hand poses	130 Hz
Fingertip tactile stream	90 Hz

TABLE III: Collecting rates of the human demonstration streams.

Fig. 11: Reconstruction of camera-frame tactile fingertip poses. (a) Wrist-mounted AprilTag fiducial frames detected in the egocentric image. (b) Manus fingertip positions (red) reconstructed using the calibrated wrist-to-cube transforms. (c) Tactile-sensor origins (blue) obtained by applying the fixed sensor-to-fingertip mounting transform. (d) Close-up of the fixed spatial offsets between the Manus fingertips (red) and tactile-sensor origins (blue).

Parameter	Value	Description
$\tau_{F}$	$0.02$	Contact threshold on the total fingertip response.
$\tau_{s}$	$0.01$	Response threshold for selecting active taxels.
$\lambda_{p}$	$100$	Camera-frame fingertip-position tracking weight.
$\lambda_{R}$	$1$	Fingertip-orientation tracking weight.
$(\beta_{0},\beta_{c})$	$(1,\,2)$	Baseline tracking multiplier and contact-dependent increment.
$\eta$	$0.05$	Pose-tracking scale applied during detected contact.

TABLE IV: Selected hyperparameters for Tac-Retarget.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,704 papers

Touch2Robot: Robot Touch in the Human Demonstration Loop
Sep 21, 2026Shengcheng Luo, Xiaoyang Cheng, Hong Ying +6Tactile Sensing for Robotic Manipulation Robotic Data Collection
ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University +1
TacOT: Learning Contact-Rich Dexterous Manipulation from Human Demonstrations via Tactile-Guided Optimal Transport
Oct 3, 2026Xingting Li, Yifan Han, Zijian Lin +4Human-to-Robot Motion Imitation Contact-Rich Robotic Manipulation
Shenzhen Ubiquitous Data Enabling Key Lab, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China. · School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai 200030, China. · Xspark AI, Shenzhen, China. +1
Kinematic-Tactile Aligned Dexterous Retargeting with Contact Wrench Space Alignment
Oct 29, 2026Xinyuan He, Ziheng Xu, Yueyuan Chen +6Motion Retargeting Dexterous Manipulation
State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · College of Electronics and Information Engineering, Tongji University, Shanghai 201804, China · International Collaboration for Creative Research Teams, Shanghai Jiao Tong University, Shanghai 200240, China +1

Sep 21, 2026J/K move · Enter open · S save

Touch2Robot: Robot Touch in the Human Demonstration Loop

Shengcheng Luo, Xiaoyang Cheng, Hong Ying, Xiaoying Zhou+5

ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University+1

Tactile Sensing for Robotic Manipulation Robotic Data Collection Motion Retargeting Robotic Manipulation Robot Imitation Learning Human-to-Robot Motion Imitation+1

Open paper AskPDF arXiv

Human demonstrations offer a scalable way to collect manipulation data, but their contacts may be unstable or infeasible when transferred to a robot hand. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases the cost of data collection. To address this trade-off, we present Touch2Robot, a framework that lets humans collect demonstrations while seeing how the target robot hand would contact the object. We capture human hand motion, tactile-glove measurements, and object motion during human manipulation. These recordings guide object-specific RL policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. We distill the learned behaviors into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2Robot improves average real-robot replay completion from 37.9% to 72.1% over visual-only feedback, while reducing the collection time per replay-successful demonstration from 58.6s to 18.2s. Reconstructed target-hand contacts achieve 44.2% F1 against real-robot tactile measurements, and policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback. These results show that bringing robot touch into the human demonstration loop improves both the quality and efficiency of scalable dexterous data collection. Project webpage: https://Touch2Robot.github.io/.

Abstract

Robotic Manipulation

Explore similar work

Touch2Robot: Robot Touch in the Human Demonstration Loop

TacOT: Learning Contact-Rich Dexterous Manipulation from Human Demonstrations via Tactile-Guided Optimal Transport

Kinematic-Tactile Aligned Dexterous Retargeting with Contact Wrench Space Alignment