Contact Imitation Interface: Transferring Dexterous Skills by Showing How Contact Unfolds

Zhixuan Xu; Yichen Li; Bingchen Wu; Rui Wang; Yiwen Hou; Zixuan Liu; Tianyu Qiu; Lin Shao

cs.ROOct 29, 2026

Contact Imitation Interface: Transferring Dexterous Skills by Showing How Contact Unfolds

Authors: Zhixuan Xu, Yichen Li, Bingchen Wu, Rui Wang, Yiwen Hou, Zixuan Liu, Tianyu Qiu, Lin Shao

Organizations: National University of Singapore · RoboScience · South China University of Technology

Abstract

Robot-free human demonstrations offer a promising route to scaling dexterous manipulation data. The key challenge is to preserve the demonstrator's natural dexterity while producing contact-grounded observations and actions that transfer to robot hands. We present Contact Imitation Interface (CIMI), a framework with contact-grounded demonstration and learning interfaces. The demonstration interface combines shared contact geometry and fingertip RGB views with contact-alignment-first retargeting and reprojection of wider scene views. The learning interface introduces a hierarchical policy in which a contact-surface planner predicts relative contact-surface trajectories and a controller coordinates wrist and finger commands to improve contact alignment. CIMI combines contact-grounded observations, retargeting, and action representations, improving mean final-stage success by at least 50 percentage points over baseline policies across six hand-task pairs on LEAP and Wuji. Hardware designs and software are publicly available.

Figures & tables

Fig. 1: CIMI transfers contact-sensitive skills from human demonstrations to robot hands. Shared fingertip modules provide corresponding contact surfaces and camera views for human demonstration and execution across different robot hands. Bottom: tasks involving deformable objects, multi-finger coordination, and precise contact.

Interface	Focus	Demonstrator dexterity		Demonstration–execution interface			Cross-hand reuse
		Motion freedom	Haptic feedback	Contact geometry	Contact observations	Action mapping
Bare-hand video [ 6 , 16 , 17 , 18 , 19 , 20 , 5 , 21 , 22 ]	Human	Unconstrained	Direct touch	Mismatched human contact surfaces	None	Hand-pose retargeting	${\color[rgb]{0.0938,0.7266,0.3047}\checkmark}$

Sensing glove [ 23 , 8 , 9 , 7 ]	Human	Near-free	Through glove	Glove contact surfaces; matching varies	Human/shared tactile (optional)	Glove-state retargeting	${\color[rgb]{0.0938,0.7266,0.3047}\checkmark}$

Teleoperation [ 24 , 11 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 12 ]	Robot	Controller/robot-limited	Remote/optional	Matched robot contact surfaces	Robot sensing (optional)	Native robot commands	${\color[rgb]{0.0938,0.7266,0.3047}\checkmark}/{\color[rgb]{1,0.1992,0.1992}\times}$

TABLE I: Demonstration interfaces: dexterity, contact geometry and observations relative to execution, and cross-hand reuse. Mixed entries vary across cited systems.

Fig. 2: Contact connects demonstration and learning in CIMI. The demonstration interface captures contact-surface trajectories and observations, then constructs robot demonstrations through contact-alignment-first retargeting and context-view reprojection. The learning interface predicts relative contact-surface trajectories and realizes them through coordinated wrist and finger control.

Fig. 5: Real-world rollouts and failure cases. Orange-bordered insets ( ) show contact-camera views; within each inset, blue lines ( ) mark the contact surface and yellow outlines ( ) mark object boundaries. Green ( ) and blue ( ) backgrounds show successful LEAP and Wuji rollouts, respectively; red ( ) marks CTX and WristJoint failures.

Policy	Visual	LEAP Hand								Wuji Hand				Mean
		Cube		Coin		Letter		Bulb		Cube		Coin
		Align	Lift	Wedge	Lift	Open	Lift	Screw	Light	Align	Lift	Wedge	Lift
WristJoint	CTX	0	0	0	0	0	0	35	5	0	0	0	0	0.8
ContactSurface	CTX	0	0	0	0	0	0	50	10	0	0	0	0	1.7
WristJoint	CTX+CC	35	20	50	35	25	15	85	70	30	15	40	30	30.8
ContactSurface	CTX+CC	85	75	95	85	90	80	95	90	75	70	90	85	80.8

TABLE II: Policy and observation comparison on LEAP and Wuji hands. SR in %; 20 rollouts per method, hand, and task. Mean SR averages final-stage SR equally over all six hand–task pairs.

Retargeting		Coin			Bulb
Target	Priority	Error (mm) $\downarrow$	SR (%) $\uparrow$		Error (mm) $\downarrow$	SR (%) $\uparrow$
			Wedge	Lift		Screw	Light
Centers	Context	15.37	0	0	7.85	10	0
Corners	Context	14.67	0	0	5.72	25	0
Centers	Contact	9.31	0	0	4.36	50	20
Corners	Contact	0.51	95	85	1.09	95	90

TABLE III: Retargeting ablation on LEAP. Priority denotes context- or contact-alignment-first. Gray: DexCap-style [ 3 ] ; blue: ours. Error is the mean contact-corner distance over valid frames. SR uses 20 rollouts per variant and task; our SRs are from Table II .

Fig. 6: 15-minute collection throughput on coin picking and bulb installation.

Variant	Coin		Bulb		Mean SR (%) $\uparrow$
	Wedge	Lift	Screw	Light	Mean SR (%) $\uparrow$
Context-view conversion
No overlay	90	80	75	45	62.5
No reprojection, no overlay	75	50	55	10	30.0
Planner proprioceptive input
No planner proprioception [ 11 ]	90	85	10	0	42.5

TABLE IV: Design ablations on LEAP. Stage-wise SR in %; 20 rollouts per variant and task. All variants use our ContactSurface + CTX+CC framework with the listed changes. Citations denote related input styles adapted to CIMI. Our SRs are from Table II . Mean SR averages Coin Lift and Bulb Light.

Fig. S1: Transformation tree for visual contact-surface tracking. Each edge specifies the child frame’s pose in its parent frame. Dashed yellow edges are estimated from videos; solid orange edges are fixed mounting calibrations.

Component	Qty.	Cost (USD)
HBVCAM-2307-FPC82 V11 contact camera	3	$28.25 each
Orbbec Gemini 305 stereo camera	2	$259.18 each
DCXG500 RGB tracking camera (middle-finger and external views)	2	$84.42 each
Fabrication materials (printed parts, silicone sleeves, friction tape, and fiducials)	1 set	$7.00 (est.)
Estimated hardware total		$778.95

TABLE S1: Bill of materials for one three-finger demonstration interface. The total covers three contact cameras, two Gemini 305 cameras, two auxiliary tracking cameras, and the combined fabrication-material estimate.

Fig. S2: Qualitative lighting robustness of fiducial detection. (a,b) Low-light examples. (c,d) Examples with spatially varying colored illumination. For each condition, the left panel shows dense keypoints with enlarged tag-grid insets, and the right panel shows detected tag outlines. All detection overlays are preserved from the original images.

Fig. S3: Contact-surface planner architecture. The upper path encodes the

V

configured RGB views and the current inter-finger geometry into a condition vector. The lower path evaluates the conditional velocity network and integrates it for three Euler steps to predict an

H

-step contact-surface trajectory. Blue groups RGB processing, green groups geometric inputs, conditioning, and the predicted trajectory, and orange groups flow prediction. Dashed boxes and arrows identify optional camera inputs. Here

F

is the number of controlled fingers and each contact-surface pose has nine coordinates (three translation and six rotation coordinates).

Setting	Value
Solver schedule	4,000 projected Adam steps, then up to 1,000 L-BFGS-B steps
Optimization variables	Scaled to $[-1,1]$
Floating-root bound	$\pm 0.20$ m per axis from initialization
Temporal-gap handling	Edge weight $1/g$ for a $g$ -frame interval; no edge across more than 16 missing frames

TABLE S2: Task-independent retargeting optimizer configuration.

Stage	Time (s)
Visual tracking (image2cube)	57
Contact-surface retargeting (cube2qpos)	43
Viewpoint reprojection and robot overlay	72
Image augmentation and RADIO caching	52
Total	224

TABLE S3: Coin-picking processing on an RTX 4090 D (10 demonstrations, 557 frames). Serial stage wall times include initialization and output writing.

Task	Retained / stored	Retention	Valid frames
Coin	209 / 210	99.5%	99.3%
Letter	205 / 205	100.0%	95.5%
Bulb	115 / 117	98.3%	79.0%

TABLE S4: Data retention for audited LEAP task datasets. Valid-frame coverage is weighted by recording length, not averaged over recordings.

Fig. S4: Visual augmentation for coin picking. Rows show the thumb and middle-finger contact views, followed by wrist context views with nominal and perturbed camera extrinsics. The perturbed wrist render also varies robot lighting and materials. Each base image has four whole-image MIL blends with photometric augmentation and four photometric-only variants. Blend columns give original/texture percentages; the two 60/40 candidates use distinct textures.

Setting	Value
Contact-surface scaling	Translation $\times 20$ ; six rotation coordinates unchanged
Direct baseline scaling	Global 2nd–98th percentile for non-rotation coordinates; rotation unchanged
Dropout	0
Batch and budget	256; up to 63,000 optimizer updates
AdamW	LR $10^{-4}$ ; $\beta=(0.95,0.999)$ ; $\epsilon=10^{-8}$ ; weight decay $10^{-6}$
LR and EMA schedules	500 warm-up updates then cosine decay; EMA inverse-gamma 1, power 0.75, maximum 0.9999

TABLE S5: Planner normalization and training settings.

Fig. S5: Contact-surface controller architecture. The deterministic geometric front end converts the planner’s relative contact-surface trajectory to ordered target corners and constructs absolute and reference-relative frame features. A shared residual MLP then decodes every horizon step in one batched forward pass. Green denotes inputs and outputs, blue denotes deterministic geometric feature construction, and orange denotes the learned decoder. Here

F

is the number of controlled four-joint fingers,

d_{x}

is the per-step feature dimension, and

H=16

is the trajectory horizon.

Parameter	Range
Temperature $\tau$	$[-0.40,0.40]$
Tint $\eta$	$[-0.12,0.12]$
Exposure (EV)	$[-1,1]$
Contrast	$[0.60,1.40]$
Saturation	$[0.60,1.40]$
Hue	$[-0.04,0.04]$

TABLE S6: Uniform photometric-augmentation ranges; hue is measured in turns. Temperature and tint set RGB gains to

(e^{\tau-\eta/2},e^{\eta},e^{-\tau-\eta/2})

. Shading applies gain

1+a(x\cos\varphi+y\sin\varphi)/\sqrt{2}

, where

x,y\in[-1,1]

are normalized image coordinates.

Setting	Value
Motion augmentation
Endpoint translation	Independent uniform offsets in $[-0.02,0.02]$ m per axis
Endpoint rotation	Uniform angle in $[-0.15,0.15]$ rad; uniformly sampled axis
Endpoint joint offsets	Gaussian; standard deviation 8% of each joint’s range
Application	Probability 0.9 per batch; independent perturbations per trajectory
Base optimization

TABLE S7: Controller motion-augmentation and base optimization settings.

Hand	Task	CTX	Added contact views
LEAP	Cube	Wrist	Thumb, index
LEAP	Coin	Wrist	Thumb, index
LEAP	Letter	Wrist	Thumb, index
LEAP	Bulb	Wrist, web-space	Thumb, index, middle
Wuji	Cube, coin	Wrist	Thumb, middle

TABLE S8: Policy camera inputs. Finger names identify the controlled robot fingers.

Timed path	Median	95th percentile
Prepared RGB to commands	7.47	7.91
Cached features to commands	2.66	2.80
Controller only (16 targets)	0.84	0.98

TABLE S9: Inference latency (ms). Both policy paths include the controller; the timings are not additive.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,736 papers

One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry
Sep 1, 2026Satvik Sharma, Samrat Sahoo, Huang Huang +4Contact-Rich Robotic Manipulation Sim-to-Real Transfer
Stanford University
Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy
Sep 23, 2026Tara Sadjadpour, Siming He, C. K. Wolfe +5Robot Imitation Learning Motion Retargeting
Department of Electrical Engineering and Computer Science at the University of California, Berkeley, CA, USA
Contact-Aware Imitation Learning Through Contact Factorization
Oct 7, 2026Jiho Hong, Daeun Song, Sanghyun Kim +1Contact-Rich Robotic Manipulation Imitation Learning
Kyung Hee University · Ewha Womans University · University of Central Florida

Sep 1, 2026J/K move · Enter open · S save

One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry

Satvik Sharma, Samrat Sahoo, Huang Huang, Fei-Fei Li+3

Stanford University

Contact-Rich Robotic Manipulation Sim-to-Real Transfer Generalization in Robotic Manipulation Robotic RL Robotic Manipulation Learning from Demonstration

Open paper AskPDF arXiv

Dexterous manipulation with multi-fingered robot hands promises human-level dexterity, but collecting large-scale dexterous robot hand data remains difficult. Learning from human demonstrations has emerged as a scalable alternative to robot teleoperation, providing strong priors on object interaction and contact strategies. Recent sim-to-real RL methods incorporate such priors, but often (i) omit rewards that explicitly incentivize precise contact, yielding weak real-world performance, and/or (ii) generalize poorly to unseen object instances. We propose DemoMimic (Dexterous Motion Mimic), a policy that manipulates objects by focusing on their geometry local to the contact points. Its contact-centric rewards encourage precise contact and improve sim-to-real consistency, yielding a single real-world policy that transfers across objects of varying shape, scale, mass, and friction wherever local contact structure is preserved. Real-world ablations show that DemoMimic achieves 71% success across 16 objects, four tasks, and two robot-hand embodiments, with the smallest sim-to-real drop compared to baselines.

Abstract

Robotic Manipulation

Explore similar work

One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry

Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy

Contact-Aware Imitation Learning Through Contact Factorization