YOCO: You Only Calibrate Once! Fast Mocap Calibration for Dexterous Teleoperation

Yu Zhang; Yunqi Li; Yushi Du; Yi Ma; Yanchao Yang

cs.ROOct 8, 2026

YOCO: You Only Calibrate Once! Fast Mocap Calibration for Dexterous Teleoperation

Authors: Yu Zhang, Yunqi Li, Yushi Du, Yi Ma, Yanchao Yang

Organizations: The University of Hong Kong · Shenzhen Loop Area Institute · TranscEngram

Abstract

Dexterous teleoperation requires reliable human-hand state estimations. However, common low-cost motion-capture gloves and markerless trackers often exhibit biases that vary across users, glove fit, and recording sessions, degrading retargeting and demonstration quality. We present YOCO, a fast few-shot, fine-tuning-free calibration framework that corrects biased hand-pose streams from a small set of paired raw and target poses. Instead of optimizing a separate model for every operator or session, YOCO conditions a calibration HyperNet on the paired examples and predicts LoRA-style updates for a frozen MANO hand-estimation module, turning per-user calibration into a lightweight feed-forward adaptation step while preserving the geometric prior of MANO and the efficiency of a compact estimator. We train YOCO with synthetic drift augmentations on InterHand2.6M and evaluate on augmented InterHand sequences, offline real glove data, and dexterous teleoperation tasks. Across these settings, YOCO improves calibration efficiency, hand-state estimation quality and teleoperation performance compared with uncalibrated input and standard calibration baselines.

Figures & tables

Figure 1: Overview of YOCO . Low-cost mocap streams suffer from user-, device-, and session-specific bias. Given a few paired calibration poses, YOCO predicts a calibration update that corrects future mocap poses and improves downstream dexterous teleoperation.

Figure 2: YOCO model architecture. Given a calibration set $\mathcal{S}=\{(x_{i}^{n},x_{i}^{c})\}_{i=1}^{K}$ of $K$ paired noisy and target poses (left), the calibration HyperNet $h_{\psi}$ (middle) encodes each pair, aggregates them through a transformer-based calibration mapper with one CLS token per LoRA matrix, and decodes the tokens into LoRA updates $\Delta W$ . These updates are injected once per session into the frozen MANO estimator $g_{\phi}$ (right) to form $g_{\phi+\Delta W}$ , which maps each raw mocap query $x_{q}^{n}$ at frame rate to calibrated MANO pose and 3D joints.

Method	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	Tip-PA $\downarrow$	AUCJ $\uparrow$	Pinch $\downarrow$	Calib. s $\downarrow$
	(mm)	(mm)	(mm)		(mm)	(s)
No calibration	21.52	16.55	37.15	0.611	48.49	0.00
Ridge	17.62	13.00	28.28	0.676	45.35	1.6
Last-layer-Finetune	16.62	12.01	25.92	0.692	39.18	45.6
Full-Finetune	18.11	12.77	29.21	0.668	37.11	95.4
LoRA-Finetune	16.38	11.67	25.33	0.694	37.28	89.4

Table 1: Calibration results on the augmented InterHand2.6M evaluation setting. Calib. s reports the total adaptation time in seconds for whole validation sets on one H100.

Figure 3: PA-MPJPE versus calibration-pair count, including YOCO followed by test-time fine-tuning (YOCO-FT). The dotted line marks no calibration.

Figure 4: Goal states for the downstream dexterous manipulation tasks: twisting and opening a bottle cap, pinch-grasping earbuds, three-finger glue grasping, and tissue-box opening.

Method	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	Tip-PA $\downarrow$	AUCJ $\uparrow$	Pinch $\downarrow$	Calib. $\downarrow$
	(mm)	(mm)	(mm)		(mm)	(s)
No calibration	20.49	14.63	36.22	0.610	51.39	0.00
Ridge	24.71	19.21	45.50	0.573	57.54	1.49
Last-layer-Finetune	18.86	14.67	33.88	0.643	43.63	5.04
Full-Finetune	20.00	15.73	36.67	0.630	49.89	13.04
LoRA-Finetune	18.72	13.94	33.23	0.648	54.01	14.57

Table 2: Real-world mocap calibration results. Metrics are averaged over 3 seeds. Calib. s reports the total conditioning or adaptation time in seconds for whole datasets on one H100. Corresponding user-level std is provided in Appendix C.5 .

Method	Bottle Cap [-0.3ex]Opening		Earbud Pinch Grasping		Glue Three-Finger [-0.3ex]Grasping		Tissue Box Opening		Overall
	Success $\uparrow$	Time (s) $\downarrow$	Success $\uparrow$	Time (s) $\downarrow$	Success $\uparrow$	Time (s) $\downarrow$	Success $\uparrow$	Time (s) $\downarrow$	Success Rate $\uparrow$	Time (s) $\downarrow$
No calibration	9/10	35.6	0/10	–	5/10	7.0	0/10	–	35%	24.4
LoRA-Finetune	9/10	34.1	1/10	6.0	8/10	6.6	8/10	13.3	65%	17.9
YOCO (ours)	10/10	27.3	8/10	4.2	8/10	8.9	10/10	17.4	90%	14.9

Table 3: Downstream dexterous teleoperation results. We evaluate 4 tasks and count each task success over 10 trials and average completion time over successful trials. Failed trials are capped at one minute and excluded from task-specific average completion time, while each failed trial is penalized by assigning it the longest observed trial time in the overall average time. More stats are provided in Appendix C.8 .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	Real glove (AVP reference)		Augmented InterHand2.6M
Method	PA-MPJPE $\downarrow$	AUCJ $\uparrow$	PA-MPJPE $\downarrow$	AUCJ $\uparrow$
LoRA-Finetune	13.94	0.648	11.67	0.694
Set-Cond. Residual	13.74	0.645	12.66	0.664
Side-Tuning [ 6 ]	13.48	0.641	15.35	0.618
Meta-LoRA	13.51	0.633	13.94	0.639
Meta-LoRA+FT	14.17	0.664	11.41	0.702

Appendix

Table 4: Learned-prior controls and additional calibration baselines at

K=4

. PA-MPJPE is in millimeters; AUCJ is the normalized joint-PCK area.

Panel of Figure 5 — Figure 5: Preset pose library used to obtain deployable calibration targets. The four prompts are index pinch, fully open, claw / half close, and four fingers straight with the thumb near the palm.

Method	Timing Source	CPU calibration time (s)
YOCO (ours)	HyperNet Forward	13.16
Ridge	Closed Form Solver	0.37
Last-layer-Finetune	Gradient Descent	34.47
LoRA-Finetune	Gradient Descent	59.58
Full-Finetune	Gradient Descent	223.25

Appendix

Table 5: Average CPU calibration time required before real deployment.

Method	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	Tip-PA $\downarrow$	AUCJ $\uparrow$	Pinch $\downarrow$	Calib. s $\downarrow$
	(mm)	(mm)	(mm)		(mm)	(s)
No calibration	1.14	0.76	3.18	0.021	7.01	0.00
Ridge	4.32	2.95	9.12	0.053	12.53	0.68
Last-layer-Finetune	0.87	1.01	2.47	0.017	6.23	0.55
Full-Finetune	1.80	1.52	4.79	0.030	7.65	0.75
LoRA-Finetune	1.22	1.06	3.73	0.023	9.50	0.80

Appendix

Table 6: Per-user standard deviation of the real-world mocap calibration results.

Method	$K$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	Tip-PA $\downarrow$	AUCJ $\uparrow$	Pinch L2 $\downarrow$
		(mm)	(mm)	(mm)		(mm)
YOCO (ours)	1	16.07	10.94	24.40	0.694	38.04
	2	16.10	10.86	24.30	0.694	36.66
	4	15.64	10.45	23.29	0.702	34.90
	8	15.21	10.19	22.26	0.709	33.87
	16	15.10	10.14	21.94	0.712	33.70

Appendix

Table 7: Detailed K-sweep results on the augmented InterHand2.6M evaluation setting. All rows use the overall split aggregated over 3 seeds and 20 augmented units.

Method	Bottle Cap Opening	Earbud Pinch Grasping	3-Finger Glue Grasping	Tissue Box Opening
	Avg. Time (s) $\downarrow$	Avg. Time (s) $\downarrow$	Avg. Time (s) $\downarrow$	Avg. Time (s) $\downarrow$
No calibration	37.9	7.0	10.5	42.0
LoRA-Finetune	37.5	6.9	8.1	19.0
YOCO (ours)	27.3	4.8	9.9	17.4
Longest trial	59.0	7.0	14.0	42.0

Appendix

Table 8: Detailed downstream teleoperation completion-time statistics. Values are average completion time in seconds over 10 trials; lower is better. The longest-trial row reports the longest observed trial time for each task.

Component	Value
CPU	2 $\times$ Intel Xeon Platinum 8468V
RAM	2.0 TiB
GPU	1 $\times$ NVIDIA H100 80 GB HBM3

Appendix

Table 9: Device and compute platform used for training and timing experiments.

Setting	Value
Seed	42
Optimizer	Adam
Total steps	10,000
Batch size	1024
Peak learning Rate	3e-4
Final Learning Rate	1e-6

Appendix

Table 10: Training hyperparameters used by YOCO .

Explore similar work

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance
Jul 9, 2026Chenxi Wang, Ying Feng, Hongjie Fang +4Dexterous Manipulation Robot Teleoperation
1Noematrix · 2Shanghai Jiao Tong University · 3Shanghai Innovation Institute
Hydra: Marker-Free RGB-D Hand-Eye Calibration
Apr 29, 2025Martin Huber, Huanyu Tian, Christopher E. Mower +4Hand-Eye Calibration Robotics
School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK · Noah’s Ark Lab, Huawei, London, UK · independent contributor
OmniCalib: Target-Free, Task-Structured Self-Calibration for Humanoid Robots
Sep 17, 2026Kaixiang Lu, Haiyu Lan, Chunxiao Qiao +6Robotics Camera Calibration

Jul 9, 2026, cs.ROJ/K move · Enter open · S save

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance

Chenxi Wang, Ying Feng, Hongjie Fang, Shangning Xia+3

1Noematrix · 2Shanghai Jiao Tong University · 3Shanghai Innovation Institute

Dexterous Manipulation Robot Teleoperation Motion Retargeting

Open paper AskPDF arXiv

Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand-crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand-specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration-free retargeting method for intuitive dexterous teleoperation across human-like dexterous hands. AnyDexRT combines self-supervised fingertip correspondence learning with few-shot human guidance to anchor the mapping in task-relevant regions, and further refines pinch-related poses using a contact classifier. Experiments on diverse dexterous hands and real-world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods. Project website: https://chenxi-wang.github.io/projects/anydexrt

Abstract

Explore similar work

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance

Hydra: Marker-Free RGB-D Hand-Eye Calibration

OmniCalib: Target-Free, Task-Structured Self-Calibration for Humanoid Robots