Kilohertz-Safe: A Scalable Framework for Constrained Dexterous Retargeting

Yinxiao Tian; Ziyi Yang; Zinan Zhao; Zhen Kan

cs.ROMar 31, 2026

Kilohertz-Safe: A Scalable Framework for Constrained Dexterous Retargeting

Authors: Yinxiao Tian, Ziyi Yang, Zinan Zhao, Zhen Kan

Organizations: Department of Automation at the University of Science and Technology of China, Hefei, Anhui, China, 230026.

Abstract

Dexterous hand teleoperation requires motion re-targeting methods that simultaneously achieve high-frequency real-time performance and enforcement of heterogeneous kinematic and safety constraints. Existing nonlinear optimization-based approaches often incur prohibitive computational cost, limiting their applicability to kilohertz-level control, while learning-based methods typically lack formal safety guarantees. This paper proposes a scalable motion retargeting framework that reformulates the nonlinear retargeting problem into a convex quadratic program in joint differential space. Heterogeneous constraints, including kinematic limits and collision avoidance, are incorporated through systematic linearization, resulting in improved computational efficiency and numerical stability. Control barrier functions are further integrated to provide formal safety guarantees during the retargeting process. The proposed framework is validated through simulations and hardware experiments on the Wuji Hand platform, outperforming state-of-the-art methods such as Dex-Retargeting and GeoRT. The framework achieves high-frequency operation with an average latency of 9.05 ms, while over 95% of retargeted frames satisfy the safety criteria, effectively mitigating self-collision and penetration during complex manipulation tasks.

Figures & tables

Fig. 1: System pipeline of the proposed high-frequency retargeting framework. High-frequency human inputs and robot state feedback are unified through velocity-level constraint linearization, forming a structured convex optimization problem that enables stable real-time retargeting under geometric, kinematic, and safety constraints.

Method	Mean (ms) $\downarrow$	Std (ms) $\downarrow$	99%ile (ms) $\downarrow$	RT@100Hz (%) $\uparrow$
Ours	9.05	2.29	13.42	85.82
Dex-Retargeting	$15.59$	$12.50$	$32.82$	$34.41$
GeoRT	$34.49$	$4.28$	$49.90$	$0.19$

TABLE I: Latency Comparison

Fig. 2: Motion preservation comparison across retargeting pipelines. (a) Global motion error over the entire interaction sequence. Ours and Dex-Retargeting are plotted against the left vertical axis, whereas GeoRT is shown using the right vertical axis due to its distinct numerical scale. Dual axes are adopted solely for visualization clarity. (b) Cumulative motion preservation advantage of our method compared with Dex-Retargeting. (c–d) Zoomed-in views of critical interaction stages, highlighting fine-grained finger posture preservation. (e) Efficiency–fidelity trade-off: mean motion preservation, averaged over all frames in the interaction sequence, versus total computation time accumulated over the same sequence for Ours, Dex-Retargeting, and GeoRT. (f) Segment-wise comparison (50-frame bins) of mean motion preservation and mean computation time: bars for motion preservation, markers for latency.

Fig. 3: Qualitative results illustrating robust human-to-robot hand retargeting across diverse hand gestures.

Fig. 4: Analysis of Safety Score Across Different Retargeting Pipelines. Annotated stages are defined based on the human hand motion and are shared across all methods.

Fig. 5: CBF ablation study comparing the safety score and minimum inter-finger distance with and without CBF constraints. The CBF-constrained method maintains consistently higher safety levels, with only minor millimeter-scale threshold violations in a few frames due to discrete-time control and system latency.

Fig. 6: Comparison of safety evaluation results for hand grasping and inter-finger contact motions with and without safety constraints. (Left) Front and side views of a grasping motion, illustrating how safety constraints prevent inter-finger collisions and geometric interpenetration. (Right) Effects of increasing activation distance, where larger activation distances lead to earlier intervention and more pronounced collision avoidance.

Fig. 8: Qualitative comparison of collision risk in the inter-finger region during hand closing. Colored markers indicate the collision risk level at specific inter-finger locations (green: low, yellow: moderate, red: high). Our method maintains lower collision risk compared to Dex-Retargeting and the variant without safety constraints.

Motion Retargeting

Motion retargeting transfers movement across different bodies, skeletons, or robots while preserving its intent and physical feasibility.

See all 96 papers

Jul 9, 2026

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance

Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand-crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand-specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration-free retargeting method for intuitive dexterous teleoperation across human-like dexterous hands. AnyDexRT combines self-supervised fingertip correspondence learning with few-shot human guidance to anchor the mapping in task-relevant regions, and further refines pinch-related poses using a contact classifier. Experiments on diverse dexterous hands and real-world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods. Project website: https://chenxi-wang.github.io/projects/anydexrt

Chenxi Wang, Ying Feng, Hongjie Fang +4

1Noematrix · 2Shanghai Jiao Tong University · 3Shanghai Innovation Institute

Dexterous Manipulation Robot Teleoperation Motion Retargeting

Ask PDF

Jul 8, 2026

Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting

Advances in learning-based robotic manipulation, such as Vision-Language-Action (VLA) models and Video Action Models (VAMs), heavily rely on high-quality teleoperation data. Their capabilities are strictly upper-bounded by the quality of the underlying human demonstrations. Current gradient-based retargeting algorithms often converge to different local minima, resulting in jitter that affects data quality and teleoperation experience. To address this, we introduce the Sampling-Based Retargeter (SBR), a novel gradient-free retargeting method drawn from the rich literature of sampling-based control and explicitly designed for low-jitter, real-time kinematic retargeting. We evaluate SBR both in simulation and through a rigorous real-world user study involving 18 participants performing 3 complex manipulation tasks. Compared to gradient-based baselines, SBR achieved the highest overall task success rate (54.1%) while significantly reducing operator cognitive fatigue, recording the lowest NASA-TLX workload score (36.4 out of 100). Ultimately, we establish SBR as a highly effective, intuitive retargeter for dexterous manipulation, providing the community with a rigorous benchmarking methodology to guide future retargeting research.

Robert Jomar Malate, Erik Bauer, Norica Bacuieti +4

ETH Zurich, Switzerland · mimic robotics, Switzerland · Stanford University, United States

Robot Teleoperation Motion Retargeting Robotic Manipulation

Ask PDF

Jul 4, 2026

ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

Learning robot dexterous manipulation from human manipulation videos requires reliably retargeting human intent to executable robot actions while maintaining stable hand-object contact, which remains a key challenge in embodied intelligence. Existing retargeting methods often ignore explicit contact modeling or rely on reinforcement learning, resulting in limited accuracy and generalization. To address this, we propose ObjRetarget, a human-to-robot motion retargeting framework for learning robot dexterous manipulation from human videos, which integrates anthropomorphic arm trajectory constraints with structured hand-object geometric modeling. For arm motion, reference trajectories extracted from human videos are used for initialization, followed by anthropomorphic constraints and redundancy-aware optimization to generate natural and accurate movements. For hand manipulation, ObjRetarget represents multi-finger contacts using polytope clusters and preserves contact structure through geometric invariants to improve stability. Experiments on real robots show that ObjRetarget improves manipulation success rates and contact stability across multiple dexterous tasks, and generalizes well to different demonstrations, object poses, and task settings.

Yuanchuan Lai, Qing Gao, Ziyan Liang +2

School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China. · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, Shenzhen 518172, China. · School of Computing, University of Portsmouth, Portsmouth PO1 3HE, UK.

Contact-Rich Robotic Manipulation Motion Retargeting Robotic Manipulation Robot Imitation Learning

Ask PDF

Abstract

Motion Retargeting

Explore similar work

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance

Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting

ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling