Scale-Invariant Manipulability Shape Tracking Across Heterogeneous Manipulators

Geunwoo Kwon; Dong-gyu Lee; Kai Li; Soonwoong Hwang; Wansoo Kim

cs.ROSep 29, 2026

Scale-Invariant Manipulability Shape Tracking Across Heterogeneous Manipulators

Authors: Geunwoo Kwon, Dong-gyu Lee, Kai Li, Soonwoong Hwang, Wansoo Kim

Organizations: Hanyang University, Seoul, Republic of Korea · Hanyang University ERICA, Ansan, Republic of Korea

Abstract

When transferring manipulability across systems with different sizes and kinematic structures, matching absolute ellipsoid scale may be unnecessary when the goal is to reproduce orientation and semi-axis length ratios. Full-matrix tracking, however, penalizes both shape and absolute-scale differences, even when only shape matching is required. We therefore propose a scale-invariant manipulability shape-tracking method that treats matrices differing only by a positive scalar factor as equivalent and uses their unit-determinant representatives. We derive the differential of the unit-determinant shape representative and an orthonormal coordinate representation of the tangent tracking residual under the affine-invariant Riemannian metric (AIRM). The resulting scale-invariant objective is integrated with position and end-effector direction tasks in a constrained joint-velocity quadratic program. Simulations with four heterogeneous robots evaluate robot-to-robot and human-to-robot transfer. On three followers, the proposed method achieves endpoint shape distances of 9.30 x 10^-5 without scale tuning. With robot-specific target scales tuned during motion, the Full method retains endpoint axis-ratio errors of 0.19-0.31 on KR500 and UR20. For human reaching with concurrent tasks, the proposed method yields dual force shapes elongated along X like the human target on all four robots, with endpoint position errors of 2.4-5.6% of reference arm length versus up to 75% for the Full method tracking the original human ellipsoid.

Figures & tables

Fig. 1: Manipulability ellipsoid (ME) shape transfer from robot or human motion to heterogeneous robots. \scriptsize1⃝ Reference and current manipulability matrices are normalized to unit-determinant shapes, preserving ME orientation and all semi-axis length ratios. \scriptsize2⃝ The Shape method follows the reference shape trajectory without requiring absolute-scale matching. Blue dashed ellipses denote reference MEs; green, red, and orange solid ellipses denote the MEs of Gen3, KR500, and UR20, respectively.

Fig. 2: Manipulability transfer from FR3 to Gen3, KR500, and UR20. (a) Robot motions: FR3 source (blue frame) and followers using the Full (teal) and Shape (orange) methods. (b) Actual (red solid) and desired (blue dashed) XZ ellipses centered at the corresponding times, displayed as $\bm{M}$ for the Full method and $\widehat{\bm{M}}$ for the Shape method.

Fig. 3: Human-to-robot transfer of wrist position and manipulability shape from a measured right-arm reach. Four heterogeneous manipulators use the Shape method to track the transferred position and velocity manipulability shape with a fixed forward end-effector direction objective. Ellipsoids show the final human target (blue) and robot (red) dual force shapes.

Phase means
		Full			Shape
Robot	Phase [s]	$\overline{d}_{\mathrm{AI}}$	$\overline{d}_{s}$	$\overline{d}_{\rho}$	$\overline{d}_{\mathrm{AI}}$	$\overline{d}_{s}$	$\overline{d}_{\rho}$
Gen3	$[0,3)$	0.0348	0.0344	0.0055	0.0651	0.0344	0.0421
	$[3,8)$	0.2210	0.2210	0.0033	0.2234	0.2210	0.0176
	$[8,11]$	0.0409	0.0408	0.0021	0.0471	0.0408	0.0101
KR500	$[0,3)$	2.7018	0.8115	2.5136	3.0878	0.0243	3.0873

TABLE I: Tracking errors

Endpoint ( $t=11$ s)
Follower	Multiplier $c$	$\|r-r_{d}\|$	$\theta\,[\mathrm{deg}]$	$d_{s}$
Gen3	1.189	0.0002	0.0008	0.0001
	11.31	1.0687	0.0005	0.4352
	5.657	0.9562	0.0005	0.3826
KR500	1.189	4.5546	0.3019	1.0694
	11.31	0.1905	0.0004	0.0684

TABLE II: Target-scale sensitivity of the Full method

Phase means
		Full			Shape
Robot	Phase [s]	$\overline{e}_{p}$	$\overline{\theta}_{\mathrm{dir}}$	$\overline{d}_{s}$	$\overline{e}_{p}$	$\overline{\theta}_{\mathrm{dir}}$	$\overline{d}_{s}$
		$[\%L_{r}]$	$[\mathrm{deg}]$		$[\%L_{r}]$	$[\mathrm{deg}]$
FR3	$[0,5)$	13.251	17.154	0.640	6.607	12.332	0.595
	$[5,6.24)$	18.341	20.085	0.741	10.695	8.451	0.480
	$[6.24,9.24]$	8.123	25.649	0.423	5.507	3.420	0.312

TABLE III: Human-transfer errors

Explore similar work

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark
Aug 13, 2026Dairu Liu, Zekun Qi, Jiayu Zeng +11Human Motion Motion Imitation
1Nankai University · 3Galbot · 2Tsinghua University +3
ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control
Jun 2, 2026Yutong Liang, Quanquan Peng, Ri-Zhao Qiu +1Dexterous Manipulation Hand
University of California San Diego
AMBIT: Anticipatory Multimodal Body Recruitment for Bimanual Tracking on a Humanoid
Sep 27, 2026Hanlong Li, Sihan Tan, Takeshi Ashizawa +2Bimanual Manipulation Humanoid
Institute of Science Tokyo, Tokyo, Japan

Aug 13, 2026, cs.ROJ/K move · Enter open · S save

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Dairu Liu, Zekun Qi, Jiayu Zeng, Ruixi Yu+10

1Nankai University · 3Galbot · 2Tsinghua University+3

Human Motion Motion Imitation Motion Human-Annotated Benchmark

Open paper AskPDF arXiv

Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kinematic errors average per-frame pose differences but miss the physical artifacts that matter most, particularly unstable support and incorrect contacts such as foot skating and mistimed touch-downs. Meanwhile, widely used test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors. We introduce HumanTracker to make humanoid tracking evaluation both perceptually aligned and scalable. The HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis. We further propose HumanScore, a preference-aligned metric trained on 12K motion pairs containing 24K motions. Across representative state-of-the-art trackers, HumanScore better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss.

Abstract

Explore similar work

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control

AMBIT: Anticipatory Multimodal Body Recruitment for Bimanual Tracking on a Humanoid