Lollypop: Camera-to-Motion-Capture Calibration Verification with a Reference Target

Tianyi Liu; Kevin Harris; Mihika Dave; Kun He

cs.CVOct 3, 2026

Lollypop: Camera-to-Motion-Capture Calibration Verification with a Reference Target

Authors: Tianyi Liu, Kevin Harris, Mihika Dave, Kun He

Organizations: Meta Redmond, WA, USA

Abstract

Camera-to-motion-capture (mocap) calibration is essential for using mocap as ground truth in robotics, AR/VR, and other computer vision tasks. However, the calibration can drift after deployment, while calibration residuals and visual inspection provide limited independent verification. We present Lollypop, a fiducial-mocap reference target for independent calibration verification. The target couples an ArUco fiducial with a mocap marker constellation so the visual center and tracked centroid represent the same physical point. Given a candidate calibration, verification projects the mocap point into the image and measures its disagreement with the detected fiducial center. Experiments show sub-pixel nominal error, sensitivity to controlled extrinsic perturbations, and increasing error during an illustrative mixed-handling sequence.

Figures & tables

Fig. 1: System overview of Lollypop calibration verification. The target creates a shared physical reference point between the camera image and the motion-capture system; verification compares the detected fiducial center with the mocap centroid projected through the candidate camera-to-mocap calibration.

Fig. 2: The Lollypop reference target. An ArUco fiducial and six retroreflective markers, with the marker centroid aligned to the fiducial center.

Pixel error $e_{2D}$ (px)
Camera	RMSE	Mean	Med.	Std	p95	Max
Front Left	0.48	0.36	0.25	0.32	1.12	1.33
Front Right	0.62	0.47	0.36	0.41	1.55	1.86
Side Left	0.97	0.88	0.80	0.42	1.48	1.62
Side Right	1.01	0.93	0.99	0.39	1.50	1.74
Combined	0.80	0.65	0.53	0.46	1.47	1.86

TABLE I: Per-camera Lollypop pixel error

e_{2D}

for the nominal calibration.

Perturbation	RMSE	Mean	Med.	Std	p95	Max
Baseline (none)	0.80	0.65	0.53	0.46	1.47	1.86
$\Delta t=1$ mm	1.43	1.29	1.19	0.62	2.43	3.41
$\Delta t=5$ mm	5.63	5.28	5.53	1.95	8.06	10.48
$\Delta\theta=0.2$ °	1.50	1.44	1.45	0.40	2.11	2.63
$\Delta\theta=1.0$ °	6.46	6.12	6.63	2.06	8.47	8.76

TABLE II: Lollypop pixel error

e_{2D}

(px) under controlled errors injected into the candidate extrinsic

T^{\mathrm{c}}_{\mathrm{rb}}

Fig. 3: Verification error during an illustrative mixed-handling sequence involving headset repositioning and occasional marker contact.

Camera Calibration

Camera calibration estimates the geometric and optical parameters that connect image measurements to real-world positions and other sensors.

See all 57 papers

Robust Camera-to-Mocap Calibration and Verification for Large-Scale Multi-Camera Data Capture
Apr 23, 2026Tianyi Liu, Christopher Twigg, Patrick Grady +3Multi-View Geometry Model Calibration
Meta
VeriCalib: Verifiable Target-Based LiDAR-Camera Calibration
Oct 20, 2026Xiangcheng Hu, Xieyuanli Chen, Feiyi Chen +8Lidar-Camera Extrinsic Calibration Camera Calibration
Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · X Square Robot Technology (Shenzhen) Co., Ltd., Shenzhen, China · College of Intelligence Science and Technology at National University of Defense Technology, Changsha, Hunan, China +4
Laser-Tracker-Assisted Camera-to-Robot Calibration for Mobile Robots
Sep 22, 2026Jan A. Rudolph, Öykü Kandemir, Markus UlrichMobile Robotics Hand-Eye Calibration
Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT), Englerstr. 7, 76131 Karlsruhe, Germany

Apr 23, 2026J/K move · Enter open · S save

Robust Camera-to-Mocap Calibration and Verification for Large-Scale Multi-Camera Data Capture

Tianyi Liu, Christopher Twigg, Patrick Grady, Kevin Harris+2

Meta

Multi-View Geometry Model Calibration Camera Calibration

Open paper AskPDF arXiv

Optical motion capture (mocap) systems are widely used for ground-truth capture in AR/VR, SLAM and robotics datasets. These datasets require extrinsic calibration to align mocap coordinates to external camera frames -- a step that is subject to multiple sources of error in practice, and failures often go undetected until they corrupt downstream data. These issues are compounded for fisheye cameras, where spatially non-uniform distortion makes both calibration and verification more challenging. We present a calibration and verification system designed for this setting. Concretely, we target robustness to board-to-marker attachment variation, optimization initialization ambiguity, and session-to-session calibration drift after deployment. The calibration jointly estimates camera extrinsics and the board-to-marker transform, and uses a staged solver to improve convergence reliability under ambiguous initialization. The verification component, \lollypop, provides fast, operator-independent assessment through a measurement chain entirely independent of the calibration data. In experiments on a Meta Quest 3 headset with fisheye cameras, our calibration outperforms existing benchwork, and lollypop reliably detects calibration degradation over time. The system has been deployed in production data collection pipelines.

Abstract

Camera Calibration

Explore similar work

Robust Camera-to-Mocap Calibration and Verification for Large-Scale Multi-Camera Data Capture

VeriCalib: Verifiable Target-Based LiDAR-Camera Calibration

Laser-Tracker-Assisted Camera-to-Robot Calibration for Mobile Robots