cs.CVOct 28, 2025
SaveKineo: Calibration-Free Metric Motion Capture From Sparse RGB Cameras
Organizations: École Centrale de Lyon, CNRS, INSA Lyon, Université Claude Bernard Lyon 1, Université Lumière Lyon 2, LIRIS, UMR5205, ENISE, Saint-Etienne, France
Abstract
Markerless multi-view motion capture is often constrained by the need for precise camera calibration, limiting accessibility for non-experts and in-the-wild captures. Existing calibration-free approaches mitigate this requirement but suffer from high computational cost and reduced reconstruction accuracy. We present Kineo, a fully automatic, calibration-free pipeline for markerless motion capture from videos captured by unsynchronized, uncalibrated, consumer-grade RGB cameras. Kineo leverages 2D keypoints from off-the-shelf detectors to simultaneously calibrate cameras, including Brown-Conrady distortion coefficients, and reconstruct 3D keypoints and dense scene pointmaps at metric scale. A confidence-aware spatio-temporal keypoint subsampling strategy, combined with graph-based global optimization, enables robust calibration at a fixed computational cost independent of sequence length. We further introduce a pairwise reprojection consistency score, a heuristic measure of the agreement between views, which we use to weight bone lengths in metric scale recovery. Evaluations on EgoHumans and Human3.6M demonstrate substantial improvements over prior calibration-free methods. Compared to previous state-of-the-art approaches, Kineo reduces camera translation error by approximately 83-85%, camera angular error by 86-92%, and world mean-per-joint position error (W-MPJPE) by 83-91%. Kineo is also efficient in real-world scenarios, processing multi-view sequences faster than their duration in a specific configuration (e.g., 36 min to process 1 h 23 min of footage). The full pipeline and evaluation code are openly released to promote reproducibility and practical adoption at: https://liris-xr.github.io/kineo/
Figures & tables
Figure 1. Overview of the Kineo pipeline for markerless motion capture from uncalibrated and unsynchronized multi-camera videos. Starting from raw video inputs, the system first performs audio-based temporal synchronization to align unsynchronized streams on a shared timeline. Next, automatic camera calibration estimates extrinsic and intrinsic parameters, including Brown-Conrady lens distortion, via a graph-based optimization over 2D keypoint correspondences selected by a confidence-aware subsampling strategy. Using the recovered cameras, 3D keypoints and scene pointmaps are reconstructed, and each triangulated keypoint is assigned a pairwise reprojection consistency score to measure the agreement between views. Finally, metric-scale recovery is achieved either through a human body prior using the SMPL model or a monocular metric depth estimator for subject-agnostic scaling. The modular design enables robust and scalable reconstruction of human subjects across long, multi-view sequences, and can extend to other subjects given a suitable 2D keypoint detector.
Figure 2. Example subsampling for a pair of views from Human3.6M. Colors indicate the pair confidence score , solid lines represent selected correspondences, and dashed lines represent correspondences that were not selected. Only correspondences whose pair score exceeds can be selected, which discards the keypoints the 2D keypoint detector does not locate confidently (e.g., occluded or poorly localized keypoints).
Figure 3. Example of camera loop closure with incorrect (left) and correct (right) relative scale factors. When the relative scales are incorrect, traversing the cycle does not return to the original camera position, resulting in . In contrast, when the correct scales are used, the composition of the relative transformations yields the identity, and the loop closes consistently with .
Figure 4. Comparison of scene pointmaps produced by two geometry estimation models. (a) shows the pointmap generated with MoGe-2, which uses single-view images and camera intrinsics to produce metric pointmaps. (b) shows the pointmap generated with VGGT, which leverages multiple views to predict a globally consistent pointmap and camera poses, aligned to our camera system. VGGT provides more spatially consistent pointmaps across views, while MoGe-2 relies on single-view depth estimation.
Figure 5. Examples of different output formats with different 2D keypoint detectors used by Kineo. (a–c) show keypoint-based representations in various formats, while (d) illustrates a full parametric SMPL body reconstruction. For the SMPL model, NLF surface keypoints are first detected and subsequently fitted using the SMPLFitter module from NLF.
Figure 6. Visualization of the proposed pairwise reprojection consistency score on 3D human skeleton reconstructions from the EgoHumans dataset, obtained using Kineo. The low score on the hands shows that it flags the joints on which the views disagree. Beyond the bone weighting of Sec. 7.1 , it could be leveraged for downstream filtering, which we leave for future work.
Figure 7. Overview of SMPL-based global scaling. The SMPL body model serves as a prior for metric scale, providing plausible human body proportions that guide the estimation of the global scale factor , which converts the scene from arbitrary units to metric units. Given an estimated skeleton in arbitrary units, we optimize while updating the SMPL shape parameters so that the SMPL skeleton closely matches the bone lengths of the scaled input skeleton, enforced by while remaining close to the mean shape, enforced by . The resulting optimized scale is then applied to rescale the entire scene, including the skeletons, cameras, and scene pointmaps, producing a metric-scale representation.
Figure 8. Overview of metric depth-based global scaling. Unlike SMPL-based scaling, which relies on a parametric human body model, metric depth-based scaling is subject-agnostic, as the metric depth map predicted directly from the RGB image by MoGe-2 ( Wang et al., 2025b ) is independent of the subjects present in the scene. The global scale factor is computed as the ratio between the joint depths in metric units obtained from the depth map and the corresponding joint depths in arbitrary units. The resulting scale is then applied to rescale the entire scene, including the skeletons, cameras, and scene pointmaps, producing a metric-scale representation.
Figure 9. Automatic calibration runtimes for different keypoint detectors across sequences of varying durations. The comparison includes three keypoint sources: NLF (batch size 1, full precision), RTMPose (batch size 1, full precision), and RTMPose (batch size 32, half precision). Sequence sizes are small (S, 7 min 10 s total across 4 views), medium (M, 43 min 03 s total across 4 views), and long (L, 1 h 23 min total across 4 views). Each stacked bar breaks down the runtime by processing stage: Intrinsics Initialization (yellow), Extrinsics Initialization (orange), Bundle Adjustment (pink), Global Scaling (purple), Keypoints Pairs Sampling (blue), and Keypoints Detection (gray). Total runtime is indicated above each bar. RTMPose (B32 Half) consistently achieves the fastest processing times, completing all stages in less time than the total duration of the original video sequences. Importantly, the calibration runtime scales efficiently, showing only moderate increases as sequence duration grows.
Figure 10. Estimated distortion on three EgoHumans ( Khirodkar et al., 2023 ) views. (a) Input frame with a grid distorted by the ground-truth camera. (b) Frame and grid undistorted with the estimated camera (red), overlaid on the ground-truth grid (black). (c) Error of the estimated distortion after focal length rescaling (Sec. 8.2.3 ), with arrows showing its direction and pink contours outlining the observed region.
Figure 11. Mean PA-MPJPE (in millimeters) against the number of views on the EgoHumans ( Khirodkar et al., 2023 ) sequences, with estimated and ground-truth intrinsics. Bands give the 95% confidence interval of the mean over sequences. Camera sets are nested from two to eight views; All uses every static camera of the sequence (8 to 19).
Figure 12. Qualitative comparison between HSfM ( red ) and Kineo ( blue ) on their respective best-performing sequences from Human3.6M. Ground-truth cameras and skeletons are shown in black . Visualization is scale-aligned.
Figure 13. Qualitative results on sequences from the EgoHumans dataset. Kineo estimations are visualized in blue , while ground-truth cameras and skeletons are shown in black . The scene pointmap is predicted as a downstream task of our method using MoGe-2 ( Wang et al., 2025b ) .
Figure 14. Examples of craftsmen recorded at an experimental archaeology worksite. Both sequences were captured using six uncalibrated and unsynchronized GoPro Hero11 Black cameras, with no calibration procedure necessary.
| Symbol | Description |
|---|---|
| Temporal synchronization (Sec. 4 ) | |
| Temporal lag of camera , relative to a reference camera | |
| Ground-truth temporal lag of camera (AIST++) | |
| Hop length used during the Short-Time Fourier Transform | |
| , | Audio sample rate and video frame rate |
| , | Distance from each camera to the sound source |
Table S1. Notation used in the main paper.
| Parameter | Value |
|---|---|
| Temporal synchronization (Sec. 4 ) | |
| Reference camera | First camera; the other audio tracks are resampled to its sample rate |
| Audio features | 13 MFCCs over 128 HTK mel bands (log-mel in dB), 2048-point FFT, 40 ms Hann window, 5 ms hop ( at 48 kHz) |
| Feature normalization | Each coefficient is standardized to zero mean and unit variance |
| Lag estimation | Maximum of the full cross-correlation, computed by FFT, without interpolation |
| 2D keypoints | |
Table S2. Implementation details and hyperparameters of each stage of the pipeline, shared by all experiments unless stated otherwise.
| Experiment | 2D keypoints | ||||
| Tables 3 , 4 , 1 , 6 and 7 , Sec. 8.2.3 , Figs. 9 and 11 ; Tables S6 , S8 and S9 , Fig. S1 | NLF | 0.6 | 3000 | 0.6 | 500 |
| Tables 11 , 6 , 8 and 9 , Fig. 9 ; Table S7 | RTMPose | 0.5 | 4000 | 0.5 | 400 |
| Table 6 ; Table S8 | DWPose | 0.5 | 4000 | 0.5 | 400 |
| Tables 5 and 6 , Human3.6M | Ground truth | 0.4 | 4000 | 0.4 | 200 |
| Tables 5 and 6 , EgoHumans | Ground truth | 0.4 | 4000 | 0.5 | 200 |
Table S3 . Confidence thresholds and sampling budgets used in the experiments of the main paper and of these supplementary materials: threshold and budget of the pairwise calibration (Sec. 5.2 ), threshold and budget of bundle adjustment (Sec. 5.6 ). Experiments that take the cameras from the ground truth do not run calibration and are omitted. Ground-truth keypoints all have a confidence of 1, so that their thresholds only discard missing or non-interpolated keypoints.
Figure S1 . Estimated distortion on EgoHumans, one view per scene, with the same panels as Fig. 10 of the main paper. The error in (c) is shown on a linear color scale saturated at 50 px, with its direction as unit arrows (dots below 1 px); hatched samples are excluded (see Protocol). Each row shows a representative sequence of its scene from the camera whose keypoints cover the largest share of the image, among the cameras of the rig’s majority focal length.
Figure S2 . Correspondences selected by (a) farthest-point sampling, (b) confidence-only sampling (Top-P) and (c) our random spatio-temporal subsampling ( ) from the same confidence-filtered candidates, for the camera pair of an EgoHumans tennis sequence where the coverage of random sampling falls furthest below FPS. Whereas the convex hull of the farthest-point selection (solid) nearly matches that of the candidates (dashed), random sampling leaves the edges of the candidate region uncovered (shaded), and confidence-only sampling covers only its center.
Figure S3 . Thumbnails of all supplementary video files provided alongside this document. Each thumbnail represents a single frame from the corresponding video; full videos are provided separately.
Figure S4 . Illustration of the procedure to visualize the interactive visualization file for the S11_Greeting sequence from Human3.6M. ( a ) Opening the .rrd file using the online rerun viewer available at https://rerun.io/viewer . ( b ) Interactive 3D visualization, showing the scene hierarchy with ground-truth and Kineo estimates.
More in this topic
Markerless Motion Capture
Markerless motion capture estimates 3D body movement from video or other sensors without attached tracking markers.
See all 70 papers