Multi-camera rigs ship pre-calibrated from the manufacturer, but their calibration goes stale each time the rig is repositioned, and drifts continuously through wear, thermal cycling, and vibration. Recovering it on-the-fly is hard: classical bundle adjustment needs a fresh multi-view capture and a global optimization whose cost grows with the rig, ruling out real-time use. In this paper we present RECALL, a feed-forward method that recalibrates a movable multi-camera rig in a single pass. From the nominal OEM calibration as an anchor and a brief observation of a known object, it predicts each camera's current intrinsics and extrinsics, written as a shared global rotation of the rig composed with a small per-camera drift. Instead of regressing the calibration in one shot, RECALL refines its estimate with a learned, rig-coupled recurrent operator that reprojects through a differentiable projection and applies attention across the cameras at each step, and solves for the shared rotational gauge in-network with a learned Gauss-Newton step rather than regressing it. Because the ground truth is synthesizable, we train entirely on data generated on-the-fly and need no captured dataset. On a synthetic 10-camera benchmark, RECALL is more accurate than per-camera and rig-coupled joint bundle adjustment (0.26∘ vs 0.47∘ rotation, 5.4 vs 8.9 px focal) and runs about 105× faster at about 0.12 ms per rig. On real data (TotalCapture, CMU Panoptic) it matches or beats offline rig-coupled bundle adjustment on absolute and relative rotation while running four-to-five orders of magnitude faster, and on the Panoptic surround rig it also matches the strongest image-only foundation models (VGGT, VGGT-Ω, π3) on relative pose while staying metric and about 15× more accurate in focal length. Code: https://github.com/theICTlab/RECALL-multicam
Figures & tables
Figure 1: Method overview. (a) On-the-fly synthesis: from the OEM calibration we sample a bounded global rotation RG (global reorientation ≤θmax ) and small OEM-anchored per-camera drift ( ≤κ ), compose ground-truth cameras via Eq. 2 , and project the known object to noisy 2 D observations, regenerated every iteration with no captured dataset. (b) At test time the observations and OEM anchor pass once through RECALL , the rig-coupled recurrent operator ( Fig. 2 ), which predicts the full per-camera calibration; during training the differentiable projection supplies the reprojection term, reaching every parameter head.
Figure 2: The RECALL operator. A context encoder embeds each camera’s 2D-3D correspondences and OEM calibration, applying attention across the cameras to a context ci from which OEM-anchored heads predict an initial estimate θi(0) . It is refined for K steps: reproject through the differentiable projection, form a bounded residual, applying attention across the cameras (rig coupling), update a per-camera GRU, and apply a bounded update; in parallel a learned Gauss-Newton step solves for the shared rotational gauge RG ( Eq. 3 ) from the same residual. Every iterate is supervised; the final one is the output.
Method
Abs. rot ↓ ( \SIUnitSymbolDegree )
Rel. rot ↓ ( \SIUnitSymbolDegree )
Translation ↓ (units)
Focal ↓ (px)
Princ. pt. ↓ (px)
Dist. err ↓
Latency ↓ (ms)
Stale-OEM (do-nothing)
15.34
1.10
42.03
20.64
9.98
0.00019
≈0
Per-camera BA
2.30
4.72
8.41
503.88
39.43
751.49030
682.21
Rig-coupled joint BA
0.47
0.61
1.30
8.88
8.14
0.89187
16767
Method A (RANSAC+EPnP)
0.65
1.00
1.94
17.03
8.21
0.00016
5.26
Method B (RANSAC+EPnP)
0.65
1.00
1.94
12.91
9.98
0.00075
61.81
Method A (RePPnP)
0.59
0.83
1.64
17.03
8.21
0.00016
10.48
Table 1: Synthetic movable-rig recalibration (all entries synthetic ). Config: Ncamera=10 (O-shape), Nfid.=8 (cube), drift κ=0.1 , tilt 30∘ , 0.5 px noise; mean over 3 seeds (std in supp.); latency on an NVIDIA H100. Among recovering methods (stale row excluded), best per metric in bold, second best underlined; all metrics lower-is-better ( ↓ ).
Method
Input
Abs. rot ↓ ( \SIUnitSymbolDegree )
RRA@ 5∘↑
RTA@ 5∘↑
AUC@ 30∘↑
Focal ↓ (px)
Latency ↓ (ms)
VGGT [ 41 ]
images
39.59
26.7
25.8
43.3
292.9
3154
VGGT- Ω [ 42 ]
images
1.16
99.1
80.0
89.0
247.2
2017
π3 [ 44 ]
images
1.63
100.0
86.7
90.7
283.5
3196
Stale-OEM (do-nothing)
prior
0.72
100.0
98.2
96.5
18.78
≈0
Per-camera BA
prior+corr.
3.93
54.0
77.6
79.0
52.08
206
Rig-coupled joint BA ( 8% conv.)
prior+corr.
0.83
99.87
98.6
96.7
17.57
65087
Table 2: Real-data comparison on CMU Panoptic (drift-only; 10 -camera HD surround rig, κ=0.1 , 94.8% visibility). Image-only models (top) predict up-to-similarity; their absolute rotation and focal use an oracle Sim(3) lift (best case), while RRA/RTA/AUC are gauge-invariant. Prior-anchored methods (bottom) receive the OEM anchor and 3D structure and are scored on a common 100 drift draws (BA’s per-instance cost prohibits the full set); foundation models are over 5 frames. Joint BA is iteration-limited ( 8% converged at the 8000 -iteration budget). Best per metric in bold, second best underlined; arrows ( ↑ / ↓ ) give the better direction; RRA/RTA/AUC in %.
Method
abs. rot ( ∘ ) ↓
RRA@ 5∘↑
AUC@ 30∘↑
focal (px) ↓
lat. (ms) ↓
TotalCapture ( 8 -camera)
Stale-OEM
20.02
100.0
97.3
15.68
≈0
Per-camera BA
5.00
45.6
74.9
69.33
219
Joint BA ( 53% conv.)
1.79
89.2
90.8
13.77
33645
Method A (RANSAC+EPnP)
1.04
98.1
95.9
14.55
3
Method B (RANSAC+EPnP)
1.04
98.1
95.9
12.97
62
Table 3: Movable-rig recalibration on real data (global reorientation + drift). A bounded 20∘ global reorientation is composed onto the real structure, leaving the stale anchor ≈20∘ off. Joint BA is run to an 8000 -iteration budget (convergence fraction in parentheses); RECALL and BA are scored on a common set of 100 draws ( RECALL ’s full-test-set means in the suppl.). Best per metric in bold, second best underlined; arrows ( ↑ / ↓ ) give the better direction.
Extrinsic calibration of multi-camera systems with non-overlapping FOVs has been a challenging problem in the robotics literature. Conventional target-based methods impose substantial target setup overhead, either deploying large calibration targets or requiring pre-measured multi-target poses. Motion-based approaches instead suffer from drift error, scale ambiguity, and motion degeneracy. Securing both accuracy and usability, we propose a novel calibration method that leverages pure rotational motion, requiring only a single static calibration board. The key idea is to make all cameras sequentially observe the same target under a shared geometric reference, even without overlapping views. To integrate these time-separated observations, we formulate the problem using a latent turntable frame and a 3D error on SE(3) within a global optimization framework. We validate the proposed method on both a controlled camera rig and a full-scale vehicle platform with heterogeneous cameras, and analyze robustness under non-ideal turntable motion. Extensive experiments show that our approach maintains competitive accuracy without specialized precision hardware, proving its strong suitability for realistic on-site deployments. Our code is publicly available here.
Gwanhyeong Song, Chaehyeon Song, Ayoung Kim
Dept. of Mechanical Engineering, SNU, Seoul, S. Korea
Mobile robots and vehicles carry synchronized multi-camera rigs, yet many streaming 3D foundation models are designed for monocular input, leaving efficient use of rig geometry a challenge. We present StreamRig, a freeze-and-stream framework that builds causal streaming odometry for calibrated rigs on a frozen multi-view 3D foundation model. The frozen front-end jointly perceives the synchronized views using rig calibration. A Rig-Resampler compresses their features, a CausalBridge applies causal attention with a key-value cache, and a lightweight head regresses rig poses. A periodic re-anchoring protocol supports stable pose estimation over long sequences. Only these modules are trained, 74.6M parameters in total, with relative poses as the sole supervision. Our two-stage training strategy combines group relocalization pretraining with causal rig training to transfer the geometric priors of the frozen front-end and the alignment ability of the pretrained modules to streaming odometry. We evaluate on NCLT, TartanGround, KITTI-360, and our self-collected humanoid-robot dataset ZJH, where training uses only simulation and real-world evaluation is zero-shot. Across all four datasets, StreamRig achieves lower translation and rotation drift than the evaluated non-oracle monocular streaming and rig-aware offline models, while maintaining low inference cost. Ablations and controlled camera-count experiments identify the sources of these gains. We further examine how longer training windows affect inference over longer horizons. Code has been released at https://github.com/WeiYuFei0217/StreamRig.
Multi-camera systems are widely employed in sports to capture the 3D motion of athletes and equipment, yet calibrating their extrinsic parameters remains costly and labor-intensive. We introduce an efficient, tool-free method for multi-camera extrinsic calibration tailored to sports involving stick-like implements (e.g., golf clubs, bats, hockey sticks). Our approach jointly exploits two complementary cues from synchronized multi-camera videos: (i) human body keypoints with unknown metric scale and (ii) a rigid stick-like implement of known length. We formulate a three-stage optimization pipeline that refines camera extrinsics, reconstructs human and stick trajectories, and resolves global scale via the stick-length constraint. Our method achieves accurate extrinsic calibration without dedicated calibration tools. To benchmark this task, we present the first dataset for multi-camera self-calibration in stick-based sports, consisting of synthetic sequences across four sports categories with 3 to 10 cameras. Comprehensive experiments demonstrate that our method delivers SOTA performance, achieving low rotation and translation errors. Our project page: https://fandulu.github.io/sport_stick_multi_cam_calib/.