RECALL: Real-Time Recalibration of Multi-Camera Rigs via Learned OEM-Anchored Residuals
Organizations: Immersive and Creative Technologies (ICT) Lab, Concordia University Montreal, Canada
Abstract
Multi-camera rigs ship pre-calibrated from the manufacturer, but their calibration goes stale each time the rig is repositioned, and drifts continuously through wear, thermal cycling, and vibration. Recovering it on-the-fly is hard: classical bundle adjustment needs a fresh multi-view capture and a global optimization whose cost grows with the rig, ruling out real-time use. In this paper we present RECALL, a feed-forward method that recalibrates a movable multi-camera rig in a single pass. From the nominal OEM calibration as an anchor and a brief observation of a known object, it predicts each camera's current intrinsics and extrinsics, written as a shared global rotation of the rig composed with a small per-camera drift. Instead of regressing the calibration in one shot, RECALL refines its estimate with a learned, rig-coupled recurrent operator that reprojects through a differentiable projection and applies attention across the cameras at each step, and solves for the shared rotational gauge in-network with a learned Gauss-Newton step rather than regressing it. Because the ground truth is synthesizable, we train entirely on data generated on-the-fly and need no captured dataset. On a synthetic 10-camera benchmark, RECALL is more accurate than per-camera and rig-coupled joint bundle adjustment ( vs rotation, vs px focal) and runs about faster at about ms per rig. On real data (TotalCapture, CMU Panoptic) it matches or beats offline rig-coupled bundle adjustment on absolute and relative rotation while running four-to-five orders of magnitude faster, and on the Panoptic surround rig it also matches the strongest image-only foundation models (VGGT, VGGT-, ) on relative pose while staying metric and about more accurate in focal length. Code: https://github.com/theICTlab/RECALL-multicam
Figures & tables
| Method | Abs. rot ( ) | Rel. rot ( ) | Translation (units) | Focal (px) | Princ. pt. (px) | Dist. err | Latency (ms) |
|---|---|---|---|---|---|---|---|
| Stale-OEM (do-nothing) | 15.34 | 1.10 | 42.03 | 20.64 | 9.98 | 0.00019 | |
| Per-camera BA | 2.30 | 4.72 | 8.41 | 503.88 | 39.43 | 751.49030 | 682.21 |
| Rig-coupled joint BA | 0.47 | 0.61 | 1.30 | 8.88 | 8.14 | 0.89187 | 16767 |
| Method A (RANSAC+EPnP) | 0.65 | 1.00 | 1.94 | 17.03 | 8.21 | 0.00016 | 5.26 |
| Method B (RANSAC+EPnP) | 0.65 | 1.00 | 1.94 | 12.91 | 9.98 | 0.00075 | 61.81 |
| Method A (RePPnP) | 0.59 | 0.83 | 1.64 | 17.03 | 8.21 | 0.00016 | 10.48 |
| Method | Input | Abs. rot ( ) | RRA@ | RTA@ | AUC@ | Focal (px) | Latency (ms) |
| VGGT [ 41 ] | images | 39.59 | 26.7 | 25.8 | 43.3 | 292.9 | 3154 |
| VGGT- [ 42 ] | images | 1.16 | 99.1 | 80.0 | 89.0 | 247.2 | 2017 |
| [ 44 ] | images | 1.63 | 100.0 | 86.7 | 90.7 | 283.5 | 3196 |
| Stale-OEM (do-nothing) | prior | 0.72 | 100.0 | 98.2 | 96.5 | 18.78 | |
| Per-camera BA | prior+corr. | 3.93 | 54.0 | 77.6 | 79.0 | 52.08 | 206 |
| Rig-coupled joint BA ( conv.) | prior+corr. | 0.83 | 99.87 | 98.6 | 96.7 | 17.57 | 65087 |
| Method | abs. rot ( ∘ ) | RRA@ | AUC@ | focal (px) | lat. (ms) |
| TotalCapture ( -camera) | |||||
| Stale-OEM | 20.02 | 100.0 | 97.3 | 15.68 | |
| Per-camera BA | 5.00 | 45.6 | 74.9 | 69.33 | 219 |
| Joint BA ( conv.) | 1.79 | 89.2 | 90.8 | 13.77 | 33645 |
| Method A (RANSAC+EPnP) | 1.04 | 98.1 | 95.9 | 14.55 | 3 |
| Method B (RANSAC+EPnP) | 1.04 | 98.1 | 95.9 | 12.97 | 62 |
Camera Calibration
Camera calibration estimates the geometric and optical parameters that connect image measurements to real-world positions and other sensors.
See all 65 papers