Authors: Xiangcheng Hu, Xieyuanli Chen, Feiyi Chen, Shengli Ren, Shiyang Chen, Guoqiang Ye, Hao Wang, Ping Tan, Steven L. Waslander, Fuchun Sun, Jin Wu
Organizations: Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · X Square Robot Technology (Shenzhen) Co., Ltd., Shenzhen, China · College of Intelligence Science and Technology at National University of Defense Technology, Changsha, Hunan, China · Independent Researcher · University of Toronto Institute for Aerospace Studies and the University of Toronto Robotics Institute, Toronto, Canada · Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Intelligent Science and Technology, University of Science and Technology Beijing, Beijing, China
Target-based extrinsic calibration estimates the rigid transform between sensors from a few observations of a known target, and its result is commonly assessed by the root-mean-square error of the fitted residuals. However, this scalar error is computed on the same observations and therefore cannot indicate whether to accept the result, re-extract, recollect or add observations. This paper presents VeriCalib, which formulates extrinsic calibration as a verifiable estimation problem and reports a calibration state. The key insight is a two-level extrinsic compensation mechanism, in which the extrinsic compensates the rigid part of each observation error at the residual level, and rotation and translation compensate each other at the parameter level. Consequently, the fitted residual can reveal neither an inconsistency between the observations and the model nor whether the estimated extrinsic is reliable in every direction. Accordingly, we assess the calibration through a structured covariance that jointly propagates the uncertainty of the camera and LiDAR board features with a known-geometry prior, and estimate its scale as a variance component from the residuals. On this covariance, we decouple rotation and translation to resolve the precision along each direction and guide data collection toward the weakest one. Since this precision presumes consistent observations, we further develop within-view and cross-view checks that examine the internal residual and the rigid discrepancy between observations, respectively, with thresholds that hold their significance level under an unknown noise level. We also extend the minimal detectable bias to the extrinsic to bound the smallest detectable fault along each direction. These criteria form a calibration state that indicates whether to accept the result or how to collect the data...
Figures & tables
Figure 1: Head-pair calibration from 14 observations, two with hole centers misplaced by automatic rim extraction ( C : true rim and center in black and white, extraction in red). A : LiDAR scans projected with the extrinsics of FAST-Calib and VeriCalib (tags: translation deviation from the reference). The RMSE of 34.0 mm neither reveals nor locates the errors, whereas VeriCalib removes both and one further observation ( B : accepted in teal, removed in red) and reports the state, validation-limited (V), with its translation precision σt .
Figure 2: VeriCalib overview on the head pair. Left : observations with projected LiDAR points (blue; red box: head camera; cyan boxes: chassis LiDAR and camera) and their fitted RMSE. Center : structured factor Fk of one observation with the 1σ hole-center residual covariance (navy ellipses), the LiDAR boundary points (blue dots), and the absorbed (green) and internal (red) residual parts; the boxes derive the scale s^ , the directional information Λ , and the MDB, which depend on one another. Right : calibration state (X/P/V/OK: inconsistent, precision-limited, validation-limited, verified), accepted (navy), p<0.05 (salmon), and removed (red) board poses, detection-limit ellipse with its weakest direction vmax (green), and recommended next pose (green dashed).
[][]
Uncertainty modeling
Constraint analysis
Result assessment
[][]
Fault test
[][]
Method
Venue
LiDAR
Camera
Scale
Analysis
Guidance
Metric
Within
Cross
Detect. limit
Zhou et al. (2018a)
IROS
×
×
×
Coupled
Empirical
Ref.
✓
×
×
Domhof et al. (2019)
ICRA
Indep. †
Indep. †
Est. †
×
×
RMSE
✓
×
×
velo2cam ( Beltrán et al., 2022 )
T-ITS
×
×
×
×
Empirical
Repeat.
✓
×
×
LCE-Calib ( Jiao et al., 2023 )
T-MECH
×
Indep.
×
×
×
Ref.
✓
×
×
Table 1: Comparison of extrinsic calibration methods.
Figure 3: Two-level extrinsic compensation ( A – C : principle; D – F : chassis-pair data). A , D : a hole-center error split into absorbed and internal parts. B , E : a pitch and the compensating translation yield nearly the same target observation. C , F : rotation precision with the translation fixed ( σf ) and marginalized ( σR ); C: σ=5 mm, b=0.2 m.
Figure 4: Shared-pose camera covariance. Left : 3σ hole displacements and correlations of the shared-pose (Joint) and independent-hole (Indep.) models. Right : hole centers of PnP solutions from noise-perturbed marker corners on a head-pair observation ( ×60 ), with 3σ ellipses of each hole and of their centroid (dashed: Indep.).
Figure 5: Short-arc LiDAR center covariance ( 3σ ) of the free-radius fit (Free) and of the radius prior with Neff points (Prior); Rim: boundary points. Left : synthetic ±25∘ arc. Right : partial-arc hole of a head-pair observation ( ×12 ).
Figure 6: Covariance shape and scale. Left : nominal (dashed) and s^ -scaled (solid) covariances of the joint and independent-hole models against the ground truth (GT). Right : ECDF of NISk/dk before (Nom.) and after ( /s^ ) scale estimation (solid: head; dashed: chassis).
Figure 7: Verification pipeline ( Algorithm 1 ).
Figure 8: Consistency tests. Left : null and fitted NIS densities with threshold τ , and rigid and local faults. Right : within-view and cross-view statistics of the head (circles) and chassis (triangles) observations, normalized by their thresholds.
Figure 9: Observation diagnosis in the influence-inconsistency plane. Left : removal criterion against leave-one-out (LOO) selection. Right : head (circles) and chassis (triangles); blue: accepted; salmon: p<0.05 ; red: removed; X: inconsistent.
Figure 10: Directional detection limits. Left : two-dimensional section (shaded: validation-limited). Right : translation detection limits of the head and chassis pairs along the principal axes, with the 1σ precision core and weakest direction.
Figure 11: Platforms and targets (navy: cameras; red: LiDARs). A – C : mobile manipulator with the head and chassis cameras. D , E : four-hole target and checkerboard. F : handheld platform with stereo cameras and an Ouster LiDAR. G : FS platform with a flash LiDAR.
Camera 1
Camera 2
Platform
Set
LiDAR
Target
FOV [ ∘ ]
K
FOV [ ∘ ]
K
Robot-A
S1
Mid-360
Four-hole
83×53
14
107×91
14
S2
Four-hole
21
26
Robot-B
S3
Mid-360
Checkerboard
83×53
37
107×91
20
S4
Four-hole
94
101
Handheld
S5
Ouster
Four-hole
80×70
147
80×70
92
Table 2: Sensors and observation counts of the real-world sets.
Rot. [ ∘ ] ↓
Trans. [mm] ↓
VR
Set
∣S∣
Ours-U
Indep.
Joint
Ours-U
Indep.
Joint
Indep.
Joint
S1-H
14
0.237
0.217
0.211
5.5
4.6
3.7
2.03
1.47
S1-C
13
0.444
0.342
0.297
12.0
11.1
8.5
2.09
1.37
S2-H
18
0.141
0.136
0.133
6.1
5.9
5.9
1.51
1.41
S2-C
23
0.371
0.138
0.126
16.7
5.7
5.1
1.24
0.96
S4-H
79
0.080
0.093
0.085
2.7
3.6
2.8
2.03
1.57
Table 3: Split-half repeatability of the extrinsic under three weightings on the real-world sets.
Figure 12: Real-data covariance and acquisition. A : variance ratio of the joint and independent-hole (Indep.) camera covariances (dashed: unity). B , C : worst-direction σt of six orderings from one initial pair on the head and chassis sets (dotted: 8 mm target; markers: first K reaching it).
Ours-U
Ours-W
PnP noise
eR [ ∘ ]
et [mm]
eR [ ∘ ]
et [mm]
Constant
0.111
5.60
0.103
5.09
Depth-dependent
0.165
7.36
0.115
4.97
Depth-dep. + offset (all)
0.216
10.32
0.189
9.02
Depth-dep. + offset (near)
0.187
9.27
0.177
8.75
Table 4: Weighting efficiency on synthetic batches.
Generating model
s^
Rej. [%]
σt [mm]
et [mm]
Nominal
1.00
8
4.4
5.3
Camera noise ×2
3.21
7
7.9
10.1
LiDAR noise ×2
1.80
8
5.9
6.5
Range-dependent shape
2.92
14
7.5
7.5
Common-mode bias
0.99
6
4.4
9.3
Table 5: Covariance-model perturbations on synthetic batches.
Figure 13: Contribution of each observation to the head-pair extrinsic, with the observations sorted by depth. A : board images shaded by the resulting difference Δσt of the worst-direction translation precision (salmon frames: accepted with p<0.05 ; red frames: removed). B : Δσt when the observation is removed from the accepted-set estimate, or added if excluded. C : resulting translation shift ∥Δt∥ .
Set
Ours
D/A/E
Rand.
Far
(ϵR,ϵt)=(0.15∘,8mm)
Synthetic
4
4
6
7
S1+S2 H, C
3 – 4
5 – 6
5 – 12
1.5× full-set precision
S4 H
24
23 – 29
41.5
72
S4 C
22.5
47.5
79
Table 6: Median number of observations ↓ of the selection rules on synthetic and real-world sets.
Detected [%]
Falsely removed [%]
Rej. [%]
Faulty
Fault
LOO
RANSAC
Huber
Ours
LOO
RANSAC
Huber
Ours
Ours
Noise at the nominal magnitude
0/8
None
–
–
–
–
12.5
2.2
2.8
1.7
10.0
1/8
Offset 30 mm
62.3
86.0
96.0
91.2
5.4
5.4
4.6
1.5
12.8
1/8
Hole swap
11.0
100.0
100.0
100.0
12.7
2.2
3.0
1.9
10.5
3/8
Indep. 30 mm
27.7
83.3
94.3
50.4
3.4
13.3
6.2
1.1
66.8
Table 7: Outlier removal on synthetic batches.
Figure 14: Rotation (left) and translation (right) deviations of head S1 extrinsics from the reference (dashed): fault-free single-observation solutions (grey), the two corrupted observations (red crosses), and the joint solutions.
Figure 15: Detection limit and calibration state on fault-free synthetic batches. A : detection rate of the focused test against a translation bias (dots: empirical; curve: predicted; dashed: 80% power). B – D : σt , MDBR , and MDBt against K (dashed: requirements). E : hold-out against random-split MDBt . Boxes: quartiles; whiskers: 5 th to 95 th percentiles; logarithmic axes in C–E.
Method
Fault-free
Indep.
Shared
Head S1
Ours-U
4.84
9.56
14.09
13.7
Ours-W
4.48
8.85
13.41
16.2
Ours-WC
4.48
6.93
13.64
0.37
Ours (full)
4.52
5.85
13.18
0.32
Diagnosis
Retained, cons.
16
13
13
12
Table 8: Translation error [mm] ↓ of the cumulative ablation.
Figure 16: Observation decisions on the combined robot-platform batches. Rows: joint extrinsic of each method; columns: an accepted observation, observations removed by the within-view and the cross-view check, and a far three-marker board explained by its covariance. Cyan: projected rim points; white and blue: camera and LiDAR hole centers; labels: held-out alignment error; bottom: 3σ residual ellipses ( ×5 ) and smallest p -value; frames: teal, accepted; salmon, p<0.05 ; red, removed.
Stage
K=5
K=14
K=35
Estimation
FAST-Calib
22.0
62.5
167.9
Ours
28.7
81.6
215.8
Verification (Ours)
Information
1.1
2.6
6.0
Diagnosis
41.9
279.8
1663.5
Table 9: Runtime [ms] on the head batch.
Figure 17: Cross-session analysis. Left : rotation difference between S1 and S2 on all observations and without gross errors (Trimmed: SODs above 100 mm removed; grey: inter-unit range). Right : residual range trend with level- α thresholds.
Figure 18: Checkerboard target ( A – F : head pair; G , H : both pairs). A , B : camera and LiDAR factors. C , D : an accepted (green) and a rejected (red) board with residual ellipses and p -values. E : eigenvalues of the residual covariance blocks. F : board consistency against depth. G : detection rate of an injected board bias. H : acquisition (dotted: precision target).
Precision
MDB
Deviation
Split
∣S∣/∣V∣
[ ∘ ]
[mm]
[ ∘ ]
[mm]
[ ∘ ]
[mm]
T
p
S2a → S2b
8/17
0.097
3.1
0.39
12.9
0.099
5.0
3.7
0.71
S2b → S2a
15/9
0.084
3.1
0.32
11.1
0.070
3.8
4.2
0.65
Table 10: Cross-day hold-out validation of the chassis pair.
Figure 19: Stereo check on S5. Left : NISk/dk against the predicted board displacement (red: excluded). Right : rotation discrepancy to the stereo reference for each displacement threshold, with states, 10 th to 90 th bootstrap percentiles, and reference uncertainty (grey).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 20: LiDAR points of held-out scenes projected with each extrinsic (rows) on three handheld windows and two Robot-A captures, colored by range. Insets: a near-vertical image edge (cyan) with the depth-discontinuity points (magenta) and their mean horizontal offset Δ [px]; tags: deviation from Ours (ref.). In each image, the inset edge is the candidate with the most depth-discontinuity points, chosen before any offset is computed.
Arc fraction
Free
Soft prior
Fixed radius
1.00
1.95
1.95
1.93
0.45
4.17
3.98
2.01
0.33
6.94
5.72
2.15
0.22
16.73
7.31
2.76
Appendix
Table 11: Circle-center RMSE [mm] ↓ of three circle fits on synthetic partial arcs.
Error [mm]
s^
ρ^z
Dataset
K
B
RI
B
RI
B
RI
FS-B
15
15.14
15.04
50.67
17.98
−0.12
−0.23
Head S2
21
6.85
6.44
14.69
3.43
−0.42
−0.31
Chassis S2
26
17.82
17.48
6.85
4.64
−0.17
+0.02
Appendix
Table 12: Comparison of the beam and range-incidence LiDAR edge-noise models on real-world sets.
Head ( K=35 )
Chassis ( K=40 )
αbias
∣S∣
t
σt [mm]
MDBt [mm]
∣S∣
t
σt [mm]
MDBt [mm]
0.002
30
−2.94
1.84
10.7
34
−2.62
2.06
11.9
0.005
29
−2.97
1.95
11.4
32
−2.57
2.02
11.7
0.01
28
−3.53
1.88
11.0
32
−2.57
2.02
11.7
0.02
26
−3.19
1.92
11.3
32
−2.57
2.02
11.7
0.05
26
−3.19
1.92
11.3
28
−2.14
1.96
11.4
Appendix
Table 13: Sensitivity to the removal level on the combined Robot-A head and chassis batches.
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Korea · Kookmin University, Seoul, 02707, Korea · LG Innotek, Seoul, 07796, Korea