Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences

Simon Bultmann; Daniele Cattaneo; Abhinav Valada

cs.ROOct 27, 2026

Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences

Authors: Simon Bultmann, Daniele Cattaneo, Abhinav Valada

Organizations: Department of Computer Science, University of Freiburg, Germany

Abstract

Accurate camera-LiDAR calibration is an essential prerequisite for robust multi-modal perception in robotics. Recent target-less methods based on learned pixel-point correspondences achieve high extrinsic calibration accuracy and generalize across environments. However, they operate on rectified images with known camera intrinsics, restricting their applicability to systems whose cameras have already been individually calibrated. We present a fully automatic, target-less framework that jointly estimates camera intrinsics, radial-tangential distortion, and camera-LiDAR extrinsics from raw images and LiDAR scans. Our method generalizes deep correspondence-based calibration from rectified images to raw cameras through intrinsic- and distortion-aware correspondence learning via synthetic augmentations. During calibration, we iteratively couple correspondence prediction and nonlinear optimization to jointly refine the camera parameters and relative sensor transformation. We evaluate our approach on an unseen camera-LiDAR configuration from the KITTI dataset and on three real robotic platforms in a zero-shot manner. The results demonstrate that joint calibration with our generalized pixel-point correspondence network improves extrinsic accuracy over correspondence-based methods using pre-calibrated cameras, while additionally recovering accurate intrinsics.

Figures & tables

Panel of Figure 1 — Fig. 1: Fully automatic, target-less joint intrinsic and extrinsic camera-LiDAR calibration: Our method generalizes to diverse robotic platforms and jointly estimates camera intrinsics $\mathbf{K}$ , radial-tangential distortion $\mathbf{D}$ , and the camera-LiDAR extrinsic transformation $\mathbf{T}_{\mathrm{calib}}$ .

Panel of Figure 2 — Fig. 2: Overview of the proposed joint camera-LiDAR calibration framework. Structure-from-motion (SfM) provides initial camera parameters and visual-LiDAR motion alignment initializes extrinsics. During iterative refinement, current intrinsic, distortion, and extrinsic estimates $\mathbf{K},\mathbf{D},\mathbf{T}$ define LiDAR projection input into the correspondence network. Predicted pixel-point correspondences are post-processed and used for joint nonlinear optimization. Updated calibration parameters are fed back into the next iteration. After convergence, the final parameters define the calibrated projection shown at the output.

Panel of Figure 3 — Fig. 3: Synthetic camera-parameter augmentation. (a): source image. (b): target pixels colored by geometric validity and source-image mapping. (c): remapped image with invalid pixels darkened; only valid pixels are retained.

Model	Transl.	Rot.	Intr.	Dist.	Used in iter.
Coarse	$\phantom{0}1.0\,\mathrm{m}$	$5.0^{\circ}$	$10\%^{\dagger}$	$15\%$	$1$
Medium	$\phantom{0}0.1\,\mathrm{m}$	$1.0^{\circ}$	$2\%$	$10\%$	$2$ – $3$
Fine	$0.05\,\mathrm{m}$	$0.5^{\circ}$	$1\%$	$5\%$	$4+$

TABLE I: Perturbation ranges during network training.

Method	Setting	Translation [cm]				Rotation [ ∘ ]				Intrinsics [%]				Reproj. [px]
Method	Setting	$x$	$y$	$z$	magnitude	roll	pitch	yaw	magnitude	$f_{x}$	$f_{y}$	$c_{x}$	$c_{y}$	full	dist.
CMRNext [ 13 ]	fixed GT	0.37	6.03	0.12	$6.04\pm 0.38$	0.01	0.01	0.28	$0.28\pm 0.006$	–	–	–	–	–	–
MDPCalib [ 11 ]	fixed GT	0.46	2.84	0.44	$2.92\pm 0.29$	0.02	0.05	0.13	$0.14\pm 0.010$	–	–	–	–	–	–
Ours	fixed GT	1.24	1.28	1.37	$\mathbf{2.25}\pm\mathbf{0.05}$	0.02	0.04	0.02	$\mathbf{0.05}\pm\mathbf{0.002}$	–	–	–	–	–	–
MDPCalib ∗	fixed COL	30.2	5.14	6.18	$31.2\pm 0.40$	0.13	0.12	0.06	$0.19\pm 0.01$	2.11	0.26	0.46	0.25	5.45	0.86
Ours	fixed COL	2.81	0.49	0.13	$\mathbf{2.86}\pm\mathbf{0.56}$	0.08	0.09	0.05	$\mathbf{0.13}\pm\mathbf{0.02}$	2.11	0.26	0.46	0.25	5.45	0.86

TABLE II: Joint intrinsic and extrinsic calibration results on KITTI (Seq. 00, right camera, unseen setup). Mean

\pm

std over 100 runs.

Augm.	RANSAC	PP prior	Trans. [cm]	Rot. [ ∘ ]	Reproj. [px]
–	–	–	$5.30\pm 1.48$	$2.73\pm 0.09$	$42.7\pm 1.95$
✓	–	–	$2.54\pm 0.13$	$0.41\pm 0.23$	$6.10\pm 3.47$
✓	✓	–	$2.48\pm 0.24$	$0.24\pm 0.12$	$4.11\pm 1.84$
✓	–	✓	$2.32\pm 0.44$	$0.27\pm 0.04$	$3.64\pm 0.75$
✓	✓	✓	$\mathbf{2.27}\pm\mathbf{0.09}$	$\mathbf{0.11}\pm\mathbf{0.02}$	$\mathbf{1.14}\pm\mathbf{0.24}$

TABLE III: Ablation study on KITTI Seq. 00 (raw right camera, unseen setup; perturbed GT + intr. init.). Mean

\pm

std over 100 runs.

Panel of Figure 7 — Fig. 4: Hyperparameter study on optimization schedule: (a) Translation and rotation errors for intrinsic refinement durations $N_{\mathrm{intr}}$ over total iterations $N$ . For $N\leq N_{\mathrm{intr}}$ , each point aggregates all runs with active intrinsic optimization. For $N>N_{\mathrm{intr}}$ , individual trajectories are shown for intrinsics fixed at resp. $N_{\mathrm{intr}}$ . (b) Intrinsic reprojection error over $N_{\mathrm{intr}}$ . The selected schedule $N_{\mathrm{intr}}=5$ and $N_{\mathrm{total}}=11$ provides the best overall trade-off between accurate intrinsic estimation and stable extrinsic refinement.

Panel of Figure 8 — Fig. 5: Qualitative calibration results on KITTI and the three robotic platforms. Blue colored near-field highlights emphasize structures near the image borders, where the camera-LiDAR projection is particularly sensitive to intrinsic and distortion parameters, demonstrating that our method recovers them accurately. Circular zoom-ins show distant structures at the annotated ranges, demonstrating precise joint calibration over long distances.

Robot	Method	Setting	Trans. [cm]	Rot. [ ∘ ]	Reproj. [px]
Car	Koide [ 19 ]	fixed GT	9.76	0.27	–
	MDP [ 11 ]	fixed GT	4.50	0.27	–
	Shen [ 29 ]	opt. COL +	$20.0\pm 13.1$	$1.24\pm 0.65$	$46.6\pm 72.0$
	Ours	opt. COL +	$\mathbf{3.02}\pm\mathbf{0.11}$	$\mathbf{0.09}\pm\mathbf{0.04}$	$\mathbf{3.63}\pm\mathbf{0.48}$
Spot	Koide [ 19 ]	fixed GT	16.21	1.34	–
	MDP [ 11 ]	fixed GT	9.54	0.38	–

TABLE IV: Target-less calibration results on real robotic platforms.

Stage	Inference [s]	Optimization [s]	Total [s]
$j\leq N_{\mathrm{intr}}$ , per it.	$16.94\pm 0.09$	$15.32\pm 5.05$	$32.25\pm 5.05$
$j>N_{\mathrm{intr}}$ , per it.	$16.94\pm 0.05$	$\phantom{0}5.10\pm 1.32$	$22.04\pm 1.32$
Complete calibration	$186.35\pm 0.66$	$107.19\pm 10.80$	$302.13\pm 16.58$

TABLE V: Runtime Evaluation. Mean

\pm

std over 100 runs.

Lidar-Camera Extrinsic Calibration

See all 18 papers

May 22, 2026

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences

Accurate camera-LiDAR calibration is a prerequisite for robust multi-modal perception in robotics. Recent target-less approaches based on deep point correspondences achieve remarkable performance for extrinsic calibration but assume rectified images with known intrinsics. In this work, we overcome this limitation and present the first fully target-less pipeline that jointly estimates camera intrinsics (pinhole model with radial-tangential distortion) and camera-LiDAR extrinsics with deep pixel-point correspondences. Our approach extends deep correspondence-based calibration by (i) automatic intrinsic initialization via structure-from-motion, (ii) generalizing camera-LiDAR matching to raw images with unknown intrinsics including distortion, and (iii) tightly coupling correspondence estimation with joint nonlinear optimization over both intrinsics and extrinsics. We evaluate our method on the KITTI dataset with unseen camera-LiDAR pairs and demonstrate that joint calibration achieves improved extrinsic accuracy while additionally recovering accurate intrinsics.

Simon Bultmann, Daniele Cattaneo, Abhinav Valada

Department of Computer Science, University of Freiburg, Germany

Camera-LiDAR Fusion Camera Calibration Lidar-Camera Extrinsic Calibration

Ask PDF

Mar 31, 2026

Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations

Accurate camera-LiDAR fusion relies on precise extrinsic calibration, which fundamentally depends on establishing reliable cross-modal correspondences under potentially large misalignments. Existing learning-based methods typically project LiDAR points into depth maps for feature fusion, which distorts 3D geometry and degrades performance when the extrinsic initialization is far from the ground truth. To address this issue, we propose an extrinsic-aware cross-attention framework that directly aligns image patches and LiDAR point groups in their native domains. The proposed attention mechanism explicitly injects extrinsic parameter hypotheses into the correspondence modeling process, enabling geometry-consistent cross-modal interaction without relying on projected 2D depth maps. Extensive experiments on the KITTI and nuScenes benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches in both accuracy and robustness. Under large extrinsic perturbations, our approach achieves accurate calibration in 88% of KITTI cases and 99% of nuScenes cases, substantially surpassing the second-best baseline. We have open sourced our code on https://github.com/gitouni/ProjFusion to benefit the community.

Ni Ou, Zhuo Chen, Xinru Zhang +1

Ni Ou, Junzheng Wang are with the School of Automation, Beijing Institute of Technology, Beijing, 100081, China. · Zhuo Chen is with the Robot Perception Lab, Centre for Robotics Research, Department of Engineering, King’s College London, London WC2R 2LS, United Kingdom. · Xinru Zhang is with the School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, 100081, China.

Lidar-Camera Extrinsic Calibration Camera Calibration Camera-LiDAR Fusion Cross-Modal Attention Cross-Modal Feature Matching

Ask PDF

Oct 3, 2026

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization

Fusing radar, LiDAR, and camera enables robust perception in diverse and adverse conditions, but the fusion performance critically depends on accurate extrinsic calibration among the three sensors. In this paper, we address the problem of online target-less extrinsic calibration for the radar-LiDAR-camera system. Existing target-less methods are mostly designed for a single sensor pair, and composing the pairwise results does not guarantee consistency across the three sensors. Moreover, the sparse and noisy radar measurements make the radar-involving pairs unreliable. To tackle these challenges, we propose a joint calibration framework that constructs residuals for each sensor pair and optimizes the extrinsics of all pairs together to minimize the overall residual. Furthermore, we introduce an adaptive radar noise filter that rejects spurious radar returns using a range-dependent margin, and a correspondence accumulation strategy that aggregates sparse radar correspondences over frames. We validate our method on an in-house radar-LiDAR-camera dataset covering diverse urban environments, where it reduces calibration errors across all sensor pairs over a state-of-the-art camera-LiDAR baseline.

Gunhee Shin, Yunsoo Kim, Chanhyuk Lee +7

School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Korea · Kookmin University, Seoul, 02707, Korea · LG Innotek, Seoul, 07796, Korea

Lidar-Camera Extrinsic Calibration Radar-LiDAR Fusion Multimodal Sensor Fusion Camera Calibration

Ask PDF

Abstract

Lidar-Camera Extrinsic Calibration

Explore similar work

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences

Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization