Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences

Simon Bultmann; Daniele Cattaneo; Abhinav Valada

cs.ROOct 27, 2026

Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences

Authors: Simon Bultmann, Daniele Cattaneo, Abhinav Valada

Organizations: Department of Computer Science, University of Freiburg, Germany

Abstract

Accurate camera-LiDAR calibration is an essential prerequisite for robust multi-modal perception in robotics. Recent target-less methods based on learned pixel-point correspondences achieve high extrinsic calibration accuracy and generalize across environments. However, they operate on rectified images with known camera intrinsics, restricting their applicability to systems whose cameras have already been individually calibrated. We present a fully automatic, target-less framework that jointly estimates camera intrinsics, radial-tangential distortion, and camera-LiDAR extrinsics from raw images and LiDAR scans. Our method generalizes deep correspondence-based calibration from rectified images to raw cameras through intrinsic- and distortion-aware correspondence learning via synthetic augmentations. During calibration, we iteratively couple correspondence prediction and nonlinear optimization to jointly refine the camera parameters and relative sensor transformation. We evaluate our approach on an unseen camera-LiDAR configuration from the KITTI dataset and on three real robotic platforms in a zero-shot manner. The results demonstrate that joint calibration with our generalized pixel-point correspondence network improves extrinsic accuracy over correspondence-based methods using pre-calibrated cameras, while additionally recovering accurate intrinsics.

Figures & tables

Panel of Figure 1 — Fig. 1: Fully automatic, target-less joint intrinsic and extrinsic camera-LiDAR calibration: Our method generalizes to diverse robotic platforms and jointly estimates camera intrinsics $\mathbf{K}$ , radial-tangential distortion $\mathbf{D}$ , and the camera-LiDAR extrinsic transformation $\mathbf{T}_{\mathrm{calib}}$ .

Panel of Figure 2 — Fig. 2: Overview of the proposed joint camera-LiDAR calibration framework. Structure-from-motion (SfM) provides initial camera parameters and visual-LiDAR motion alignment initializes extrinsics. During iterative refinement, current intrinsic, distortion, and extrinsic estimates $\mathbf{K},\mathbf{D},\mathbf{T}$ define LiDAR projection input into the correspondence network. Predicted pixel-point correspondences are post-processed and used for joint nonlinear optimization. Updated calibration parameters are fed back into the next iteration. After convergence, the final parameters define the calibrated projection shown at the output.

Panel of Figure 3 — Fig. 3: Synthetic camera-parameter augmentation. (a): source image. (b): target pixels colored by geometric validity and source-image mapping. (c): remapped image with invalid pixels darkened; only valid pixels are retained.

Model	Transl.	Rot.	Intr.	Dist.	Used in iter.
Coarse	$\phantom{0}1.0\,\mathrm{m}$	$5.0^{\circ}$	$10\%^{\dagger}$	$15\%$	$1$
Medium	$\phantom{0}0.1\,\mathrm{m}$	$1.0^{\circ}$	$2\%$	$10\%$	$2$ – $3$
Fine	$0.05\,\mathrm{m}$	$0.5^{\circ}$	$1\%$	$5\%$	$4+$

TABLE I: Perturbation ranges during network training.

Method	Setting	Translation [cm]				Rotation [ ∘ ]				Intrinsics [%]				Reproj. [px]
Method	Setting	$x$	$y$	$z$	magnitude	roll	pitch	yaw	magnitude	$f_{x}$	$f_{y}$	$c_{x}$	$c_{y}$	full	dist.
CMRNext [ 13 ]	fixed GT	0.37	6.03	0.12	$6.04\pm 0.38$	0.01	0.01	0.28	$0.28\pm 0.006$	–	–	–	–	–	–
MDPCalib [ 11 ]	fixed GT	0.46	2.84	0.44	$2.92\pm 0.29$	0.02	0.05	0.13	$0.14\pm 0.010$	–	–	–	–	–	–
Ours	fixed GT	1.24	1.28	1.37	$\mathbf{2.25}\pm\mathbf{0.05}$	0.02	0.04	0.02	$\mathbf{0.05}\pm\mathbf{0.002}$	–	–	–	–	–	–
MDPCalib ∗	fixed COL	30.2	5.14	6.18	$31.2\pm 0.40$	0.13	0.12	0.06	$0.19\pm 0.01$	2.11	0.26	0.46	0.25	5.45	0.86
Ours	fixed COL	2.81	0.49	0.13	$\mathbf{2.86}\pm\mathbf{0.56}$	0.08	0.09	0.05	$\mathbf{0.13}\pm\mathbf{0.02}$	2.11	0.26	0.46	0.25	5.45	0.86

TABLE II: Joint intrinsic and extrinsic calibration results on KITTI (Seq. 00, right camera, unseen setup). Mean

\pm

std over 100 runs.

Augm.	RANSAC	PP prior	Trans. [cm]	Rot. [ ∘ ]	Reproj. [px]
–	–	–	$5.30\pm 1.48$	$2.73\pm 0.09$	$42.7\pm 1.95$
✓	–	–	$2.54\pm 0.13$	$0.41\pm 0.23$	$6.10\pm 3.47$
✓	✓	–	$2.48\pm 0.24$	$0.24\pm 0.12$	$4.11\pm 1.84$
✓	–	✓	$2.32\pm 0.44$	$0.27\pm 0.04$	$3.64\pm 0.75$
✓	✓	✓	$\mathbf{2.27}\pm\mathbf{0.09}$	$\mathbf{0.11}\pm\mathbf{0.02}$	$\mathbf{1.14}\pm\mathbf{0.24}$

TABLE III: Ablation study on KITTI Seq. 00 (raw right camera, unseen setup; perturbed GT + intr. init.). Mean

\pm

std over 100 runs.

Panel of Figure 7 — Fig. 4: Hyperparameter study on optimization schedule: (a) Translation and rotation errors for intrinsic refinement durations $N_{\mathrm{intr}}$ over total iterations $N$ . For $N\leq N_{\mathrm{intr}}$ , each point aggregates all runs with active intrinsic optimization. For $N>N_{\mathrm{intr}}$ , individual trajectories are shown for intrinsics fixed at resp. $N_{\mathrm{intr}}$ . (b) Intrinsic reprojection error over $N_{\mathrm{intr}}$ . The selected schedule $N_{\mathrm{intr}}=5$ and $N_{\mathrm{total}}=11$ provides the best overall trade-off between accurate intrinsic estimation and stable extrinsic refinement.

Panel of Figure 8 — Fig. 5: Qualitative calibration results on KITTI and the three robotic platforms. Blue colored near-field highlights emphasize structures near the image borders, where the camera-LiDAR projection is particularly sensitive to intrinsic and distortion parameters, demonstrating that our method recovers them accurately. Circular zoom-ins show distant structures at the annotated ranges, demonstrating precise joint calibration over long distances.

Robot	Method	Setting	Trans. [cm]	Rot. [ ∘ ]	Reproj. [px]
Car	Koide [ 19 ]	fixed GT	9.76	0.27	–
	MDP [ 11 ]	fixed GT	4.50	0.27	–
	Shen [ 29 ]	opt. COL +	$20.0\pm 13.1$	$1.24\pm 0.65$	$46.6\pm 72.0$
	Ours	opt. COL +	$\mathbf{3.02}\pm\mathbf{0.11}$	$\mathbf{0.09}\pm\mathbf{0.04}$	$\mathbf{3.63}\pm\mathbf{0.48}$
Spot	Koide [ 19 ]	fixed GT	16.21	1.34	–
	MDP [ 11 ]	fixed GT	9.54	0.38	–

TABLE IV: Target-less calibration results on real robotic platforms.

Stage	Inference [s]	Optimization [s]	Total [s]
$j\leq N_{\mathrm{intr}}$ , per it.	$16.94\pm 0.09$	$15.32\pm 5.05$	$32.25\pm 5.05$
$j>N_{\mathrm{intr}}$ , per it.	$16.94\pm 0.05$	$\phantom{0}5.10\pm 1.32$	$22.04\pm 1.32$
Complete calibration	$186.35\pm 0.66$	$107.19\pm 10.80$	$302.13\pm 16.58$

TABLE V: Runtime Evaluation. Mean

\pm

std over 100 runs.

Lidar-Camera Extrinsic Calibration

See all 18 papers

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences
May 22, 2026Simon Bultmann, Daniele Cattaneo, Abhinav ValadaCamera-LiDAR Fusion Camera Calibration
Department of Computer Science, University of Freiburg, Germany
Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations
Mar 31, 2026Ni Ou, Zhuo Chen, Xinru Zhang +1Lidar-Camera Extrinsic Calibration Camera Calibration
Ni Ou, Junzheng Wang are with the School of Automation, Beijing Institute of Technology, Beijing, 100081, China. · Zhuo Chen is with the Robot Perception Lab, Centre for Robotics Research, Department of Engineering, King’s College London, London WC2R 2LS, United Kingdom. · Xinru Zhang is with the School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, 100081, China.
Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization
Oct 3, 2026Gunhee Shin, Yunsoo Kim, Chanhyuk Lee +7Lidar-Camera Extrinsic Calibration Radar-LiDAR Fusion
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Korea · Kookmin University, Seoul, 02707, Korea · LG Innotek, Seoul, 07796, Korea

May 22, 2026J/K move · Enter open · S save

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences

Simon Bultmann, Daniele Cattaneo, Abhinav Valada

Department of Computer Science, University of Freiburg, Germany

Camera-LiDAR Fusion Camera Calibration Lidar-Camera Extrinsic Calibration

Open paper AskPDF arXiv

Accurate camera-LiDAR calibration is a prerequisite for robust multi-modal perception in robotics. Recent target-less approaches based on deep point correspondences achieve remarkable performance for extrinsic calibration but assume rectified images with known intrinsics. In this work, we overcome this limitation and present the first fully target-less pipeline that jointly estimates camera intrinsics (pinhole model with radial-tangential distortion) and camera-LiDAR extrinsics with deep pixel-point correspondences. Our approach extends deep correspondence-based calibration by (i) automatic intrinsic initialization via structure-from-motion, (ii) generalizing camera-LiDAR matching to raw images with unknown intrinsics including distortion, and (iii) tightly coupling correspondence estimation with joint nonlinear optimization over both intrinsics and extrinsics. We evaluate our method on the KITTI dataset with unseen camera-LiDAR pairs and demonstrate that joint calibration achieves improved extrinsic accuracy while additionally recovering accurate intrinsics.

Abstract

Lidar-Camera Extrinsic Calibration

Explore similar work

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences

Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization