CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

Marcell Kegl; Andras Palffy; Csaba Benedek; Dariu M. Gavrila

cs.CVMar 16, 2026

CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

Authors: Marcell Kegl, Andras Palffy, Csaba Benedek, Dariu M. Gavrila

Organizations: Hungarian Research Network Institute for Computer Science and Control (HUN-REN SZTAKI), 1111 Budapest, Kende utca 13-17 · Pázmány Péter Catholic University, Faculty of Information Technology and Bionics, 1083 Budapest, Práter utca 50/A · Perciv AI, Delft, the Netherlands · Intelligent Vehicles Group, TU Delft, 2628 CD Delft, Netherlands

Abstract

In this paper, we address extrinsic calibration for camera, lidar, and 4D radar sensors. Accurate extrinsic calibration of radar remains a challenge due to the sparsity of its data. We propose CLRNet, a novel, multi-modal end-to-end deep learning (DL) calibration network capable of addressing joint camera-lidar-radar calibration, or pairwise calibration between any two of these sensors. We incorporate equirectangular projection, camera-based depth image prediction, additional radar channels, and leverage lidar with a shared feature space and loop closure loss. In extensive experiments using the View-of-Delft and Dual-Radar datasets, we demonstrate superior calibration accuracy compared to existing state-of-the-art methods, reducing both median translational and rotational calibration errors by at least 50%. Finally, we examine the domain transfer capabilities of the proposed network and baselines, when evaluating across datasets. The code will be made publicly available upon acceptance at: https://github.com/tudelft-iv.

Figures & tables

Method	Year	Camera	Lidar	4D Radar	Category	Assumptions	1 Frame	Open Dataset	Open-source
Scholler et al. [ 15 ]	2019	✓	✗	✓	end-to-end DL	specific environment, training data, rotation only	✓	✗	✗
Heng [ 27 ]	2020	✗	✓	✓	explicit geometry + optimization	sensor motion	✗	✗	✗
Persic et al. [ 28 ]	2020	✓	✓	3D radar	explicit geometry + optimization	dynamic objects, only yaw angle	✗	nuScenes	✗
Cheng and Cao [ 29 ]	2023	✓	✗	✓	DL + optimization	objects (pedestrian, car), training data	✗	✗	✗
Wise et al. [ 30 ]	2023	✓	✗	✓	explicit geometry + optimization	sensor motion	✗	✗	✗
SensorX2Car [ 31 ]	2023	✓	✓	3D radar	explicit geometry + optimization	sensor motion, only yaw angle	✗	✗	✓

TABLE I: Targetless extrinsic calibration approaches involving radar

Fig. 1: CLRNet network architecture overview. On the lidar and radar branch, the input point clouds are projected using equirectangular projection before (a) feature extraction. After the pairwise (b) feature matching layers, we (c) concatenate the output to have shared information for the (d) parameter regression layers, which are used to predict the final extrinsic parameters for each three sensor pairs.

Model configuration	camera - radar MAE
Model configuration	Transl. (cm)	Rot. (°)
CRNet vanilla	20.0	0.8
CRNet vanilla + equirectangular projection	14.0	0.6
CRNet vanilla + 5-frame-input	18.2	0.7
CRNet vanilla + additional radar channels	16.6	0.6
CRNet vanilla + camera depth branch	18.8	0.9

TABLE II: Mean average camera-radar calibration error using the VoD dataset,

\pm 20\,\text{cm}

and

\pm 1\,\text{\textdegree}

miscalibration, and different configurations.

Calibration Scenario	Method	Camera - Radar Error				Camera - Lidar Error
		Translation (cm)		Rotation (°)		Translation (cm)		Rotation (°)
		mean	median	mean	median	mean	median	mean	median
non-rigid system one model $[\pm 20\,\text{cm,}\pm 1\,\text{\textdegree}]$	Scholler et al. [ 15 ] 1	-	-	1.0 $\pm$ 0.0	1.0	-	-	-	-
	Method based on [ 33 ] 1	18.9 $\pm$ 0.4	18.6	0.8 $\pm$ 0.0	0.7	8.8 $\pm$ 0.3	7.6	0.5 $\pm$ 0.0	0.4
	4DRC-OC [ 16 ] 2	18.5 $\pm$ 0.3	18.6	0.8 $\pm$ 0.0	0.8	-	-	-	-
	LCCNet [ 12 ] 2	-	-	-	-	8.5 $\pm$ 0.2	6.8	0.3 $\pm$ 0.0	0.2

TABLE III: Comparison of our proposed methods to SOTA reference methods on the VoD dataset. Mean average and median calibration error in different calibration scenarios. Values following the symbol

\pm

denote the 95% confidence interval.

Fig. 2: Qualitative results of lidar point projection to the camera image with two reference methods [ 12 , 33 ] and the proposed CLRNet model. CLRNet is superior to [ 33 ] , also using three sensors; while our approach similarly performs to [ 12 ] , which is a pairwise lidar-camera calibration baseline method.

Fig. 3: Qualitative results of radar point projection to the camera image in three test scenes by using the calibration parameters of the (b) ground truth, (c)-(e) the three reference methods [ 15 , 33 , 16 ] and (f) the proposed CLRNet model.

Method	Train	Test	camera–radar MAE
Method	Train	Test	Transl. (cm)	Rot. (°)
4DRC-OC	VoD	Dual	170 / 186 / 43	9.0 / 8.3 / 2.3
	Dual	Dual	4.8 / 3.8 / 4.5	0.4 / 0.2 / 0.4
	Dual	VoD	190 / 194 / 55	7.0 / 6.6 / 1.5
	VoD	VoD	67.5 / 72.5 / 26.8	2.2 / 0.9 / 3.6
CRNet (ours)	VoD	Dual	134 / 139 / 46.8	6.5 / 6.3 / 1.3

TABLE IV: Domain transfer results for 4DRC-OC [ 16 ] and CRNet (ours) involving the Dual-Radar and VoD datasets. Numbers show mean / median / std MAE for translation (cm) and rotation (°), correcting

[\pm 1.0\,\text{m},\pm 20^{\circ}]

miscalibrations using iterative refinement and assuming rigid platforms.

Camera Calibration

Camera calibration estimates the geometric and optical parameters that connect image measurements to real-world positions and other sensors.

See all 55 papers

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization
Oct 3, 2026Gunhee Shin, Yunsoo Kim, Chanhyuk Lee +7Radar-LiDAR Fusion Multimodal Sensor Fusion
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Korea · Kookmin University, Seoul, 02707, Korea · LG Innotek, Seoul, 07796, Korea
Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences
May 22, 2026Simon Bultmann, Daniele Cattaneo, Abhinav ValadaCamera-LiDAR Fusion Camera Calibration
Department of Computer Science, University of Freiburg, Germany
SynCalib: Breaking Dataset Shortcuts in LiDAR-Camera Calibration via Extrinsic-Controllable Synthesis
Oct 20, 2026Huihua Zhu, Shuyang Zhang, Dongqing Chen +6Camera Calibration Shortcut Learning
College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen, Guangdong, 518060, China. · Robot Perception and Learning Lab, Intelligent Robotics, Department of Computer Science, University College London, Gower Street, WC1E 6BT, London, UK. · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong. +1

Oct 3, 2026J/K move · Enter open · S save

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization

Gunhee Shin, Yunsoo Kim, Chanhyuk Lee, Wanhee Kim+6

School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Korea · Kookmin University, Seoul, 02707, Korea · LG Innotek, Seoul, 07796, Korea

Radar-LiDAR Fusion Multimodal Sensor Fusion Camera Calibration

Open paper AskPDF arXiv

Fusing radar, LiDAR, and camera enables robust perception in diverse and adverse conditions, but the fusion performance critically depends on accurate extrinsic calibration among the three sensors. In this paper, we address the problem of online target-less extrinsic calibration for the radar-LiDAR-camera system. Existing target-less methods are mostly designed for a single sensor pair, and composing the pairwise results does not guarantee consistency across the three sensors. Moreover, the sparse and noisy radar measurements make the radar-involving pairs unreliable. To tackle these challenges, we propose a joint calibration framework that constructs residuals for each sensor pair and optimizes the extrinsics of all pairs together to minimize the overall residual. Furthermore, we introduce an adaptive radar noise filter that rejects spurious radar returns using a range-dependent margin, and a correspondence accumulation strategy that aggregates sparse radar correspondences over frames. We validate our method on an in-house radar-LiDAR-camera dataset covering diverse urban environments, where it reduces calibration errors across all sensor pairs over a state-of-the-art camera-LiDAR baseline.

Abstract

Camera Calibration

Explore similar work

Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization

Joint Target-Less Intrinsic and Extrinsic Camera-LiDAR Calibration using Deep Point Correspondences

SynCalib: Breaking Dataset Shortcuts in LiDAR-Camera Calibration via Extrinsic-Controllable Synthesis