Accurate extrinsic calibration between event-based and frame-based cameras remains a practical bottleneck for heterogeneous stereo systems. Existing approaches often require sensor or target motion, precise synchronization, or computationally expensive event-to-image reconstruction. We propose a simple, motion-free cross-modal calibration framework that uses a temporally modulated, blended ChArUco target presented on standard consumer displays. By alternating between the original pattern and a partially blended version, the target reliably triggers events while remaining continuously observable to a frame-based camera, avoiding blank frames and reducing synchronization constraints to a coarse, trigger-based alignment. We discretize events into frames coarsely aligned with the RGB images, apply lightweight denoising, and perform ChArUco-based intrinsic and stereo extrinsic calibration. Extensive experiments assess robustness to blending opacity, display brightness, external illumination, viewing angle, and handheld acquisition. Compared to the strongest motion-based reference (E2Calib + Kalibr) and a non-motion-based reference (Plasberg et al.), our approach reduces the mean reprojection error by 44% and 6%, respectively, while substantially simplifying the calibration procedure. Finally, we demonstrate practical utility in a robotic eye-to-hand calibration case study, showing consistent transformations and stable downstream geometric measurements even under partial occlusions. Code is publicly available at https://github.com/nhessenthaler/simple-evrgb-cal.
Figures & tables
Figure 1: Pipeline for cross‑modal stereo calibration using a temporally modulated ChArUco pattern. A pattern of blended images is generated via linear interpolation and modulated over time, enabling robust detection in both frame‑based and event cameras.
Figure 2: Detection rate and stereo reprojection error as a function of the blending opacity α . Frame-based detection remains close to 100% across most opacity levels, with a slight degradation for α→1 due to the sporadic invisibility of the pattern in the absence of temporal display synchronization. In contrast, event-based detection increases with α and saturates for α≥0.4 . The stereo reprojection error reaches a minimum at α=0.6 , indicating an optimal trade-off between detection robustness and calibration accuracy.
Brightness [%]
Lwhite [cd/m 2 ]
Lblack [cd/m 2 ]
uc [%] ↑
εrep [px] ↓
0
35.0
0.03
15.3
–
20
78.0
0.06
98.5
0.442
40
121.0
0.10
100.0
0.418
60
164.0
0.13
100.0
0.405
80
236.0
0.19
100.0
0.395
100
391.0
0.32
100.0
0.384
Table 1: Influence of display brightness on stereo calibration performance for a single experimental run, evaluated across brightness levels from 0% to 100% . The table reports the measured luminance of the white ( Lwhite ) and black ( Lblack ) display states, the combined successful corner detection rate ( uc ), and the stereo reprojection error ( εrep ), with arrows ( ↑,↓ ) indicating the direction of improved performance.
Sensor
uc [%] ↑
fx [px]
fy [px]
cx [px]
cy [px]
εrep [px] ↓
RGB
( μ )
100.0
2771.5
2772.6
800.1
450.8
0.142
( σ )
0.0
3.5
3.4
1.8
1.4
0.006
Event
( μ )
100.0
2571.1
2572.3
609.3
378.2
0.449
( σ )
0.0
26.4
26.3
10.7
9.1
0.013
Table 2: Statistical robustness evaluation of the intrinsic parameters over 20 independent calibration runs using a robotic setup. The table reports the mean ( μ ) and standard deviation ( σ ) for the corner detection success rate ( uc ), focal lengths ( fx,fy ), principal point ( cx,cy ), and reprojection error ( εrep ).
Sensor
uc [%] ↑
∥r∥ [°]
∥t∥ [m]
εrep [px] ↓
RGB + Event
( μ )
99.8
1.077
0.0346
0.383
( σ )
0.5
0.193
0.00017
0.012
Table 3: Statistical robustness evaluation of the extrinsic parameters over 20 runs. The table reports the successful corner detection rate ( uc ), the magnitude of the estimated rotation ( ∥r∥ ) and translation ( ∥t∥ ), and the reprojection error ( εrep ).
Sensor
uc [%] ↑
∥r∥ [°]
∥t∥ [m]
εrep [px] ↓
RGB + Event
( μ )
92.8
1.252
0.0345
0.402
( σ )
10.8
0.266
0.00012
0.017
Table 4: Illumination robustness evaluation of the extrinsic parameters over 5 runs with varying spotlight positions. The table reports the successful corner detection rate ( uc ), the magnitude of the estimated rotation ( ∥r∥ ) and translation ( ∥t∥ ), and the reprojection error ( εrep ).
Vertical (V)
Horizontal (H)
Diagonal (D)
Angle
uc
εrep
uc
εrep
uc
εrep
[%] ↓
[px] ↑
[%] ↓
[px] ↑
[%] ↓
[px] ↑
30 ∘
100.0
0.322
100.0
0.320
100.0
0.313
40 ∘
100.0
0.363
100.0
0.342
100.0
0.388
50 ∘
100.0
0.374
100.0
0.359
89.2
0.482
55 ∘
100.0
0.436
56.7
0.374
48.3
0.926
Table 5: Viewing angle stability evaluation across viewing angles from 30∘ to 70∘ , evaluated along the vertical (V), horizontal (H), and diagonal (D) axes. The table reports the combined corner detection rate ( uc ) and reprojection error ( εrep ), averaged over 5 runs per direction.
Sensor
uc [%] ↑
∥r∥ [°]
∥t∥ [m]
εrep [px] ↓
RGB + Event
( μ )
100.0
0.9435
0.0343
0.361
( σ )
0.0
0.285
0.00027
0.041
Table 6: Handheld calibration evaluation of the extrinsic parameters over 5 runs. The table reports the successful corner detection rate ( uc ), the magnitude of the estimated rotation ( ∥r∥ ) and translation ( ∥t∥ ), and the reprojection error ( εrep ).
Figure 3: E2VID reconstructed intensity frames used for the original E2Calib baseline (checkerboard) and our adapted ChArUco target-based variant.
Method
uc [%] ↑
∥r∥ [°]
∥t∥ [m]
εrep [px] ↓
E2Calib
( μ )
47.7
1.437
0.0339
1.235
( σ )
5.6
0.386
0.0004
0.156
E2Calib + ChArUco
( μ )
67.7
1.791
0.0338
1.161
( σ )
3.7
0.381
0.0003
0.108
E2Calib + Kalibr
( μ )
1.581
0.0360
0.687
( σ )
0.178
0.0002
0.038
Table 7: Extrinsic calibration performance of the motion-requiring baseline (E2Calib), its ChArUco-based adaptation (E2Calib + ChArUco), a variant using Kalibr instead of OpenCV for calibration (E2Calib + Kalibr), and the static baseline method of Plasberg et al., each evaluated over 5 runs. The table reports the successful corner detection rate ( uc ), the magnitude of the estimated rotation ( ∥r∥ ) and translation ( ∥t∥ ), and the reprojection error ( εrep ).
Figure 4: Geometric transformation pipeline for eye-to-hand calibration, illustrating the mapping of a target point pt from the ChArUco target frame ( t ) to the event camera frame ( e ) via eTt , and subsequently to the robot base frame ( B ) using BTe , with the final relation to the TCP defined through the robot kinematics tcpTB .
Statistic
tx [m]
ty [m]
tz [m]
∥t∥ [m]
θ [°]
BTe
( μ )
0.1258
-0.5678
0.3839
0.6970
173.76
( σ )
0.0093
0.0076
0.0061
0.0056
2.80
Table 8: Hand-eye calibration results aggregated over 20 valid runs. The table reports the mean ( μ ) and standard deviation ( σ ) for the estimated translation vector ( t=(tx,ty,tz) ) between the event camera and robot base, its Euclidean norm ( ∥t∥ ), and the rotation magnitude ( θ ) obtained from the rotation matrix R via axis-angle representation.
Evaluation Mode
d [mm]
σd [mm] ↓
dgt [mm]
MAE [mm] ↓
Target-to-TCP
67.09
1.86
67.80
0.71
Base-Frame Distance
Scene dep.
0.66
Scene dep.
1.40
Table 9: Spatial consistency evaluation results averaged over 5 runs, with each run consisting of 22 diverse poses for the target-to-TCP evaluation and 6 trajectories for the base-frame distance evaluation. The table reports the mean Euclidean distance ( d ) between the transformed calibration point (ID9) and the TCP, its standard deviation ( σd ), the ground truth distance ( dgt ), and the mean absolute error ( MAE ) with respect to the ground truth.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: The hybrid frame-event stereo rig is mounted on a UR5e robotic arm to ensure precise trajectory repeatability. The Dell S2722QC monitor displays the calibration pattern, serving as the cross-modal target for both the frame-based and event sensors.
Parameter UI-5250CP
Value
Exposure Time
100.0ms
Gain
100
Frame Rate
10.0fps
Gamma
100
Appendix
Table 10: IDS uEye frame-based camera configuration used in all experiments. The gamma value is specified in the camera parameterization scheme, where a value of 100 corresponds to a linear gamma of γ=1.0 . All parameters not explicitly listed were kept at their hardware default settings.
Parameter EVK4
Value
Event Trail Filter
Enabled
Filtering Type
STC_CUT_TRAIL
Trail Threshold
100000
bias_diff_off
−15
bias_diff_on
−35
bias_fo
−35
Appendix
Table 11: Prophesee EVK4 sensor configuration used in all experiments. Bias values denote relative offsets from the hardware defaults and were tuned to optimize the signal-to-noise ratio for flickering pattern detection. All parameters not explicitly listed were kept at their hardware default settings.
Figure 6: Qualitative example from the illumination robustness experiment. Both images depict the same scene. Differences in perspective are due to the stereo baseline offset between the frame-based and event camera sensors. (a) RGB image frame captured under strong external spotlight interference, causing overexposure on the calibration target. (b) Corresponding event frame recorded under the same conditions, illustrating the effect of intense illumination on event generation and pattern visibility.
Sensor
uc [%] ↑
fx [px]
fy [px]
cx [px]
cy [px]
εrep [px] ↓
RGB
( μ )
100.0
2764.8
2766.1
797.1
449.5
0.140
( σ )
0.0
2.4
2.3
5.0
1.8
0.003
Event
( μ )
98.5
2568.9
2569.6
600.5
367.8
0.456
( σ )
3.4
11.4
11.6
9.3
10.2
0.007
Appendix
Table 12: Illumination robustness evaluation of the intrinsic parameters with our approach over 5 runs with varying spotlight positions. The table reports the corner detection success rate ( uc ), focal lengths ( fx,fy ), principal point ( cx,cy ), and reprojection error ( εrep ).
Figure 7: Robustness of the proposed method under occlusion during eye-to-hand calibration. Even in the presence of partial occlusions caused by the robot actuator, salient ChArUco corners (highlighted in red) are reliably detected, enabling accurate calibration.
Conventional frame-based cameras capture rich contextual information but suffer from limited temporal resolution and motion blur in dynamic scenes. Event cameras offer an alternative visual representation with higher dynamic range free from such limitations. The complementary characteristics of the two modalities make event-frame asymmetric stereo promising for reliable 3D perception under fast motion and challenging illumination. However, the modality gap often leads to marginalization of domain-specific cues essential for cross-modal stereo matching. In this paper, we introduce Bi-CMPStereo, a novel bidirectional cross-modal prompting framework that fully exploits semantic and structural features from both domains for robust matching. Our approach learns finely aligned stereo representations within a target canonical space and integrates complementary representations by projecting each modality into both event and frame domains. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods in accuracy and generalization.
Ninghui Xu, Fabio Tosi, Lihui Wang +5
School of Instrument Science and Engineering, Southeast University, State Key Lab of Comprehensive PNT Network and Equipment Technology, Key Lab of Micro-Inertial Instrument and Advanced Navigation Technology, MOE · University of Bologna · Department of Computer Science and Engineering, University of Bologna +2
Extrinsic calibration of multi-camera systems with non-overlapping FOVs has been a challenging problem in the robotics literature. Conventional target-based methods impose substantial target setup overhead, either deploying large calibration targets or requiring pre-measured multi-target poses. Motion-based approaches instead suffer from drift error, scale ambiguity, and motion degeneracy. Securing both accuracy and usability, we propose a novel calibration method that leverages pure rotational motion, requiring only a single static calibration board. The key idea is to make all cameras sequentially observe the same target under a shared geometric reference, even without overlapping views. To integrate these time-separated observations, we formulate the problem using a latent turntable frame and a 3D error on SE(3) within a global optimization framework. We validate the proposed method on both a controlled camera rig and a full-scale vehicle platform with heterogeneous cameras, and analyze robustness under non-ideal turntable motion. Extensive experiments show that our approach maintains competitive accuracy without specialized precision hardware, proving its strong suitability for realistic on-site deployments. Our code is publicly available here.
Gwanhyeong Song, Chaehyeon Song, Ayoung Kim
Dept. of Mechanical Engineering, SNU, Seoul, S. Korea
In this work, we propose StereoGeo, an end-to-end network-based approach for stereo camera calibration. Our method estimates the focal lengths and gravity directions of the left and right cameras, as well as the relative extrinsic transformation relating them. Existing methods often rely on calibration patterns in structured environments or address only a single camera configuration, being limited to either intrinsic or extrinsic estimation, and depending on a multi-view setups. StereoGeo extends the GeoCalib algorithm, integrating deep neural network feature extraction with a differentiable optimizer. Extensive experiments on real-world benchmarks demonstrate that StereoGeo achieves competitive performance for intrinsic calibration and provides accurate stereo extrinsic estimation, outperforming existing methods that are limited to monocular settings. The dataset used in this work is partially publicly available at https://github.com/meddourimane/StereoGeo-dataset.