Event-Based Method for High-Speed 3D Deformation Measurement under Extreme Illumination Conditions
Authors: Banglei Guan, Yifei Bian, Zibin Liu, Haoyang Li, Xuanyu Bai, Taihang Lei, Bin Li, Yang Shang, Qifeng Yu
Organizations: College of Aerospace Science and Engineering, National University of Defense Technology, Changsha 410073, China. · Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation, Changsha 410073, China.
Background: Large engineering structures, such as space launch towers and suspension bridges, are subjected to extreme forces that cause high-speed 3D deformation and compromise safety. These structures typically operate under extreme illumination conditions. Traditional cameras often struggle to handle strong light intensity, leading to overexposure due to their limited dynamic range. Objective: Event cameras have emerged as a compelling alternative to traditional cameras in high dynamic range and low-latency applications. This paper presents an integrated method, from calibration to measurement, using a multi-event camera array for high-speed 3D deformation monitoring of structures in extreme illumination conditions. Methods: Firstly, the proposed method combines the characteristics of the asynchronous event stream and temporal correlation analysis to extract the corresponding marker center point. Subsequently, the method achieves rapid calibration by solving the Kruppa equations in conjunction with a parameter optimization framework. Finally, by employing a unified coordinate transformation and linear intersection, the method enables the measurement of 3D deformation of the target structure. Results: Experiments confirmed that the relative measurement error is below 0.08%. Field experiments under extreme illumination conditions, including self-calibration of a multi-event camera array and 3D deformation measurement, verified the performance of the proposed method. Conclusions: This paper addressed the critical limitation of traditional cameras in measuring high-speed 3D deformations under extreme illumination conditions. The experimental results demonstrate that, compared to other methods, the proposed method can accurately measure 3D deformations of structures under harsh lighting conditions, and the relative error of the measured deformation is less than 0.1%.
Figures & tables
Figure 1: Flow diagram of the event-based measurement method for high-speed 3D deformation under extreme illumination conditions.
Figure 2: Corresponding points extraction from event streams generated by various event cameras.
Input
Corresponding points for each camera P , Iteration count i , Initial value of camera projection matrix M
Output
Intrinsic parameters K , extrinsic parameters R,T
1
Calculate scale factor
2
Optimize the projection structure using the bundle adjustment method
3
Decompose the measurement matrix to obtain the projection reconstruction of the camera matrix and structure matrix
4
Perform Euclidean reconstruction to obtain camera parameters
5
By calculating the reprojection errors of polar geometry and dynamic landmark points, outlier projection points are eliminated
Table 1: Calibration algorithm.
Figure 3: Reference coordinate system transformation.
Figure 4: Experimental configuration.
Extrinsics
Event camera 1
Event camera 2
Event camera 3
(fx,fy)
(1778.5077, 1772.3397)
(1756.2173, 1757.6988)
(1816.1340, 1809.6153)
(Cx,Cy)
(639.5, 359.5)
(639.5, 359.5)
(639.5, 359.5)
(k1,k2)
(-0.05359, 0.33899)
(-0.09119, 0.38824)
(-0.11437, 0.91938)
(p1,p2)
(-0.00157, -0.00479)
(0.00149, -0.00318)
(-0.00144, 0.00497)
Table 2: Calibration results of each camera intrinsics.
Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.
Yuliang Wu, Wei Zhai, Yuxin Cui +3
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Fuzhou University
This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second.
Yu-Hsiang Huang, Levi Burner, Sachin Shah +3
University of Maryland, College Park MD 20742, USA · Dartmouth College, Hanover NH 03755, USA
Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.
Zhichao Li, Chiara Bartolozzi, Lorenzo Natale +1
Event-driven Perception for Robotics, Istituto Italiano di Tecnologia, Italy · Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Italy · University of Genoa, Genoa, Italy