Event-based Visual Deformation Measurement

Yuliang Wu; Wei Zhai; Yuxin Cui; Tiesong Zhao; Yang Cao; Zheng-Jun Zha

cs.CVFeb 16, 2026

Event-based Visual Deformation Measurement

Authors: Yuliang Wu, Wei Zhai, Yuxin Cui, Tiesong Zhao, Yang Cao, Zheng-Jun Zha

Organizations: MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Fuzhou University

Abstract

Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.

Figures & tables

Figure 1: Overview of Event-based Deformation Measurement. (a) Our approach regresses deformation fields by leveraging affine-invariant spatiotemporal trajectory flows extracted from both events and images. (b) Measuring tire deformation while rolling over a step obstacle. Demonstrating robust and accurate measurement capabilities of our event-based VDM approach for objects experiencing rapid self-motion.

Figure 2: Illustration of the key steps of our method. (a) Data input and event warping strategy. (b) Overall pipeline of the proposed event-based VDM framework. (c) Illustration of event-to-subregion association through vector cross product with anchor trajectories and subsequent displacement interpolation. (d) Image intensity sampling scheme within each subregion.

Figure 3: The optimization steps. (a) Hierarchical subregion optimization: we first estimate rigid motion parameters of the object, then progressively split and optimize subregions from coarse to fine. (b) Neighborhood-greedy optimization: well-optimized subregions serve as anchors to guide neighboring regions toward better convergence, achieving faster convergence to the global optimum compared to direct high dimensional optimization.

Figure 4: Hardware configuration of the hybrid event-frame system. (a) Left: Measurement setup where a pressure/stretching machine applies controlled force to the rubber sample. Right: The hybrid system comprises an event camera (Prophesee EVK4) and a 210fps grayscale camera with identical pixel size and flange distance. A signal generator produces square wave synchronization signals. The cameras are collocated by mounting a 50:50 split ratio beam splitter in front of them. Spatial calibration is performed before each data acquisition. (b) Data transmission topology of the system. A 210Hz square wave synchronization signal is used for triggering the frame camera and synchronizing the event stream. Simultaneously acquired event streams and frame data are transmitted to the computer for processing.

Method	Data input		5-20 pixels			20-100 pixels			100+ pixels
Method	Event	Frame	EPE $\downarrow$	SEPE $\downarrow$	Survival $\uparrow$	EPE $\downarrow$	SEPE $\downarrow$	Survival $\uparrow$	EPE $\downarrow$	SEPE $\downarrow$	Survival $\uparrow$
Opencorr [ 21 ]		$\checkmark$	0.132	0.108	99.1%	1.727	0.478	82.5%	39.931	2.782	25.0%
Opencorr [ 21 ] + Timelens [ 41 ]	$\checkmark$	$\checkmark$	0.227	0.226	99.5%	0.819	0.657	89.8%	3.830	1.201	41.3%
StrainNet [ 4 ]		$\checkmark$	1.553	1.146	91.0%	3.413	1.230	61.2%	41.870	1.138	18.8%
StrainNet [ 4 ] + Timelens [ 41 ]	$\checkmark$	$\checkmark$	0.978	0.911	95.2%	1.745	1.603	91.8%	5.189	2.315	35.4%
E-Raft [ 12 ]	$\checkmark$		6.574	1.510	21.7%	21.422	1.632	6.7%	60.312	1.700	2.1%

Table 1: Quantitative comparisons of our method to the model-based [ 21 ] and learning-based [ 4 ] VDM algorithms, event-based dense optical flow methods [ 12 ] , and long-term point tracking algorithms [ 23 ] . Image-based methods are enhanced with event-based frame interpolator [ 41 ] for thorough comparison. In terms of metrics, lower is better for EPE and SEPE, while higher is better for Survival Rate.

Figure 5: Qualitative comparisons. Results on samples under clamping and twisting (I, II), and subjected to tip pressure (III, IV).

Figure 6: Results of our system across diverse scenarios. Each row corresponds to a different deformation scenario. From left to right, each column shows: (i) the scene overview, (ii) event count map before motion estimation, (iii) warped event count map after optimization, (iv) regressed subregion anchor trajectories, and (v) von Mises deformation fields at two time instances during the deformation process.

Data Input			Data Size	EPE $\downarrow$	SEPE $\downarrow$	Survival $\uparrow$
Event	Frame	Frame Fps	( frames + events )	EPE $\downarrow$	SEPE $\downarrow$	Survival $\uparrow$
$\checkmark$	$\checkmark$	1	5.6Mb + 78.3Mb	4.710	2.533	32.6%
$\checkmark$	$\checkmark$	5	28.1Mb + 78.3Mb	3.204	0.813	65.7%
$\checkmark$	$\checkmark$	10	56.3Mb + 78.3Mb	2.110	0.795	67.5%
$\checkmark$	$\checkmark$	20	112.6Mb + 78.3Mb	1.618	0.573	71.2%
	$\checkmark$	5	28.1Mb	39.931	2.782	25.0%

Table 2: Comparison of data storage and processing consumption on the 100+ pixel displacement test set. The upper panel illustrates the performance of our method with event and various frame rate inputs, while the lower shows the performance of the baseline method [ 21 ] used for ground truth calculation at different frame rates. The green bars represent our method with 5 fps input, requiring only 18.9% data storage, while the blue bars present the baseline method at 100 fps with comparable performance.

Strategy	Survival $\uparrow$	mean converge time $\downarrow$
Neighborhood-greedy optimization	65.7 %	7.2s
Vanilla optimization	39.0 %	26.5s

Table 3: Ablation studies on optimization strategies. Experiments are conducted using the Adam optimizer with PyTorch framework on a single NVIDIA RTX 4090 GPU (24GB).

Figure 7: Ablation study of displacement interpolation methods. Performance comparison of our affine-invariant interpolation method against mean interpolation, neighborhood interpolation, Inverse Distance Weighting (IDW), and Gaussian Weighted Interpolation (GWI). Our method demonstrates superior measurement accuracy across different interpolation strategies.

Figure 8: Failure cases of our system are demonstrated in scenarios involving plasterboard cracking and specular reflection.

Deformable Image Registration

Deformable image registration aligns images by estimating local spatial transformations that account for anatomical change, motion, and differences between scans.

See all 77 papers

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation
Mar 10, 2026Yinrui Ren, Jinjing Zhu, Kanghao Chen +8Event-Based Vision Depth Estimation
Hong Kong University of Science and Technology (Guangzhou) · South China Normal University · Chinese University of Hong Kong
MotionDLO: Hybrid Event- and Frame-Based Tracking of Deformable Linear Objects
Aug 23, 2026Annalena Hartmann, Priyamvada Ajithkumar, Patrick Bründl +1Visual Object Tracking Deformable Shape Reconstruction
Institute for Factory Automation and Production Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Fürther Straße 246b, 90429 Nürnberg, Germany
Motion-aware Event Suppression for Event Cameras
Feb 26, 2026Roberto Pellerito, Nico Messikommer, Giovanni Cioffi +2Event-Based Vision
Robotics and Perception Group, University of Zurich, Switzerland

Mar 10, 2026J/K move · Enter open · S save

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li+7

Hong Kong University of Science and Technology (Guangzhou) · South China Normal University · Chinese University of Hong Kong

Event-Based Vision Depth Estimation Monocular Depth Estimation Knowledge Distillation

Open paper AskPDF arXiv

Event cameras offer superior sensitivity to high-speed motion and extreme lighting, making event-based monocular depth estimation a promising approach for robust 3D perception in challenging conditions. However, progress is severely hindered by the scarcity of dense depth annotations. While recent annotation-free approaches mitigate this by distilling knowledge from Vision Foundation Models (VFMs), a critical limitation persists: they process event streams as independent frames. By neglecting the inherent temporal continuity of event data, these methods fail to leverage the rich temporal priors encoded in VFMs, ultimately yielding temporally inconsistent and less accurate depth predictions. To address this, we introduce EventVGGT, a novel framework that explicitly models the event stream as a coherent video sequence. To the best of our knowledge, we are the first to distill spatio-temporal and multi-view geometric priors from the Visual Geometry Grounded Transformer (VGGT) into the event domain. We achieve this via a comprehensive tri-level distillation strategy: (i) Cross-Modal Feature Mixture (CMFM) bridges the modality gap at the output level by fusing RGB and event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT's powerful spatio-temporal representations at the feature level; and (iii) Temporal Consistency Distillation (TCD) enforces cross-frame coherence at the temporal level by aligning inter-frame depth changes. Extensive experiments demonstrate that EventVGGT consistently outperforms existing methods -- reducing the absolute mean depth error at 30m by over 53% on EventScape (from 2.30 to 1.06) -- while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. The code is available at https://github.com/yinruiRen/EventVGGT.

Abstract

Deformable Image Registration

Explore similar work

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

MotionDLO: Hybrid Event- and Frame-Based Tracking of Deformable Linear Objects

Motion-aware Event Suppression for Event Cameras