Depth Hypothesis Guided Iterative Refinement for Event-Image Monocular Depth Estimation

Daikun Liu; Teng Wang; Changyin Sun

cs.CVOct 2, 2026

Depth Hypothesis Guided Iterative Refinement for Event-Image Monocular Depth Estimation

Authors: Daikun Liu, Teng Wang, Changyin Sun

Organizations: Southeast University

Abstract

Event cameras hold excellent dynamic properties, showing great potential for monocular depth estimation (MDE). However, existing methods mainly improve performance by optimizing contextual features, but still struggle with the ill-posed and nonlinear nature of direct full-depth regression. In this paper, we propose HypoDepth, the first event-image monocular depth iterative refinement framework. By introducing a discrete Depth Hypothesis Volume (DHV), we transform the depth regression problem into a constrained depth search task. Specifically, we construct a 3D cost volume between the DHV features and contextual features and perform a multi-scale correlation search to guide stable residual optimization. This lightweight cost volume enables efficient global-to-local refinement across multi-resolution. Our method outperforms existing approaches on DSEC and MVSEC with state-of-the-art results and strong zero-shot generalization. Meanwhile, our tiny model achieves an excellent balance between accuracy and efficiency, enabling real-time performance on resource-limited devices.

Figures & tables

Figure 1: Visualization of the correspondence between contextual features and DHV embedding features. (d) shows the cost volume distribution of the patch in (f), where the depth value of the DHV at the highest similarity point closely matches the predicted depth, and neighboring pixels exhibit similar distributions. (e) visualizes the correlation map, revealing clear spatial structures.

Figure 2: Overview of our proposed HypoDepth . The network architecture comprises three main components: 1) Backbone : The image and event backbones respectively extract multi-resolution contextual features from image frames and event streams. 2) EIFusion . The fusion module adaptively fuses information from both modalities across multiple resolutions to produce unified contextual representations. 3) Depth Decoder . This decoder constructs a Depth Hypothesis Volume (DHV) and computes a 3D cost volume between DHV and the fused contextual features for lookup and geometry encoding. Subsequently, a GRU-based iterative unit progressively predicts residual depth and finally regresses the complete depth map.

Figure 3: Illustration of EIFusion module. (a) The cross-attention-based fusion strategy for low-resolution (1/16 and 1/8 resolution) features. (b) The efficient convolution-based fusion method for high-resolution (1/4 resolution) features.

Figure 4: Illustration of correlation lookup and geometry encoder. (a) Cost volume lookup with correlation pooling. The red arrow indicates the index of the depth value in the DHV that is closest to the previous iteration’s depth updation at pixel $P$ . This position serves as the center for the current correlation lookup $C_{P}$ of the pixel $P$ . (b) Geometry feature encoder for depth update.

Input	Method	$\delta$ 1 $\uparrow$	$\delta$ 2 $\uparrow$	$\delta$ 3 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	FLOPs (G)	Param (M)	Infer time (ms)
E	E2Depth † [ 12 ]	0.447	0.755	0.915	0.344	13.467	0.376	93	10.7	2.3
	EReformer † [ 24 ]	0.486	0.776	0.892	0.387	13.954	0.395	36	41.8	50.1
	EReformer-C ‡ [ 24 ]	0.463	0.755	0.899	0.399	8.635	0.382	109	41.8	144.3
	DepthAnyEvent [ 1 ]	0.585	0.860	0.955	0.277	11.117	0.292	33	24.2	1.8
	DepthAnyEvent-R [ 1 ]	0.592	0.900	0.971	0.252	9.824	0.268	39	25.5	12.9
	Ours-E-T	0.755	0.934	0.995	0.168	5.377	0.215	5	1.4	11.8

Table 1: Evaluation on DSEC [ 9 ] . The best results are in bold, while the second-best ones are underlined. † and ‡ denote the result from DepthAnyEvent [ 1 ] and this paper, respectively. ‘E’ and ‘E+I’ denote event-only and event–image inputs, respectively.

Figure 5: Qualitative results of monocular depth predictions on DSEC [ 9 ] . Notable areas are marked by bounding boxes.

Input	Method	Outdoor_day1						Outdoor_night1
Input	Method	$\delta$ 1 $\uparrow$	$\delta$ 2 $\uparrow$	$\delta$ a3 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	$\delta$ 1 $\uparrow$	$\delta$ 2 $\uparrow$	$\delta$ 3 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$
E	E2Depth [ 12 ]	0.567	0.772	0.876	0.346	8.564	0.421	0.408	0.615	0.754	0.591	11.21	0.646
	EReformer [ 24 ]	0.664	0.831	0.923	0.271	-	0.333	0.547	0.753	0.881	0.317	-	0.415
	HMNet [ 11 ]	0.690	0.849	0.930	0.254	6.890	0.319	0.513	0.714	0.837	0.323	9.008	0.482
E + I	SRFNet [ 28 ]	0.637	0.810	0.900	0.268	8.453	0.375	0.433	0.662	0.8	0.371	11.469	0.521
	RAMNet [ 8 ]	0.541	0.778	0.877	0.303	8.526	0.424	0.296	0.502	0.635	0.583	13.340	0.830

Table 2: Evaluation on MVSEC [ 40 ] . The best results are in bold, while the second-best ones are underlined.

Dataset	Method	$\delta$ 1 $\uparrow$	$\delta$ 2 $\uparrow$	$\delta$ 3 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$
MVSEC outdoor_day1	EReFormer-C [ 24 ]	0.157	0.256	0.348	0.580	14.510	1.336
	PCDepth [ 22 ]	0.226	0.446	0.607	0.471	12.650	0.874
	Ours-B	0.248	0.486	0.640	0.432	12.480	0.809
MVSEC outdoor_night1	EReFormer-C [ 24 ]	0.117	0.216	0.300	0.621	16.35	1.472
	PCDepth [ 22 ]	0.197	0.400	0.591	0.544	12.820	0.795
	Ours-B	0.241	0.471	0.653	0.490	12.300	0.723

Table 3: Zero-shot Generalization on MVSEC [ 40 ] and DSEC [ 9 ] . All methods are trained on EventScape [ 8 ] . The best are in bold.

Figure 6: Abs Rel vs. Inference time (ms) and model size (Param, M). All models are tested with one NVIDIA 4090 GPU. The size of the circle corresponds to the model size, as shown in the legend.

	Settings	$\delta$ 1 $\uparrow$	$\delta$ 2 $\uparrow$	$\delta$ 3 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	FLOPs (G)	Pram (M)
Components	W/o EIFusion	0.877	0.978	0.995	0.110	3.885	0.148	80	57.6
	W/o CA in decoder	0.869	0.977	0.996	0.111	4.074	0.153	80	57.3
	W/o GRU	0.858	0.978	0.995	0.115	4.234	0.158	69	56.8
	W/o context in decoder	0.876	0.977	0.995	0.112	3.921	0.150	73	57.1
	W/o multi-scale in lookup	0.875	0.977	0.995	0.110	3.969	0.151	76	57.4
	W/o DHV	0.873	0.977	0.995	0.113	3.964	0.152	81	57.4

Table 4: Ablation studies on DSEC [ 9 ] with scaled-down model.

Figure 7: The refinement of Abs Rel under different

D

settings.

Figure 8: Visualization of multi-resolution iterative depth refinement and the evolution of error maps across iterations.

Dataset	Input	Method	$\delta$ 1 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$
DSEC	I	Ours-I-B	0.859	0.116	4.183
	E	Ours-E-B	0.839	0.129	4.442
	E+I	Ours-B	0.901	0.099	3.583
MVSEC_od1 / on1	I	Ours-I-B	0.692 / 0.534	0.233 / 0.330	7.020/8.537
	E	Ours-E-B	0.690 / 0.600	0.243 / 0.289	6.732 / 7.201
	E+I	Ours-T	0.682 / 0.573	0.260 / 0.312	7.157 / 7.597

Table 5: More results on MVSEC [ 40 ] , DSEC [ 9 ] , and blurred DSEC. The best are in bold.

Figure 9: Qualitative results on blurred DSEC dataset.

Duration (ms)	$\delta$ 1 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$
20	0.885	0.105	3.857
50	0.887	0.106	3.705
100	0.876	0.112	3.869

Table 6: Ablation studies on the duration of event slices.

Depth range	$\delta$ 1 $\uparrow$	Abs Rel $\downarrow$	RMSE $\downarrow$
$[0.05d_{min},2d_{max}]$	0.884	0.107	3.833
$[d_{min},d_{max}]$	0.885	0.105	3.857
$[2d_{min},0.7d_{max}]$	0.793	0.141	4.533

Table 7: Ablation studies on the depth-range of DHV.

Figure 10: Failure case analysis. (e) shows the cost-volume distribution of the yellow patch in (d). (f) shows the log-normalized error. Large errors appear between the predicted depth and the ground truth. In particular, low-texture or homogeneous regions lead to ambiguous matching patterns in the cost volume, resulting in unreliable depth refinement.

	Ours-B	Ours-E-B	Ours-E-S	Ours-E-T	PCDepth-Ours	Ours scaled-down
Backbone	Swin-T	Swin-T	Swin-T	ResNet	Swin-T	Swin-T
Backbone out dims	(128, 96, 64)	(128, 96, 64)	(96, 64, 32)	(64, 64, 32)	-	(128, 64, 48)
EIFusion out dims	(128, 96, 64)	-	-	-	-	(128, 64, 48)
DHV embedding dims	(128, 96, 64)	(128, 96, 64)	(96, 64, 32)	(64, 64, 32)	128	(128, 64, 48)
Iterations	(8, 8, 8)	(8, 8, 8)	(3, 3, 3)	(2, 2, 1)	8	(8, 8, 8)
$D$ values of DHV	(32, 64, 96)	(32, 64, 96)	(32, 64, 96)	(16, 32, 64)	64	(32, 64, 96)

Table 8: Detailed network parameter settings. PCDepth-Ours only performs iterative refinement at 1/8 resolution. The values in ‘()’ correspond to the parameters for stages 1/16, 1/8, and 1/4, respectively.

Figure 11: More qualitative monocular depth estimation results on DSEC [ 9 ] . The first and fourth rows correspond to daytime scenes, while the second and third rows represent night scenes.

Figure 13: Qualitative results on the outdoor_day1 sequence on MVSEC [ 40 ] .

Monocular Depth Estimation

Monocular depth estimation infers scene geometry and distance from a single camera image, enabling 3D perception without stereo or depth sensors.

See all 135 papers

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation
Mar 10, 2026Yinrui Ren, Jinjing Zhu, Kanghao Chen +8Event-Based Vision Depth Estimation
Hong Kong University of Science and Technology (Guangzhou) · South China Normal University · Chinese University of Hong Kong
Neuromorphic Monocular Depth Estimation with Uncertainty Modeling
May 11, 2026Viktor Bergkvist, Felix Rydell, Per-Erik Forssén +2Event-Based Vision Depth Estimation
Swedish Defence Research Agency Linköping, Sweden · Swedish Defence Research Agency Kista, Sweden · Linköping University Linköping, Sweden
SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation
Sep 29, 2026Thai Duy Nguyen, Addison Lin WangEvent-Based Vision Monocular Depth Estimation
Nanyang Technological University, Singapore.

Mar 10, 2026J/K move · Enter open · S save

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li+7

Hong Kong University of Science and Technology (Guangzhou) · South China Normal University · Chinese University of Hong Kong

Event-Based Vision Depth Estimation Monocular Depth Estimation Knowledge Distillation

Open paper AskPDF arXiv

Event cameras offer superior sensitivity to high-speed motion and extreme lighting, making event-based monocular depth estimation a promising approach for robust 3D perception in challenging conditions. However, progress is severely hindered by the scarcity of dense depth annotations. While recent annotation-free approaches mitigate this by distilling knowledge from Vision Foundation Models (VFMs), a critical limitation persists: they process event streams as independent frames. By neglecting the inherent temporal continuity of event data, these methods fail to leverage the rich temporal priors encoded in VFMs, ultimately yielding temporally inconsistent and less accurate depth predictions. To address this, we introduce EventVGGT, a novel framework that explicitly models the event stream as a coherent video sequence. To the best of our knowledge, we are the first to distill spatio-temporal and multi-view geometric priors from the Visual Geometry Grounded Transformer (VGGT) into the event domain. We achieve this via a comprehensive tri-level distillation strategy: (i) Cross-Modal Feature Mixture (CMFM) bridges the modality gap at the output level by fusing RGB and event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT's powerful spatio-temporal representations at the feature level; and (iii) Temporal Consistency Distillation (TCD) enforces cross-frame coherence at the temporal level by aligning inter-frame depth changes. Extensive experiments demonstrate that EventVGGT consistently outperforms existing methods -- reducing the absolute mean depth error at 30m by over 53% on EventScape (from 2.30 to 1.06) -- while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. The code is available at https://github.com/yinruiRen/EventVGGT.

Abstract

Monocular Depth Estimation

Explore similar work

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

Neuromorphic Monocular Depth Estimation with Uncertainty Modeling

SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation