Depth Hypothesis Guided Iterative Refinement for Event-Image Monocular Depth Estimation
Organizations: Southeast University
Abstract
Event cameras hold excellent dynamic properties, showing great potential for monocular depth estimation (MDE). However, existing methods mainly improve performance by optimizing contextual features, but still struggle with the ill-posed and nonlinear nature of direct full-depth regression. In this paper, we propose HypoDepth, the first event-image monocular depth iterative refinement framework. By introducing a discrete Depth Hypothesis Volume (DHV), we transform the depth regression problem into a constrained depth search task. Specifically, we construct a 3D cost volume between the DHV features and contextual features and perform a multi-scale correlation search to guide stable residual optimization. This lightweight cost volume enables efficient global-to-local refinement across multi-resolution. Our method outperforms existing approaches on DSEC and MVSEC with state-of-the-art results and strong zero-shot generalization. Meanwhile, our tiny model achieves an excellent balance between accuracy and efficiency, enabling real-time performance on resource-limited devices.
Figures & tables
| Input | Method | 1 | 2 | 3 | Abs Rel | RMSE | RMSE log | FLOPs (G) | Param (M) | Infer time (ms) |
| E | E2Depth † [ 12 ] | 0.447 | 0.755 | 0.915 | 0.344 | 13.467 | 0.376 | 93 | 10.7 | 2.3 |
| EReformer † [ 24 ] | 0.486 | 0.776 | 0.892 | 0.387 | 13.954 | 0.395 | 36 | 41.8 | 50.1 | |
| EReformer-C ‡ [ 24 ] | 0.463 | 0.755 | 0.899 | 0.399 | 8.635 | 0.382 | 109 | 41.8 | 144.3 | |
| DepthAnyEvent [ 1 ] | 0.585 | 0.860 | 0.955 | 0.277 | 11.117 | 0.292 | 33 | 24.2 | 1.8 | |
| DepthAnyEvent-R [ 1 ] | 0.592 | 0.900 | 0.971 | 0.252 | 9.824 | 0.268 | 39 | 25.5 | 12.9 | |
| Ours-E-T | 0.755 | 0.934 | 0.995 | 0.168 | 5.377 | 0.215 | 5 | 1.4 | 11.8 |
| Input | Method | Outdoor_day1 | Outdoor_night1 | ||||||||||
| 1 | 2 | a3 | Abs Rel | RMSE | RMSE log | 1 | 2 | 3 | Abs Rel | RMSE | RMSE log | ||
| E | E2Depth [ 12 ] | 0.567 | 0.772 | 0.876 | 0.346 | 8.564 | 0.421 | 0.408 | 0.615 | 0.754 | 0.591 | 11.21 | 0.646 |
| EReformer [ 24 ] | 0.664 | 0.831 | 0.923 | 0.271 | - | 0.333 | 0.547 | 0.753 | 0.881 | 0.317 | - | 0.415 | |
| HMNet [ 11 ] | 0.690 | 0.849 | 0.930 | 0.254 | 6.890 | 0.319 | 0.513 | 0.714 | 0.837 | 0.323 | 9.008 | 0.482 | |
| E + I | SRFNet [ 28 ] | 0.637 | 0.810 | 0.900 | 0.268 | 8.453 | 0.375 | 0.433 | 0.662 | 0.8 | 0.371 | 11.469 | 0.521 |
| RAMNet [ 8 ] | 0.541 | 0.778 | 0.877 | 0.303 | 8.526 | 0.424 | 0.296 | 0.502 | 0.635 | 0.583 | 13.340 | 0.830 | |
| Dataset | Method | 1 | 2 | 3 | Abs Rel | RMSE | RMSE log |
| MVSEC outdoor_day1 | EReFormer-C [ 24 ] | 0.157 | 0.256 | 0.348 | 0.580 | 14.510 | 1.336 |
|---|---|---|---|---|---|---|---|
| PCDepth [ 22 ] | 0.226 | 0.446 | 0.607 | 0.471 | 12.650 | 0.874 | |
| Ours-B | 0.248 | 0.486 | 0.640 | 0.432 | 12.480 | 0.809 | |
| MVSEC outdoor_night1 | EReFormer-C [ 24 ] | 0.117 | 0.216 | 0.300 | 0.621 | 16.35 | 1.472 |
| PCDepth [ 22 ] | 0.197 | 0.400 | 0.591 | 0.544 | 12.820 | 0.795 | |
| Ours-B | 0.241 | 0.471 | 0.653 | 0.490 | 12.300 | 0.723 |
| Settings | 1 | 2 | 3 | Abs Rel | RMSE | RMSE log | FLOPs (G) | Pram (M) | |
| Components | W/o EIFusion | 0.877 | 0.978 | 0.995 | 0.110 | 3.885 | 0.148 | 80 | 57.6 |
| W/o CA in decoder | 0.869 | 0.977 | 0.996 | 0.111 | 4.074 | 0.153 | 80 | 57.3 | |
| W/o GRU | 0.858 | 0.978 | 0.995 | 0.115 | 4.234 | 0.158 | 69 | 56.8 | |
| W/o context in decoder | 0.876 | 0.977 | 0.995 | 0.112 | 3.921 | 0.150 | 73 | 57.1 | |
| W/o multi-scale in lookup | 0.875 | 0.977 | 0.995 | 0.110 | 3.969 | 0.151 | 76 | 57.4 | |
| W/o DHV | 0.873 | 0.977 | 0.995 | 0.113 | 3.964 | 0.152 | 81 | 57.4 |
| Dataset | Input | Method | 1 | Abs Rel | RMSE |
| DSEC | I | Ours-I-B | 0.859 | 0.116 | 4.183 |
| E | Ours-E-B | 0.839 | 0.129 | 4.442 | |
| E+I | Ours-B | 0.901 | 0.099 | 3.583 | |
| MVSEC_od1 / on1 | I | Ours-I-B | 0.692 / 0.534 | 0.233 / 0.330 | 7.020/8.537 |
| E | Ours-E-B | 0.690 / 0.600 | 0.243 / 0.289 | 6.732 / 7.201 | |
| E+I | Ours-T | 0.682 / 0.573 | 0.260 / 0.312 | 7.157 / 7.597 |
| Duration (ms) | 1 | Abs Rel | RMSE |
| 20 | 0.885 | 0.105 | 3.857 |
| 50 | 0.887 | 0.106 | 3.705 |
| 100 | 0.876 | 0.112 | 3.869 |
| Depth range | 1 | Abs Rel | RMSE |
| 0.884 | 0.107 | 3.833 | |
| 0.885 | 0.105 | 3.857 | |
| 0.793 | 0.141 | 4.533 |
| Ours-B | Ours-E-B | Ours-E-S | Ours-E-T | PCDepth-Ours | Ours scaled-down | |
| Backbone | Swin-T | Swin-T | Swin-T | ResNet | Swin-T | Swin-T |
| Backbone out dims | (128, 96, 64) | (128, 96, 64) | (96, 64, 32) | (64, 64, 32) | - | (128, 64, 48) |
| EIFusion out dims | (128, 96, 64) | - | - | - | - | (128, 64, 48) |
| DHV embedding dims | (128, 96, 64) | (128, 96, 64) | (96, 64, 32) | (64, 64, 32) | 128 | (128, 64, 48) |
| Iterations | (8, 8, 8) | (8, 8, 8) | (3, 3, 3) | (2, 2, 1) | 8 | (8, 8, 8) |
| values of DHV | (32, 64, 96) | (32, 64, 96) | (32, 64, 96) | (16, 32, 64) | 64 | (32, 64, 96) |
Monocular Depth Estimation
Monocular depth estimation infers scene geometry and distance from a single camera image, enabling 3D perception without stereo or depth sensors.
See all 135 papers