EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors

Luca Bartolomei; Fabio Tosi; Matteo Poggi; Stefano Mattoccia; Guillermo Gallego

cs.CVApr 2, 2026

EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors

Authors: Luca Bartolomei, Fabio Tosi, Matteo Poggi, Stefano Mattoccia, Guillermo Gallego

Organizations: Advanced Research Center on Electronic System (ARCES) · Department of Computer Science and Engineering (DISI), University of Bologna, Italy · TU Berlin, Robotics Institute Germany, Einstein Center Digital Future, SCIoI Excellence Cluster, Germany

Abstract

We propose EventHub, a novel framework for training deep-event stereo networks without ground truth annotations from costly active sensors, relying instead on standard color images. From these images, we derive either proxy annotations and proxy events through state-of-the-art novel view synthesis techniques, or simply proxy annotations when images are already paired with event data. Using the training set generated by our data factory, we repurpose state-of-the-art stereo models from RGB literature to process event data, obtaining new event stereo models with unprecedented generalization capabilities. Experiments on widely used event stereo datasets support the effectiveness of EventHub and show how the same data distillation mechanism can improve the accuracy of RGB stereo foundation models in challenging conditions such as nighttime scenes.

Figures & tables

Figure 2: Limitations of LiDAR-supervised real-world datasets. Despite their popularity [ 18 , 7 , 84 ] , LiDAR annotations remain sparse (A), poorly capture dynamic scenes (B–C), are prone to reprojection errors (D), and struggle on transparent or reflective surfaces (E).

Figure 3: Framework Overview : We obtain training data through two complementary approaches: (i) Event Data Factory : SVRaster [ 59 ] generates synthetic event stereo pairs and depth labels from sparse RGB images via virtual camera trajectories (left); (ii) Stereo Cross-Modal Distillation : existing RGB stereo models produce proxy depth labels for real event data in calibrated RGB-Event stereo setups (top right). (iii) Both data sources are combined in EventHub to train/adapt event stereo networks (bottom right).

Panel of Figure 4 — Figure 4: Qualitative examples of events and proxy annotations by EventHub. From top to bottom, examples obtained from NeRF-Stereo [ 63 ] , ScanNet++ [ 75 ] through novel view synthesis, and from DSEC [ 18 ] through cross-modal distillation.

Dataset	MIX 1	MIX 2	MIX 3	MIX 4
NeRF-Stereo [ 63 ]	✓	✓		✓
ScanNet++ [ 75 ]		✓		✓
DSEC [ 18 ]			✓	✓

Table 1: Combinations of datasets used by EventHub. Proxy labels are applied to annotate each dataset.

	Training Method	SE-CFF [ 49 ]				EMatch [ 82 ]				E-FoundationStereo				E-StereoAnywhere				Avg Rank
	Training Method	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	Avg Rank
(A)	Photometric [ 63 ]	88.54	71.73	55.35	7.94	92.31	69.17	44.90	3.37	93.85	73.12	49.82	3.65	92.55	72.25	51.65	4.11	6.94
(B)	EV-SceneFlow [ 17 , 45 ]	66.30	50.18	41.47	3.50	71.64	53.67	42.02	3.56	61.80	48.04	41.68	3.10	64.97	49.54	41.74	3.21	6.06
(C)	MIX 1	45.61	25.15	16.54	1.87	47.17	23.23	13.58	1.70	38.70	15.63	8.83	1.39	35.86	15.13	9.03	1.36	5.00
	MIX 2	38.52	18.17	11.02	1.56	41.06	17.81	9.97	1.45	27.20	9.72	5.53	1.04	31.49	12.45	7.32	1.23	4.00
	MIX 3	24.73	8.58	5.08	1.01	31.30	11.14	5.90	1.15	20.99	6.82	4.10	0.89	24.35	8.22	4.76	0.99	2.75

Table 2: In-domain experimental results – DSEC [ 18 ] dataset. We train four event stereo models according to different training protocols, not exploiting LiDAR annotations (A,B,C), compared against in-domain training on DSEC with LiDAR labels (D).

Panel of Figure 5 — Figure 5: Qualitative results on DSEC dataset [ 18 ] . Predictions by E-FoundationStereo trained according to different protocols.

Model	Training Method	M3ED (Day)				M3ED (Night)				M3ED (Indoor)				Avg Rank
Model	Training Method	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	Avg Rank
SE-CFF [ 49 ]	MIX 3	46.84	25.31	16.99	2.81	58.32	36.07	24.45	3.50	51.03	31.43	23.84	4.52	2.50
	MIX 4	35.65	15.18	9.37	1.22	51.57	26.84	15.33	1.70	48.56	27.36	19.33	2.95	1.08
	LiDAR (GT)	58.82	41.41	32.93	3.05	58.94	36.78	23.55	2.20	45.33	28.39	21.59	4.48	2.42
EMatch [ 82 ]	MIX 3	86.18	77.61	72.13	40.36	90.02	84.94	82.26	45.41	76.99	65.23	58.98	15.80	3.00
	MIX 4	43.99	20.87	12.93	2.23	63.69	38.74	26.38	5.03	58.42	32.16	21.48	3.10	1.00

Table 3: Out-of-domain experimental results – M3ED [ 7 ] dataset. We compare the generalization capability of the four event stereo models trained with MIX 3 and MIX 4 against their counterparts trained using DSEC LiDAR labels.

Model	Training Method	MVSEC (Day)				MVSEC (Night)				MVSEC (Indoor)				Avg Rank
Model	Training Method	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	1PE	2PE	3PE	MAE	Avg Rank
SE-CFF [ 49 ]	MIX 3	77.13	53.87	31.12	3.21	78.19	54.75	31.24	3.64	42.14	24.53	18.04	3.30	2.17
	MIX 4	31.99	12.00	6.88	1.11	40.54	19.00	10.13	1.45	29.66	12.34	6.89	1.39	1.00
	LiDAR (GT)	97.82	94.85	90.47	6.12	96.89	92.95	87.14	6.07	46.67	26.37	15.43	1.78	2.83
EMatch [ 82 ]	MIX 3	93.80	80.23	59.05	6.00	91.74	75.37	49.51	4.67	58.44	35.05	24.24	3.02	2.08
	MIX 4	56.29	21.61	6.67	1.39	68.51	40.48	14.92	1.81	46.03	21.40	12.36	1.93	1.00

Table 4: Out-of-domain experimental results – MVSEC [ 84 ] dataset. We compare the generalization capability of the four event stereo models trained with MIX 3 and MIX 4 against their counterparts trained using DSEC LiDAR labels.

Panel of Figure 6 — Figure 6: Qualitative results on M3ED [ 7 ] and MVSEC [ 84 ] . Predictions by the four models trained with LiDAR labels or MIX 4.

Model	Training Method	DSEC (Night)
Model	Training Method	1PE	2PE	3PE	MAE
FoundationStereo (ViT-S)	Author’s Weights [ 69 ]	68.40	47.69	35.80	3.89
FoundationStereo (ViT-S)	MIX 3 (Night)	24.75	8.09	4.25	1.01
FoundationStereo (ViT-L)	Author’s Weights [ 69 ]	30.06	14.87	10.88	1.87
FoundationStereo (ViT-L)	MIX 3 (Night)	25.33	8.48	4.56	1.06
StereoAnywhere (ViT-S)	Author’s Weights [ 6 ]	33.01	14.24	8.93	1.61

Table 5: Experimental results on DSEC night images [ 18 ] – RGB stereo models. Fine-tuning SFMs on proxy labels derived from our event models yields improvements on nighttime images.

Row	$\lambda_{{N-\text{mean}}}$	$\lambda_{N-\text{med}}$	$\lambda_{\text{asc}}$	$\lambda_{\text{sparse}}$	$\lambda_{\text{DAv2}}$	$\lambda_{\text{MASt3R}}$	PSNR	SSIM( $\times 100$ )	MAE (cm)	$\delta\leq 1.25$ (%)
1	-	-	-	-	-	-	33.85	87.36	8.81	93.51
2	0.001	0.001	-	-	-	-	33.25	86.77	9.03	92.43
3	0.001	0.001	0.01	-	-	-	33.25	86.77	9.03	92.47
4	0.001	0.001	0.01	0.01	-	-	33.25	86.77	8.99	92.52
5	0.001	0.001	0.01	0.01	0.01	-	33.19	86.73	6.61	96.23
6	0.001	0.001	0.01	0.01	0.01	0.01	26.86	79.64	38.71	67.31

Table 6: Depth regularization ablation. PSNR values are given in decibels. SSIM values are multiplied by a 100 factor. MAE values are reported in centimeters.

Model	PSNR $\uparrow$	MAE (cm) $\downarrow$	$\delta\leq 1.25$ (%) $\uparrow$	Setup Time (min/scene) $\downarrow$	FPS $\uparrow$
Depth Anything v3 [ 40 ]	19.19	41.94	53.92	$\sim$ 1	165
Instant-NGP [ 48 ]	29.21	24.68	82.88	$\sim$ 8	5
3DGS [ 31 ]	32.51	22.36	76.23	$\sim$ 20	165
SVRaster [ 59 ]	33.37	6.38	96.44	$\sim$ 20	143

Table 7: Comparison between different NVS engines. SVRaster achieves the best trade-off between rendering quality, setup time and rendering speed.

Confidence	Threshold	MAE (cm)	$\delta\leq 1.25$ (%)	Density (%)
-	-	6.38	96.44	100.00
$\mathbf{C}_{\text{AO}}$	0.35	6.26	96.60	95.45
$\mathbf{C}_{\text{Vsize}}$	0.75	6.23	96.57	97.42
$\mathbf{C}_{\text{AO}}$	0.40	6.22	96.66	92.04
$\mathbf{C}_{\text{Vsize}}$	0.80	6.15	96.61	95.56
$\mathbf{C}_{\text{AO}}$	0.45	6.20	96.72	87.38

Table 8: Confidence threshold study. Comparison between ambient occlusion confidence

\mathbf{C}_{\text{AO}}

[ 63 ] and our voxel-based confidence

\mathbf{C}_{\text{Vsize}}

on ScanNet++ [ 75 ] . MAE values are reported in centimeters.

Training Method	SE-CFF				E-FoundationStereo
Training Method	1PE $\downarrow$	2PE $\downarrow$	3PE $\downarrow$	MAE $\downarrow$	1PE $\downarrow$	2PE $\downarrow$	3PE $\downarrow$	MAE $\downarrow$
MIX 3 (SVRaster [ 59 ] )	24.73	8.58	5.08	1.01	20.99	6.82	4.10	0.89
MIX 3 (Depth Anything v3 [ 40 ] )	74.35	47.82	30.63	3.18	71.37	41.01	23.99	2.54
EV-SceneFlow [ 17 , 45 ]	66.30	50.18	41.47	3.50	61.80	48.04	41.68	3.10
EV-(SceneFlow+TartanAir) [ 17 , 45 , 65 ]	57.78	33.01	19.75	2.17	41.86	23.13	16.58	1.76

Table 9: Further in-domain experimental results – DSEC dataset [ 18 ] . On top: comparison between SVRaster and Depth Anything v3 generated data. At the bottom: results by extending the synthetic data used to generate proxy events.

Model	Parameters (M)	FLOPs (G)	Runtime (ms)	Peak Memory (MB)
SE-CFF [ 49 ]	2.97	85.98	46.27	379.13
EMatch [ 82 ]	6.71	501.95	115.20	3090.49
E-StereoAnywhere	39.96	1566.58	219.81	1479.82
E-FoundationStereo	60.09	4445.51	280.11	1525.13

Table 10: Hardware analysis on DSEC dataset [ 18 ] . Measurements taken on a X GPU.

Panel of Figure 7 — Figure 7: Evaluation on DSEC after different numbers of training steps.

Panel of Figure 8 — Figure 8: Global trajectory construction.

Panel of Figure 9 — Figure 9: Qualitative examples of training data generated by EventHub on DSEC [ 18 ] . (a) left color image, (b) left event frame, (c) proxy disparity labels predicted by FoundationStereo [ 69 ] and projected on the event frame.

Panel of Figure 10 — Figure 10: Qualitative examples of training data generated by EventHub on NeRFStereo [ 63 ] . (a) rendered color image, (b) rendered event frame, (c) rendered depth and (d) confidence (the brighter, the higher the confidence in the estimated depth values), and (e) rendered depth masked according to confidence thresholding.

Panel of Figure 11 — Figure 11: Qualitative examples of training data generated by EventHub on ScanNet++ [ 75 ] . (a) rendered color image, (b) rendered event frame, (c) rendered depth and (d) confidence, (e) rendered depth masked according to confidence thresholding.

Panel of Figure 12 — Figure 12: Qualitative results on DSEC [ 18 ] dataset. Predictions by the four models trained with LiDAR labels, MIX 3 or MIX 4.

Panel of Figure 13 — Figure 13: Qualitative results on M3ED [ 7 ] dataset. Predictions by the four models trained with LiDAR labels, MIX 3 or MIX 4.

Panel of Figure 14 — Figure 14: Qualitative results on MVSEC [ 84 ] dataset. Predictions by the four models trained with LiDAR labels, MIX 3 or MIX 4.

Panel of Figure 15 — Figure 15: Improved RGB FoundationStereo at Night. Qualitative comparison on the zurich_city_09_d night sequence. (a) left RGB, (b) prediction by baseline FoundationStereo VIT-L, and (c) its fine-tuned counterpart using proxy labels from E-FoundationStereo VIT-S.

Panel of Figure 16 — Figure 16: Improved RGB FoundationStereo at Night. Qualitative comparison on the zurich_city_10_b night sequence. (a) left RGB, (b) prediction by baseline FoundationStereo VIT-S, and (c) its fine-tuned counterpart using proxy labels from E-FoundationStereo VIT-S.

Event-Based Vision

Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.

See all 232 papers

Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision
Jul 26, 2026Jianing Li, Yunjian Zhang, Haiqian Han +2Event-Based Vision Depth Estimation
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, P.R. China
REALM: An RGB- and Event-Aligned Latent Manifold for Cross-Modal Perception
Apr 30, 2026Vincenzo Polizzi, David B. Lindell, Jonathan KellyMultimodal Foundation Models Event-Based Vision
University of Toronto, Robotics Institute · University of Toronto, Department of Computer Science
EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation
Mar 10, 2026Yinrui Ren, Jinjing Zhu, Kanghao Chen +8Event-Based Vision Depth Estimation
Hong Kong University of Science and Technology (Guangzhou) · South China Normal University · Chinese University of Hong Kong

Jul 26, 2026J/K move · Enter open · S save

Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision

Jianing Li, Yunjian Zhang, Haiqian Han, Kangyao Huang+1

Department of Computer Science and Technology, Tsinghua University, Beijing 100084, P.R. China

Event-Based Vision Depth Estimation Stereo Matching Stereo Vision

Open paper AskPDF arXiv

Conventional frame-based imaging for active stereo systems has encountered major challenges in fast-motion scenarios. However, how to design a novel paradigm for ultrafast depth sensing remains an open issue. In this paper, we propose a novel problem setting, namely active event-based stereo vision, which attempts to integrate binocular event cameras and an infrared 2D pattern projector for high-speed dense depth sensing. Technically, we first build a stereo camera prototype system and present a real-world dataset with over 21.5k spatiotemporal synchronized labels at 15 Hz, while also establishing a realistic synthetic dataset with stereo event streams and 23.8k synchronized labels at 20 Hz. Then, we propose ActiveEventNet+, a lightweight yet effective event-based stereo matching neural network that learns to generate high-quality dense disparity maps from stereo event streams with low latency. Our ActiveEventNet+ mainly involves three innovations: incorporating lightweight blocks into event-based stereo matching frameworks, designing a novel cost volume with dynamic interactions between stereo pairs, and presenting an effective temporal consistency architecture to fully use rich temporal cues in event streams. The results show that our ActiveEventNet+ outperforms state-of-the-art methods while significantly reducing computational complexity. Our solution offers superior depth sensing performance compared to conventional frame-based stereo cameras in high-speed scenes. In particular, the lightweight ActiveEventNet enables the prototype system to achieve real-time processing at speeds up to 150 FPS. We believe that this novel active event-based stereo vision paradigm can provide new insights into the design of future high-speed depth sensing camera systems. Our dataset and code can be available at https://github.com/jianing-li/active_event_based_stereo.

Abstract

Event-Based Vision

Explore similar work

Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision

REALM: An RGB- and Event-Aligned Latent Manifold for Cross-Modal Perception

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation