RGB--Event Object Detection Across Operating Frequencies via Reliability-Supervised Routing and Asymmetric-Resolution Fusion

Peijun Shi; Jiannan Zhao; Xiujin Zhu; Zhenguo Lao; Chee-Onn Chow; Wei Ru Wong

cs.CVOct 19, 2026

RGB--Event Object Detection Across Operating Frequencies via Reliability-Supervised Routing and Asymmetric-Resolution Fusion

Authors: Peijun Shi, Jiannan Zhao, Xiujin Zhu, Zhenguo Lao, Chee-Onn Chow, Wei Ru Wong

Organizations: Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603 Lembah Pantai, Kuala Lumpur, Malaysia. · School of Electrical Engineering, Guangxi University, Nanning 530004, China · Computational Intelligence Laboratory, School of Computer Science, University of Lincoln, Lincoln LN6 7TS, U.K.

Abstract

Autonomous vehicles must detect objects faster than a frame camera delivers images. Event cameras record brightness changes with microsecond resolution, so RGB--event detectors can decide between frames, at an operating frequency, the rate of decisions, far above the frame rate. As this rate rises, each decision receives a sparser event window, and the reliability of the events relative to the frame varies over the scene and over time. Existing detectors weight the sensors per sample and encode the frame at the reduced event resolution. Reliability-supervised routing assigns a fusion weight to every location and supervises it with targets from the losses of two single-modality classifiers used only in training, moving loss-based supervision of fusion weights from whole samples to individual locations. Asymmetric-resolution fusion lets the frame carry the decision where the events fail: it encodes the frame at native resolution and pools every feature level onto the event grid, reusing all pretrained weights. On the DSEC-Det dataset, routing halves the accuracy loss from 20 to 180 Hz, from 6.3 to 3.1 mAP, and lowers it in every seed. The resulting detector, ERA-Det, sets a new state of the art at 63.2 mAP, 3.7 mAP above FlexEvent, leads at every operating frequency, and at 180 Hz is more accurate than FlexEvent at 20 Hz. On two DAVIS346 datasets, the router lowers the frame weight at night and in low light and raises it when the events vanish. ERA-Det upgrades released FlexEvent weights in 5,000 iterations on one GPU.

Figures & tables

Fig. 1: Architecture of ERA-Det. Top: the event histogram and the native-resolution RGB frame are encoded by an RVT encoder and a ResNet-50 with a feature pyramid, asymmetric-resolution fusion maps every RGB level onto the event grid, and reliability-supervised routing fuses the two pyramids at strides 8, 16 and 32 before the PAFPN neck and the YOLOX head. Panel 1: asymmetric-resolution fusion, a depthwise pooling initialized as exact averaging. Panel 2: the per-location router, which corrects the per-sample gate of the base detector at every location. Panel 3: the training-only reliability supervision, whose routing target is computed from two single-modality classifiers and supervises the RGB weight of the router. The inputs are a frame and the following 50 ms of events of the DSEC-Det sequence thun_02_a.

Input	Method	mAP	AP 50	AP 75	AP S	AP M	AP L	AP car	AP ped
E	RVT † [ 11 ]	38.4	58.7	41.3	29.5	50.3	81.7	–	–
E	SAST † [ 12 ]	38.1	60.1	40.0	29.8	48.9	79.7	–	–
E	SSM † [ 13 ]	38.0	55.2	40.6	28.8	52.2	77.8	–	–
E	LEOD † [ 14 ]	41.1	65.2	43.6	35.1	47.3	73.3	–	–
E+F	DAGr-18 † [ 2 ]	37.6	–	–	–	–	–	–	–
E+F	DAGr-34 † [ 2 ]	39.0	–	–	–	–	–	–	–

TABLE I: Comparison with Event-Only and RGB–Event Detectors on the DSEC-Det Test Split, Two Classes. E: Events. F: Frames. † Results Reported in [ 3 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.

Method	Empty	180 Hz	90 Hz	60 Hz	45 Hz	36 Hz	30 Hz	20 Hz	Drop	Retention
FlexEvent Global [ 3 ]	34.20	46.98	55.60	57.69	59.10	60.17	60.46	60.12	13.14	78.1%
FlexEvent Balanced [ 3 ]	37.40	52.64	56.77	58.40	59.32	59.66	59.62	59.06	6.42	89.1%
FlexEvent + ARF	51.51 $\pm$ 3.92	58.07 $\pm$ 1.76	63.00 $\pm$ 1.40	64.41 $\pm$ 1.55	65.11 $\pm$ 1.52	65.52 $\pm$ 1.51	65.41 $\pm$ 1.53	64.37 $\pm$ 1.58	6.29 $\pm$ 0.65	90.2%
ERA-Det	55.55 $\pm$ 3.96	61.34 $\pm$ 1.84	64.24 $\pm$ 1.73	65.43 $\pm$ 1.86	65.77 $\pm$ 1.95	65.98 $\pm$ 1.87	65.68 $\pm$ 1.90	64.40 $\pm$ 1.89	3.07 $\pm$ 0.20	95.2%

TABLE II: Interframe Evaluation on the DSEC-Det Test Split. AP at Each Operating Frequency, Drop from 20 Hz to 180 Hz and Retention at 180 Hz. Empty Denotes Detection Without Events. FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.

Fig. 2: Frequency robustness on the DSEC-Det test split. Left: AP from the Empty setting without events to 20 Hz; the shaded bands show one standard deviation over three seeds. Right: AP drop from 20 Hz to 180 Hz for each seed of ERA-Det and of the same detector without routing.

Method	Global	Empty	180 Hz	90 Hz	60 Hz	45 Hz	36 Hz	30 Hz	20 Hz	Drop	Retention
FlexEvent Global [ 3 ]	53.40	27.87	41.85	47.70	49.68	51.95	52.95	54.29	54.62	12.77	76.6%
FlexEvent Balanced [ 3 ]	51.76	33.99	45.57	50.82	52.66	53.18	53.86	54.19	53.38	7.81	85.4%
FlexEvent + ARF	55.12 $\pm$ 0.99	38.23 $\pm$ 2.80	48.58 $\pm$ 0.38	52.94 $\pm$ 1.17	53.90 $\pm$ 1.35	55.28 $\pm$ 1.30	55.68 $\pm$ 1.37	56.64 $\pm$ 1.12	56.26 $\pm$ 0.94	7.68 $\pm$ 0.59	86.3%
ERA-Det	55.71 $\pm$ 0.98	42.42 $\pm$ 3.42	52.01 $\pm$ 0.73	54.52 $\pm$ 0.57	55.16 $\pm$ 0.72	56.32 $\pm$ 0.56	56.45 $\pm$ 0.73	57.11 $\pm$ 0.83	56.90 $\pm$ 1.10	4.89 $\pm$ 0.62	91.4%

TABLE III: Global and Interframe Evaluation with the Eight-Class Protocol on the DSEC-Det Test Split: Pedestrian, Rider, Car, Bus, Truck, Bicycle and Motorcycle. Global Denotes mAP at the Annotated Timestamps. The Values of FlexEvent Balanced Are Those of the Official Release. FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.

Configuration	Global	Empty	180 Hz	20 Hz	Drop
FlexEvent Global	59.53	34.20	46.98	60.12	13.14
+ fine-tuning	59.56 $\pm$ 0.55	–	–	–	–
+ ARF	63.41 $\pm$ 1.55	51.51 $\pm$ 3.92	58.07 $\pm$ 1.76	64.37 $\pm$ 1.58	6.29 $\pm$ 0.65
+ ARF + RSR	63.18 $\pm$ 1.86	55.55 $\pm$ 3.96	61.34 $\pm$ 1.84	64.40 $\pm$ 1.89	3.07 $\pm$ 0.20

TABLE IV: Contribution of Each Component on the DSEC-Det Test Split. ARF: Asymmetric-Resolution Fusion. RSR: Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds.

Fig. 3: Detections on two scenes of the DSEC-Det test split under the interframe protocol: a day scene of thun_02_a, top, and a night scene of zurich_city_12_a, bottom. Each row pairs the same RGB frame with 50 ms of events, 20 Hz, with 5.6 ms of events, 180 Hz, or with no events, Empty. Columns, from left: events of the primary window, positive in red and negative in blue; detections of FlexEvent Global and of ERA-Det with a score of at least 0.3, cars in green and pedestrians in cyan, with missed ground-truth objects as red dashed boxes and the number of detected ground-truth objects at the upper left; the RGB weight $g^{8}_{R}(\mathbf{x})$ of ERA-Det at stride 8 over the frame, orange where the frame dominates and blue where the events dominate. The value at the lower right of the FlexEvent panels is its RGB weight at stride 8, shared by all locations.

Variant	Global	Empty	180 Hz	90 Hz	20 Hz	Drop
Full routing	64.65	57.41	62.62	65.86	65.90	3.28
Without per-location weights	63.92	56.07	60.75	64.74	65.22	4.47
Without routing supervision	64.53	56.09	61.54	65.28	65.56	4.02
Without statistics $\boldsymbol{\pi}^{s}$	64.79	57.78	62.86	66.05	65.94	3.08
Without routing	61.66	47.40	56.41	61.47	62.60	6.19

TABLE V: Analysis of Reliability-Supervised Routing on the DSEC-Det Test Split, First Seed. All Variants Use Asymmetric-Resolution Fusion.

Model	Setting	$s=8$	$s=16$	$s=32$
FlexEvent Global	20 Hz	0.41	0.45	0.48
	180 Hz	0.43	0.48	0.49
	Empty	0.40	0.46	0.49
ERA-Det	20 Hz	0.55	0.49	0.46
	180 Hz	0.62	0.55	0.48
	Empty	0.58	0.54	0.48

TABLE VI: Mean RGB Weight

g^{s}_{R}

per Stride

s

under the Interframe Protocol on the DSEC-Det Test Split, First Seed of ERA-Det. FlexEvent Global Uses One Weight per Sample.

Method	AP 50	mAP	Day	Night
SENet † [ 45 ]	81.6	42.4	43.7	37.0
ECA-Net † [ 46 ]	82.2	40.8	42.2	36.1
CBAM † [ 47 ]	81.9	42.8	44.2	38.0
SA-Gate † [ 48 ]	82.0	43.4	44.9	38.0
DCF † [ 49 ]	83.4	42.5	43.4	39.0
SPNet † [ 50 ]	84.7	43.3	44.9	37.1

TABLE VII: Comparison on the PKU-DDD17-Car Test Split, Car Class. Day and Night: mAP on the 704 Day and 209 Night Test Samples. † Results Reported in [ 16 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. The Last Three Rows Are Fine-Tuned from Their DSEC-Det Models; Mean and Standard Deviation over Three Seeds.

Input	Method	mAP	AP 50	AP 75	AP S	AP M	Normal	Low light	Motion blur
E	VIT-S5 † [ 13 ]	23.2	46.6	–	–	–	–	–	–
E	RVT † [ 11 ]	25.6	50.3	–	–	–	–	–	–
F	YOLOX † [ 38 ]	27.4	50.9	–	–	–	–	–	–
F	VIT-S5 † [ 13 ]	28.2	52.2	–	–	–	–	–	–
F	RVT † [ 11 ]	27.9	53.0	–	–	–	–	–	–
E+F	JDF † [ 7 ]	–	44.2	–	–	–	–	–	–

TABLE VIII: Comparison on the PKU-DAVIS-SOD Test Split with Paired Frames and Events. E: Events. F: Frames. Normal, Low Light and Motion Blur: mAP per Test Condition. † Results Reported in [ 18 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. The Last Three Rows Are Fine-Tuned from Their DSEC-Det Eight-Class Models; Mean and Standard Deviation over Three Seeds.

	Event window						Frame rate $25/N$ Hz
Method	50 ms	20 ms	10 ms	5 ms	0 ms	Drop	$N=1$	$N=2$	$N=4$	$N=6$	$N=8$	$N=10$	MDrop
SODFormer † [ 8 ]	–	–	–	–	–	–	20.7	18.6	15.4	14.2	12.6	11.7	9.0
EFNet † [ 55 ]	–	–	–	–	–	–	26.4	24.8	21.6	19.8	16.7	15.9	10.5
AFNet † [ 56 ]	–	–	–	–	–	–	27.5	25.3	21.2	20.0	16.5	15.1	12.4
FF-KDT † [ 57 ]	–	–	–	–	–	–	28.5	26.7	22.6	21.1	17.9	16.5	12.0
ReNet † [ 15 ]	–	–	–	–	–	–	28.8	27.0	22.8	21.4	18.3	17.0	11.8

TABLE IX: Event Window and Frame Rate on the PKU-DAVIS-SOD Test Split, mAP. Event Window: the Frame of the Decision with the Last 50 to 0 ms of Events. Frame Rate: the Mismatch Protocol of FAOD, Frames at

25/N

Hz and Events at 25 Hz; MDrop Is the Largest Loss Relative to

N=1

. † Results Reported in [ 18 ] . First Seed for the Last Three Rows.

Model	Setting	$s=8$	$s=16$	$s=32$
FlexEvent	Normal	0.43	0.52	0.52
	Low light	0.42	0.47	0.49
	Motion blur	0.42	0.50	0.51
	All	0.43	0.51	0.51
	All, no events	0.49	0.57	0.55
FlexEvent + ARF	Normal	0.46	0.51	0.51

TABLE X: Mean RGB Weight

g^{s}_{R}

at Object Locations per Stride

s

on the PKU-DAVIS-SOD Test Split, Mean over Three Seeds. All: All Test Conditions. FlexEvent and FlexEvent + ARF Use One Weight per Sample.

Fig. 4: Detections on three scenes of the PKU-DAVIS-SOD test split, one per test condition: normal, 005_test_normal, top; low light, 002_test_low_light, middle; and motion blur, 009_test_motion_blur, bottom. Each row pairs the same frame with the 50 ms of events that precede it, left, and with no events, right. Columns, from left: events of the window, positive in red and negative in blue; detections of FlexEvent + ARF and of ERA-Det with a score of at least 0.3, cars in green, pedestrians in cyan and two-wheelers in magenta, with missed ground-truth objects as red dashed boxes and the number of detected ground-truth objects at the upper left; the RGB weight $g^{8}_{R}(\mathbf{x})$ of ERA-Det at stride 8 over the frame, orange where the frame dominates and blue where the events dominate. The value at the lower right of the FlexEvent + ARF panels is its RGB weight at stride 8, shared by all locations.

Model	Params/M	GFLOPs	Latency/ms	mAP
FlexEvent Global	45.59	36	19.1	59.53
+ ARF	45.59	95	19.2	63.41
ERA-Det	45.64	95	22.0	63.18

TABLE XI: Cost per Time Step at Batch Size 1 and Half Precision on an NVIDIA RTX 6000D GPU, with Global mAP.

RGB-Event Sensor Fusion

See all 15 papers

AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery
May 23, 2026Raju Imandi, Chethana B, Bharatesh Chakravarthi +3Event-Based Object Detection Object Detection
SRM University AP, India · Aptiv, Bengaluru, India · Arizona State University, USA +2
Beyond Symmetric Fusion: Exploiting Task-Dependent Modality Strengths for RGB-Event Small Object Detection
Aug 2, 2026Ziheng Wang, Chaolang Li, Yutong Yang +5Small Object Detection Detection Transformer
Sun Yat-sen University · Shenzhen University
Reusing Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection
May 31, 2026Yefeng WuMultimodal Robustness Adaptive Model Routing

May 23, 2026J/K move · Enter open · S save

AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery

Raju Imandi, Chethana B, Bharatesh Chakravarthi, Yong-Guk Kim+2

SRM University AP, India · Aptiv, Bengaluru, India · Arizona State University, USA+2

Event-Based Object Detection Object Detection Multimodal Fusion Multimodal Sensor Fusion RGB-Event Sensor Fusion

Open paper AskPDF arXiv

Detecting objects reliably under extreme low-light conditions is an open problem in computer vision, with practical urgency in applications ranging from nighttime surveillance to search-and-rescue robotics. Conventional RGB cameras degrade sharply at low photon flux, while event cameras which record asynchronous per-pixel brightness changes at microsecond resolution and high dynamic range provide complementary structural cues that are largely illumination-invariant. We present AdaFuse-Det, a dual-stream framework that fuses CLAHE-enhanced RGB frames with voxelized event tensors through an Adaptive Cross-Modal Fusion (ACMF) module grounded in minimum-variance linear estimation theory. We formally show that the learned attention map asymptotically recovers the Gauss-Markov optimal fusion weights, and establish event conservation and temporal resolution bounds for the voxelization stage. On the LLE-VOS benchmark, AdaFuse-Det achieves a Recall of

65.54\%

, Precision of

53.85\%

, and F1-Score of

59.12\%

under severe illumination degradation, outperforming single-modality detectors in recall by a margin that reflects the theoretically predicted illumination-adaptation behavior.

Abstract

RGB-Event Sensor Fusion

Explore similar work

AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery

Beyond Symmetric Fusion: Exploiting Task-Dependent Modality Strengths for RGB-Event Small Object Detection

Reusing Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection