Organizations: Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603 Lembah Pantai, Kuala Lumpur, Malaysia. · School of Electrical Engineering, Guangxi University, Nanning 530004, China · Computational Intelligence Laboratory, School of Computer Science, University of Lincoln, Lincoln LN6 7TS, U.K.
Autonomous vehicles must detect objects faster than a frame camera delivers images. Event cameras record brightness changes with microsecond resolution, so RGB--event detectors can decide between frames, at an operating frequency, the rate of decisions, far above the frame rate. As this rate rises, each decision receives a sparser event window, and the reliability of the events relative to the frame varies over the scene and over time. Existing detectors weight the sensors per sample and encode the frame at the reduced event resolution. Reliability-supervised routing assigns a fusion weight to every location and supervises it with targets from the losses of two single-modality classifiers used only in training, moving loss-based supervision of fusion weights from whole samples to individual locations. Asymmetric-resolution fusion lets the frame carry the decision where the events fail: it encodes the frame at native resolution and pools every feature level onto the event grid, reusing all pretrained weights. On the DSEC-Det dataset, routing halves the accuracy loss from 20 to 180 Hz, from 6.3 to 3.1 mAP, and lowers it in every seed. The resulting detector, ERA-Det, sets a new state of the art at 63.2 mAP, 3.7 mAP above FlexEvent, leads at every operating frequency, and at 180 Hz is more accurate than FlexEvent at 20 Hz. On two DAVIS346 datasets, the router lowers the frame weight at night and in low light and raises it when the events vanish. ERA-Det upgrades released FlexEvent weights in 5,000 iterations on one GPU.
Figures & tables
Fig. 1: Architecture of ERA-Det. Top: the event histogram and the native-resolution RGB frame are encoded by an RVT encoder and a ResNet-50 with a feature pyramid, asymmetric-resolution fusion maps every RGB level onto the event grid, and reliability-supervised routing fuses the two pyramids at strides 8, 16 and 32 before the PAFPN neck and the YOLOX head. Panel 1: asymmetric-resolution fusion, a depthwise pooling initialized as exact averaging. Panel 2: the per-location router, which corrects the per-sample gate of the base detector at every location. Panel 3: the training-only reliability supervision, whose routing target is computed from two single-modality classifiers and supervises the RGB weight of the router. The inputs are a frame and the following 50 ms of events of the DSEC-Det sequence thun_02_a.
Input
Method
mAP
AP 50
AP 75
AP S
AP M
AP L
AP car
AP ped
E
RVT † [ 11 ]
38.4
58.7
41.3
29.5
50.3
81.7
–
–
E
SAST † [ 12 ]
38.1
60.1
40.0
29.8
48.9
79.7
–
–
E
SSM † [ 13 ]
38.0
55.2
40.6
28.8
52.2
77.8
–
–
E
LEOD † [ 14 ]
41.1
65.2
43.6
35.1
47.3
73.3
–
–
E+F
DAGr-18 † [ 2 ]
37.6
–
–
–
–
–
–
–
E+F
DAGr-34 † [ 2 ]
39.0
–
–
–
–
–
–
–
TABLE I: Comparison with Event-Only and RGB–Event Detectors on the DSEC-Det Test Split, Two Classes. E: Events. F: Frames. † Results Reported in [ 3 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.
Method
Empty
180 Hz
90 Hz
60 Hz
45 Hz
36 Hz
30 Hz
20 Hz
Drop
Retention
FlexEvent Global [ 3 ]
34.20
46.98
55.60
57.69
59.10
60.17
60.46
60.12
13.14
78.1%
FlexEvent Balanced [ 3 ]
37.40
52.64
56.77
58.40
59.32
59.66
59.62
59.06
6.42
89.1%
FlexEvent + ARF
51.51 ± 3.92
58.07 ± 1.76
63.00 ± 1.40
64.41 ± 1.55
65.11 ± 1.52
65.52 ± 1.51
65.41 ± 1.53
64.37 ± 1.58
6.29 ± 0.65
90.2%
ERA-Det
55.55 ± 3.96
61.34 ± 1.84
64.24 ± 1.73
65.43 ± 1.86
65.77 ± 1.95
65.98 ± 1.87
65.68 ± 1.90
64.40 ± 1.89
3.07 ± 0.20
95.2%
TABLE II: Interframe Evaluation on the DSEC-Det Test Split. AP at Each Operating Frequency, Drop from 20 Hz to 180 Hz and Retention at 180 Hz. Empty Denotes Detection Without Events. FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.
Fig. 2: Frequency robustness on the DSEC-Det test split. Left: AP from the Empty setting without events to 20 Hz; the shaded bands show one standard deviation over three seeds. Right: AP drop from 20 Hz to 180 Hz for each seed of ERA-Det and of the same detector without routing.
Method
Global
Empty
180 Hz
90 Hz
60 Hz
45 Hz
36 Hz
30 Hz
20 Hz
Drop
Retention
FlexEvent Global [ 3 ]
53.40
27.87
41.85
47.70
49.68
51.95
52.95
54.29
54.62
12.77
76.6%
FlexEvent Balanced [ 3 ]
51.76
33.99
45.57
50.82
52.66
53.18
53.86
54.19
53.38
7.81
85.4%
FlexEvent + ARF
55.12 ± 0.99
38.23 ± 2.80
48.58 ± 0.38
52.94 ± 1.17
53.90 ± 1.35
55.28 ± 1.30
55.68 ± 1.37
56.64 ± 1.12
56.26 ± 0.94
7.68 ± 0.59
86.3%
ERA-Det
55.71 ± 0.98
42.42 ± 3.42
52.01 ± 0.73
54.52 ± 0.57
55.16 ± 0.72
56.32 ± 0.56
56.45 ± 0.73
57.11 ± 0.83
56.90 ± 1.10
4.89 ± 0.62
91.4%
TABLE III: Global and Interframe Evaluation with the Eight-Class Protocol on the DSEC-Det Test Split: Pedestrian, Rider, Car, Bus, Truck, Bicycle and Motorcycle. Global Denotes mAP at the Annotated Timestamps. The Values of FlexEvent Balanced Are Those of the Official Release. FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds for the Last Two Rows.
Configuration
Global
Empty
180 Hz
20 Hz
Drop
FlexEvent Global
59.53
34.20
46.98
60.12
13.14
+ fine-tuning
59.56 ± 0.55
–
–
–
–
+ ARF
63.41 ± 1.55
51.51 ± 3.92
58.07 ± 1.76
64.37 ± 1.58
6.29 ± 0.65
+ ARF + RSR
63.18 ± 1.86
55.55 ± 3.96
61.34 ± 1.84
64.40 ± 1.89
3.07 ± 0.20
TABLE IV: Contribution of Each Component on the DSEC-Det Test Split. ARF: Asymmetric-Resolution Fusion. RSR: Reliability-Supervised Routing. Mean and Standard Deviation over Three Seeds.
Fig. 3: Detections on two scenes of the DSEC-Det test split under the interframe protocol: a day scene of thun_02_a, top, and a night scene of zurich_city_12_a, bottom. Each row pairs the same RGB frame with 50 ms of events, 20 Hz, with 5.6 ms of events, 180 Hz, or with no events, Empty. Columns, from left: events of the primary window, positive in red and negative in blue; detections of FlexEvent Global and of ERA-Det with a score of at least 0.3, cars in green and pedestrians in cyan, with missed ground-truth objects as red dashed boxes and the number of detected ground-truth objects at the upper left; the RGB weight gR8(x) of ERA-Det at stride 8 over the frame, orange where the frame dominates and blue where the events dominate. The value at the lower right of the FlexEvent panels is its RGB weight at stride 8, shared by all locations.
Variant
Global
Empty
180 Hz
90 Hz
20 Hz
Drop
Full routing
64.65
57.41
62.62
65.86
65.90
3.28
Without per-location weights
63.92
56.07
60.75
64.74
65.22
4.47
Without routing supervision
64.53
56.09
61.54
65.28
65.56
4.02
Without statistics πs
64.79
57.78
62.86
66.05
65.94
3.08
Without routing
61.66
47.40
56.41
61.47
62.60
6.19
TABLE V: Analysis of Reliability-Supervised Routing on the DSEC-Det Test Split, First Seed. All Variants Use Asymmetric-Resolution Fusion.
Model
Setting
s=8
s=16
s=32
FlexEvent Global
20 Hz
0.41
0.45
0.48
180 Hz
0.43
0.48
0.49
Empty
0.40
0.46
0.49
ERA-Det
20 Hz
0.55
0.49
0.46
180 Hz
0.62
0.55
0.48
Empty
0.58
0.54
0.48
TABLE VI: Mean RGB Weight gRs per Stride s under the Interframe Protocol on the DSEC-Det Test Split, First Seed of ERA-Det. FlexEvent Global Uses One Weight per Sample.
Method
AP 50
mAP
Day
Night
SENet † [ 45 ]
81.6
42.4
43.7
37.0
ECA-Net † [ 46 ]
82.2
40.8
42.2
36.1
CBAM † [ 47 ]
81.9
42.8
44.2
38.0
SA-Gate † [ 48 ]
82.0
43.4
44.9
38.0
DCF † [ 49 ]
83.4
42.5
43.4
39.0
SPNet † [ 50 ]
84.7
43.3
44.9
37.1
TABLE VII: Comparison on the PKU-DDD17-Car Test Split, Car Class. Day and Night: mAP on the 704 Day and 209 Night Test Samples. † Results Reported in [ 16 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. The Last Three Rows Are Fine-Tuned from Their DSEC-Det Models; Mean and Standard Deviation over Three Seeds.
Input
Method
mAP
AP 50
AP 75
AP S
AP M
Normal
Low light
Motion blur
E
VIT-S5 † [ 13 ]
23.2
46.6
–
–
–
–
–
–
E
RVT † [ 11 ]
25.6
50.3
–
–
–
–
–
–
F
YOLOX † [ 38 ]
27.4
50.9
–
–
–
–
–
–
F
VIT-S5 † [ 13 ]
28.2
52.2
–
–
–
–
–
–
F
RVT † [ 11 ]
27.9
53.0
–
–
–
–
–
–
E+F
JDF † [ 7 ]
–
44.2
–
–
–
–
–
–
TABLE VIII: Comparison on the PKU-DAVIS-SOD Test Split with Paired Frames and Events. E: Events. F: Frames. Normal, Low Light and Motion Blur: mAP per Test Condition. † Results Reported in [ 18 ] . FlexEvent + ARF Is ERA-Det without Reliability-Supervised Routing. The Last Three Rows Are Fine-Tuned from Their DSEC-Det Eight-Class Models; Mean and Standard Deviation over Three Seeds.
Event window
Frame rate 25/N Hz
Method
50 ms
20 ms
10 ms
5 ms
0 ms
Drop
N=1
N=2
N=4
N=6
N=8
N=10
MDrop
SODFormer † [ 8 ]
–
–
–
–
–
–
20.7
18.6
15.4
14.2
12.6
11.7
9.0
EFNet † [ 55 ]
–
–
–
–
–
–
26.4
24.8
21.6
19.8
16.7
15.9
10.5
AFNet † [ 56 ]
–
–
–
–
–
–
27.5
25.3
21.2
20.0
16.5
15.1
12.4
FF-KDT † [ 57 ]
–
–
–
–
–
–
28.5
26.7
22.6
21.1
17.9
16.5
12.0
ReNet † [ 15 ]
–
–
–
–
–
–
28.8
27.0
22.8
21.4
18.3
17.0
11.8
TABLE IX: Event Window and Frame Rate on the PKU-DAVIS-SOD Test Split, mAP. Event Window: the Frame of the Decision with the Last 50 to 0 ms of Events. Frame Rate: the Mismatch Protocol of FAOD, Frames at 25/N Hz and Events at 25 Hz; MDrop Is the Largest Loss Relative to N=1 . † Results Reported in [ 18 ] . First Seed for the Last Three Rows.
Model
Setting
s=8
s=16
s=32
FlexEvent
Normal
0.43
0.52
0.52
Low light
0.42
0.47
0.49
Motion blur
0.42
0.50
0.51
All
0.43
0.51
0.51
All, no events
0.49
0.57
0.55
FlexEvent + ARF
Normal
0.46
0.51
0.51
TABLE X: Mean RGB Weight gRs at Object Locations per Stride s on the PKU-DAVIS-SOD Test Split, Mean over Three Seeds. All: All Test Conditions. FlexEvent and FlexEvent + ARF Use One Weight per Sample.
Fig. 4: Detections on three scenes of the PKU-DAVIS-SOD test split, one per test condition: normal, 005_test_normal, top; low light, 002_test_low_light, middle; and motion blur, 009_test_motion_blur, bottom. Each row pairs the same frame with the 50 ms of events that precede it, left, and with no events, right. Columns, from left: events of the window, positive in red and negative in blue; detections of FlexEvent + ARF and of ERA-Det with a score of at least 0.3, cars in green, pedestrians in cyan and two-wheelers in magenta, with missed ground-truth objects as red dashed boxes and the number of detected ground-truth objects at the upper left; the RGB weight gR8(x) of ERA-Det at stride 8 over the frame, orange where the frame dominates and blue where the events dominate. The value at the lower right of the FlexEvent + ARF panels is its RGB weight at stride 8, shared by all locations.
Model
Params/M
GFLOPs
Latency/ms
mAP
FlexEvent Global
45.59
36
19.1
59.53
+ ARF
45.59
95
19.2
63.41
ERA-Det
45.64
95
22.0
63.18
TABLE XI: Cost per Time Step at Batch Size 1 and Half Precision on an NVIDIA RTX 6000D GPU, with Global mAP.
Detecting objects reliably under extreme low-light conditions is an open problem in computer vision, with practical urgency in applications ranging from nighttime surveillance to search-and-rescue robotics. Conventional RGB cameras degrade sharply at low photon flux, while event cameras which record asynchronous per-pixel brightness changes at microsecond resolution and high dynamic range provide complementary structural cues that are largely illumination-invariant. We present AdaFuse-Det, a dual-stream framework that fuses CLAHE-enhanced RGB frames with voxelized event tensors through an Adaptive Cross-Modal Fusion (ACMF) module grounded in minimum-variance linear estimation theory. We formally show that the learned attention map asymptotically recovers the Gauss-Markov optimal fusion weights, and establish event conservation and temporal resolution bounds for the voxelization stage. On the LLE-VOS benchmark, AdaFuse-Det achieves a Recall of 65.54%, Precision of 53.85%, and F1-Score of 59.12% under severe illumination degradation, outperforming single-modality detectors in recall by a margin that reflects the theoretically predicted illumination-adaptation behavior.
State-of-the-art RGB-Event detectors improve the detection of small, fast-moving objects by combining complementary features from RGB and Event data, yet they typically fuse the two modalities into a unified representation for both localization and classification. Such a task-symmetric design is inconsistent with the intuition that the two modalities should play different roles according to their task-specific strengths. To examine this issue, we conduct a modality-specific evaluation and find that the relative advantage of the two modalities reverses across tasks: Event data are substantially more effective for class-agnostic localization, whereas RGB data provide stronger category evidence within localized target regions. Motivated by this task-dependent asymmetry, we propose an Asymmetric Event-RGB Object Detection Transformer (AERODet). During class-agnostic localization, Scale-wise Uncertainty-aware Reliability Estimation (SURE) calculates the relative reliability of the two modalities from their objectness response heatmaps and accordingly calibrates their contributions when the decoder aggregates multimodal features. Once the candidate boxes are obtained, Task-Decoupled Semantic Refinement (TDSR) decouples classification from localization and uses RGB RoI features for fine-grained classification. Extensive experiments on FRED and NeRDD demonstrate that AERODet achieves state-of-the-art performance. In particular, it surpasses the strongest RGB-Event baseline by 10.7 mAP points on the FRED challenging split.
RGB-infrared detectors typically discard the statistics generated during cross-modal fusion, leaving downstream modules unaware of whether the current interaction is reliable. We propose to extract a parameter-free, 7-dimensional spectral reliability descriptor -- summarizing band energy, amplitude ratio, phase consistency, and cross-modal correlation -- and to reuse it beyond the fusion stage. The descriptor drives both Spectral Reliability Fusion (SRF), which gates a spectral residual against a conservative spatial base, and Reliability-Conditioned Expert Routing (RCER), which combines the descriptor with pooled content to steer sparse post-fusion experts. Under matched ablations, descriptor-aware gating improves mAP50 over content-only adaptive gating; a 2×2 factorial analysis further shows that descriptor-conditioned routing provides the larger marginal gain over expert architecture alone at near-equal parameter count. Under six synthetic degradations on DroneVehicle, average retention rises to 95.0%, versus 92.0% for content-only MoE and 87.9% for concatenation, with the largest gain under modality drop; the same model also improves mAP50 by +5.2/+5.3 on the natural day/night split. These results suggest that preserving fusion-time reliability as an explicit signal benefits both adaptive fusion and post-fusion conditional computation.