An event camera emits events at a rate given by the scene motion, object texture, illumination, its own contrast thresholds, and other settings. Across the N-Caltech101 dataset, recorded by a single event camera, the event rate varies by roughly 26x. Standard event-based Graph Neural Network GNN pipelines do not make use of the variable event rate. They either normalize each event's timestamp into a fixed window or train the network to be invariant to the event rate. We argue that both choices lose information that could instead be used to improve the performance of a pipeline. Per-recording normalization is non-invertible, incurring about 6pp (percentage points) of natural-rate accuracy loss in our setting. And invariance training trades accuracy for robustness to shifts in the event rate. We instead take a different path and condition on the event rate directly. We add a per-recording log-rate scalar that modulates every stage of a PointNet++ backbone through Feature-wise Linear Modulation (FiLM), initialized to identity, so that training begins at the unmodified baseline. A two-stage curriculum (natural event rates first, then rate-augmented fine-tuning) reaches 67.2% on N-Caltech101 (+6.3pp) and 82.5% on DVS128 Gesture (+14.0pp) with an additional ~0.2M parameters. A matched-capacity control experiment shows that the gain is not from the increase in model capacity but from the rate conditioning. We will make the implementation publicly available upon acceptance of this work.
Figures & tables
Figure 1: Overview of the proposed FiLM-Rate framework. An event stream is converted into a spatio-temporal radius graph and processed by a PointNet++ [ 15 ] backbone. FiLM layers [ 14 ] condition the intermediate representations using a centered log-rate signal after each set-abstraction stage (SA) and after global pooling. The resulting pooled graph descriptor, fE , is used for classification and can optionally be augmented at test time using RA-TTA (rate-adapter test-time adaptation) to improve robustness to extreme event-rate shifts while keeping the backbone frozen.
Method
Representation
Params ↓
Top-1 (%) ↑
EventNet [ 18 ]
point
–
42.5
HATS [ 19 ]
time surfaces
–
64.2
EST [ 6 ]
learned grid + CNN
∼ 21M
75.3 †
RG-CNN [ 3 ]
graph
19.46M
65.7
AEGNN [ 17 ]
graph
20.40M
66.8
EV-VGCNN [ 4 ]
voxel graph
0.84M
74.8
Table 1: Top-1 accuracy on N-Caltech101. This is not a controlled comparison: protocols differ in splits, pre-processing, input representation, and pre-training. Non-graph methods are shown in gray. Numbers from the original papers are taken from [ 21 ] ; Parameter counts for RG-CNN, AEGNN, EV-VGCNN, and VMV-GCN are as measured on N-Caltech101 in [ 23 ] . Our models use no pre-training and a deliberately small backbone.
Natural-rate ↑
Synthetic-rate ↑
Method
Avg.
Worst
Avg.
Worst
1.0 ×↑
Params ↓
Baseline
60.9
50.0
16.0
3.4
60.6
163K
Baseline widehead
62.0
50.8
16.0
2.4
61.9
208K
RateConditioned (head)
60.0
52.0
17.5
5.3
59.9
163K
ra-aug-only
60.3
49.2
47.0
8.0
59.8
163K
R-Canon on baseline
44.7
34.3
25.2
8.6
44.8
163K+264K
Table 2: Main results on N-Caltech101 (identical backbone, data, and schedule). We report Top-1 accuracy, aggregated as the average and worst-case across conditions [ 7 , 16 ] . “Natural-rate” aggregates over 7 equi-quantile natural-rate buckets. “Synthetic-rate” aggregates over 7 synthetic factors {0.125,…,8.0} ; 1.0× = native rate. R-Canon denotes a baseline that learns a per-recording rescaling factor to canonicalize the event rate before a frozen classifier. Bold best, underline second best.
Figure 2: Left: Distribution of recording-level event rates on N-Caltech101. Red vertical markers indicate the median event rate of each of the 101 classes (marker height has no meaning and is shown uniformly for clarity). The figure illustrates that the event rate varies substantially across object categories. The natural spread is ∼26× and depends strongly on the class, yet standard normalization collapses it to a constant. Right: natural-rate vs. synthetic-rate mean accuracy of every method in Tab. 2 . Rate-invariance training moves right but down; our curriculum and its RA-TTA extension sit on the upper frontier.
Capacity / placement
Δ params ↓
Nat. mean ↑
Δ↑
Baseline
–
60.9
–
Baseline widehead
+44K
62.0
+1.1
FiLM-Global-Only
+34K
60.3
− 0.6
FiLM-Rate (per-layer)
+49K
64.4
+3.5
Curriculum source
Init
Aug range
Nat. mean ↑
1.0 ×↑
FiLM (no aug)
scratch
{1.0}
64.4
63.4
Table 3: Consolidated ablations. Top: capacity vs. mechanism vs. placement: matched extra parameters achieve +1.1 pp, per-layer rate conditioning additional +2.4 pp, while conditioning only the pooled feature representation fE ( i.e . , after global pooling) provides no benefit. Bottom: the curriculum’s gain comes from the warm-start initialization, not the augmentation range. Bold marks the best value in each block.
Method
DVS128 Gesture
N-Caltech101
CIFAR10-DVS
N-Cars
(11-cls)
(101-cls)
(10-cls)
(2-cls, sat.)
Baseline
68.6
60.9
51.5
93.9
FiLM-Rate (no aug)
71.2
64.4
53.2
93.6
FiLM-Rate (curr.)
82.5
67.2
56.7
93.5
Δ vs. baseline
+14.0 pp
+6.3 pp
+5.2 pp
− 0.4 pp ∗
∗ Robustness improves instead: 73.5% vs. 61.7% at 0.125× ; synth. mean 87.9% vs. 84.9%.
Table 4: Natural-rate mean accuracy across four benchmarks with identical architecture, schedule, and pre-processing schema; only per-sensor set-abstraction (SA) radii differ. Bold marks the best value per column.
Backbone
Baseline
FiLM-Rate (curr.)
Δ
PointNet++ ( Sec. 4.3 )
60.9
67.2
+6.3 pp
EventMamba (state-space)
67.4
74.1
+6.7 pp
Table 5: Cross-architecture summary: does the FiLM-Rate curriculum improve natural-rate Top-1 accuracy over an unconditioned baseline of the same architecture? PointNet++ is evaluated on N-Caltech101; EventMamba on DVS128 Gesture. Bold marks the better (FiLM-Rate curriculum) value per row.
N-Caltech101
DVS128 Gesture
I(logλ^E;y)/H(y)
6.65%
1.75%
Per-class mean log-rate spread
1.43 nats
0.11 nats
Chance (most-frequent class)
9.18
21.79
Rate-only classifier (best)
16.48
21.79
gain over chance
+7.30
+0.00
FiLM-Rate curriculum gain over baseline
+6.3 pp
+14.0 pp
Table S6 : Rate/class leakage versus accuracy gain, on the two datasets where the recipe helps. If the gain were a rate-label shortcut, it would grow with leakage. It does the opposite: on DVS128 Gesture a rate-only classifier is exactly at chance, and that is where the recipe gains most.
Lighting condition
Baseline
FiLM-Rate (curr.)
Δ
n
lab
55.87
65.47
+9.60
12,905
LED
88.71
91.38
+2.67
3,375
fluorescent
80.67
83.15
+2.48
5,809
natural ∗
76.92
87.18
+10.26
39
∗ Sample count too small for a reliable estimate; reported for completeness only.
Table S7 : Per-lighting-condition Top-1 accuracy on DVS128 Gesture (EventMamba backbone), stratified by the recording’s native illumination tag. The curriculum improves over the baseline under every condition, with the largest gain in the hardest one. Conditions are strongly imbalanced and we report n accordingly; the natural row is shown for completeness but is too small ( n=39 ) to support a conclusion, and no test recordings carry the fluorescent+LED tag Bold marks the better (FiLM-Rate curriculum) value per row.
Method
b0
b1
b2
b3
b4
b5
b6
Mean
Worst
Baseline
65.1
65.9
68.3
59.8
60.6
56.6
50.0
60.9
50.0
FiLM-Rate (no aug)
65.9
72.3
69.1
67.9
62.3
57.8
55.7
64.4
55.7
FiLM-Rate (curr.)
67.5
76.3
72.7
70.3
67.1
63.1
53.6
67.2
53.6
Δ vs. baseline
+2.4
+10.4
+4.4
+10.5
+6.5
+6.5
+3.6
+6.3
+3.6
Table S8 : Per-bucket natural-rate accuracy (%) of the top three methods (b0 = lowest event rate, b6 = highest). The curriculum improves over FiLM (no aug) in 6 of 7 buckets; the single regression (b6) is discussed in Sec. 8 . Bold marks the best value per column.
Figure S3 : Left: mean learned scale parameter, ∣γ∣ (solid) and learned shift parameter, ∣β∣ (dashed) per FiLM layer across natural-rate buckets. The FiLM modulation is non-negligible at every layer and, for most channels, increases slightly with event rate. The per-node layers (SA1–SA3) carry signal that the variant which only modulates the pooled feature representation fE ( i.e . , after global pooling) cannot represent. Right: cosine-silhouette of the class structure in the pooled feature representation fE , per bucket: the curriculum’s features are more class-separable everywhere, with the largest gaps in the buckets where its accuracy gains are largest ( Tab. S8 ).
Dataset
Between-class variance share
Δ curr. vs. baseline (pp)
N-Cars
0.0003
−0.5
CIFAR10-DVS
0.133
+5.2
N-Caltech101
0.374
+6.3
DVS128 Gesture
0.438
+14.0
Table S9 : Between-class share of the per-recording log-rate variance, and the curriculum’s accuracy gain over the baseline, across four datasets. The gain tracks the fraction of rate variance that separates classes, not the size of the rate spread itself. For N-Cars, of a total log-rate variance of 0.9062 , the within-class component is 0.9060 and the between-class component only 0.0003 .
Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution events. Each individual event, however, only carries minimal semantic value, merely signaling a localized brightness change. To derive meaningful signals, downstream algorithms need to quickly integrate cues from a potentially massive torrent of low-information events. Current architectures, however, are easily overwhelmed, struggling to balance capturing fine-grained temporal dynamics and maintaining a manageable data throughput. This paper proposes a framework to re-tokenize event streams into a small set of highly informative neural events, each representing a local spatio-temporal context window with a discrete learnable code. Every time this code flips, a neural event is triggered, yielding a highly compressed data stream. We demonstrate that, across object detection and classification, networks trained on neural events are on par or surpass the performance of state-of-the-art approaches while reducing the event rate by a factor of 2.0.
Roberto Pellerito, Daniel Gehrig, Shintaro Shiba +1
Robotics and Perception Group, University of Zurich · University of Pennsylvania · The University of Tokyo +1
Event cameras have recently shown promising capabilities in instantaneous motion estimation due to their robustness to low light and fast motions. However, computing wide-baseline correspondence between two arbitrary views remains a significant challenge, since event appearance changes substantially with motion, and learning-based approaches are constrained by both scalability and limited wide-baseline supervision. We therefore introduce the first event matching model that achieves cross-dataset wide-baseline correspondence in a zero-shot manner: a single model trained once is deployed on unseen datasets without any target-domain fine-tuning or adaptation. To enable this capability, we introduce a motion-robust and computationally efficient attention backbone that learns multi-timescale features from event streams, augmented with sparsity-aware event token selection, making large-scale training on diverse wide-baseline supervision computationally feasible. To provide the supervision needed for wide-baseline generalization, we develop a robust event motion synthesis framework to generate large-scale event-matching datasets with augmented viewpoints, modalities, and motions. Extensive experiments across multiple benchmarks show that our framework achieves a 37.7% improvement over the previous best event feature matching methods. Code and data are available at: https://github.com/spikelab-jhu/Match-Any-Events.
Ruijun Zhang, Hang Su, Kostas Daniilidis +1
Johns Hopkins University, Baltimore MD 21218, USA · ShanghaiTech University, Shanghai, China · University of Pennsylvania, Philadelphia PA 19104, USA +1
Event cameras are bio-inspired sensors that asynchronously capture logarithmic intensity changes, offering inherent advantages in high-speed and high-dynamic-range scenarios. However, the sparse and asynchronous nature of event streams poses a fundamental challenge for modern deep learning architectures. To enable compatibility with standard models, most existing approaches partition the accumulation window into fixed temporal sub-bins. While effective for spatial processing, this internal discretization discards fine-grained temporal structure and constrains inference to the low temporal frequencies imposed by training supervision. To address this limitation, we propose FATE, a unified framework built upon a novel Pillar Encoding (PE). While operating over discrete macro-accumulation windows dictated by the target frequency, PE avoids internal temporal sub-binning. It organizes events into spatial pillars and approximates their intra-window evolution via projection onto a continuous-time orthogonal polynomial basis. This formulation yields an L2-optimal representation that retains rich temporal dynamics in a dense pseudo-image, mitigating information loss under sparse event conditions. To fully leverage this representation, we introduce Frequency-Aware Training (FAT), a soft mean-teacher curriculum that generates temporally dense pseudo-labels, effectively bridging the mismatch between low-frequency supervision and high-frequency inference. Extensive experiments demonstrate that FATE generalizes across architectural paradigms and consistently outperforms strong baselines. It enables robust object detection at high temporal resolutions up to 200 Hz, while incurring minimal overhead in parameter count and inference latency
Md Tawheedul Islam Bhuian, Kyoung-Don Kang
School of Computing State University of New York at Binghamton