Rate as a Feature, Not a Nuisance: Curriculum FiLM Training for Event-Based Graph Neural Networks

Kavyansh Tyagi; Andreas Ziegler

cs.CVOct 17, 2026

Rate as a Feature, Not a Nuisance: Curriculum FiLM Training for Event-Based Graph Neural Networks

Authors: Kavyansh Tyagi, Andreas Ziegler

Organizations: National Institute of Technology Kurukshetra, Kurukshetra, India · University of Toronto, Toronto, Canada

Abstract

An event camera emits events at a rate given by the scene motion, object texture, illumination, its own contrast thresholds, and other settings. Across the N-Caltech101 dataset, recorded by a single event camera, the event rate varies by roughly 26x. Standard event-based Graph Neural Network GNN pipelines do not make use of the variable event rate. They either normalize each event's timestamp into a fixed window or train the network to be invariant to the event rate. We argue that both choices lose information that could instead be used to improve the performance of a pipeline. Per-recording normalization is non-invertible, incurring about 6pp (percentage points) of natural-rate accuracy loss in our setting. And invariance training trades accuracy for robustness to shifts in the event rate. We instead take a different path and condition on the event rate directly. We add a per-recording log-rate scalar that modulates every stage of a PointNet++ backbone through Feature-wise Linear Modulation (FiLM), initialized to identity, so that training begins at the unmodified baseline. A two-stage curriculum (natural event rates first, then rate-augmented fine-tuning) reaches 67.2% on N-Caltech101 (+6.3pp) and 82.5% on DVS128 Gesture (+14.0pp) with an additional ~0.2M parameters. A matched-capacity control experiment shows that the gain is not from the increase in model capacity but from the rate conditioning. We will make the implementation publicly available upon acceptance of this work.

Figures & tables

Figure 1: Overview of the proposed FiLM-Rate framework. An event stream is converted into a spatio-temporal radius graph and processed by a PointNet++ [ 15 ] backbone. FiLM layers [ 14 ] condition the intermediate representations using a centered log-rate signal after each set-abstraction stage (SA) and after global pooling. The resulting pooled graph descriptor,

f_{E}

, is used for classification and can optionally be augmented at test time using RA-TTA (rate-adapter test-time adaptation) to improve robustness to extreme event-rate shifts while keeping the backbone frozen.

Method	Representation	Params $\downarrow$	Top-1 (%) $\uparrow$
EventNet [ 18 ]	point	–	42.5
HATS [ 19 ]	time surfaces	–	64.2
EST [ 6 ]	learned grid + CNN	${\sim}$ 21M	75.3 †
RG-CNN [ 3 ]	graph	19.46M	65.7
AEGNN [ 17 ]	graph	20.40M	66.8
EV-VGCNN [ 4 ]	voxel graph	0.84M	74.8

Table 1: Top-1 accuracy on N-Caltech101. This is not a controlled comparison: protocols differ in splits, pre-processing, input representation, and pre-training. Non-graph methods are shown in gray. Numbers from the original papers are taken from [ 21 ] ; Parameter counts for RG-CNN, AEGNN, EV-VGCNN, and VMV-GCN are as measured on N-Caltech101 in [ 23 ] . Our models use no pre-training and a deliberately small backbone.

	Natural-rate $\uparrow$		Synthetic-rate $\uparrow$
Method	Avg.	Worst	Avg.	Worst	1.0 $\times\uparrow$	Params $\downarrow$
Baseline	60.9	50.0	16.0	3.4	60.6	163K
Baseline widehead	62.0	50.8	16.0	2.4	61.9	208K
RateConditioned (head)	60.0	52.0	17.5	5.3	59.9	163K
ra-aug-only	60.3	49.2	47.0	8.0	59.8	163K
R-Canon on baseline	44.7	34.3	25.2	8.6	44.8	163K+264K

Table 2: Main results on N-Caltech101 (identical backbone, data, and schedule). We report Top-1 accuracy, aggregated as the average and worst-case across conditions [ 7 , 16 ] . “Natural-rate” aggregates over 7 equi-quantile natural-rate buckets. “Synthetic-rate” aggregates over 7 synthetic factors

\{0.125,\dots,8.0\}

;

1.0\times

= native rate. R-Canon denotes a baseline that learns a per-recording rescaling factor to canonicalize the event rate before a frozen classifier. Bold best, underline second best.

Figure 2: Left: Distribution of recording-level event rates on N-Caltech101. Red vertical markers indicate the median event rate of each of the 101 classes (marker height has no meaning and is shown uniformly for clarity). The figure illustrates that the event rate varies substantially across object categories. The natural spread is

{\sim}26\times

and depends strongly on the class, yet standard normalization collapses it to a constant. Right: natural-rate vs. synthetic-rate mean accuracy of every method in Tab. 2 . Rate-invariance training moves right but down; our curriculum and its RA-TTA extension sit on the upper frontier.

Capacity / placement		$\Delta$ params $\downarrow$	Nat. mean $\uparrow$	$\Delta\uparrow$
Baseline		–	60.9	–
Baseline widehead		+44K	62.0	+1.1
FiLM-Global-Only		+34K	60.3	$-$ 0.6
FiLM-Rate (per-layer)		+49K	64.4	+3.5
Curriculum source	Init	Aug range	Nat. mean $\uparrow$	1.0 $\times\uparrow$
FiLM (no aug)	scratch	$\{1.0\}$	64.4	63.4

Table 3: Consolidated ablations. Top: capacity vs. mechanism vs. placement: matched extra parameters achieve +1.1 pp, per-layer rate conditioning additional +2.4 pp, while conditioning only the pooled feature representation

f_{E}

( i.e . , after global pooling) provides no benefit. Bottom: the curriculum’s gain comes from the warm-start initialization, not the augmentation range. Bold marks the best value in each block.

∗ Robustness improves instead: 73.5% vs. 61.7% at $0.125\times$ ; synth. mean 87.9% vs. 84.9%.
Method	DVS128 Gesture	N-Caltech101	CIFAR10-DVS	N-Cars
	(11-cls)	(101-cls)	(10-cls)	(2-cls, sat.)
Baseline	68.6	60.9	51.5	93.9
FiLM-Rate (no aug)	71.2	64.4	53.2	93.6
FiLM-Rate (curr.)	82.5	67.2	56.7	93.5
$\Delta$ vs. baseline	+14.0 pp	+6.3 pp	+5.2 pp	$-$ 0.4 pp ∗

Table 4: Natural-rate mean accuracy across four benchmarks with identical architecture, schedule, and pre-processing schema; only per-sensor set-abstraction (SA) radii differ. Bold marks the best value per column.

Backbone	Baseline	FiLM-Rate (curr.)	$\Delta$
PointNet++ ( Sec. 4.3 )	60.9	67.2	$+6.3$ pp
EventMamba (state-space)	67.4	74.1	$+6.7$ pp

Table 5: Cross-architecture summary: does the FiLM-Rate curriculum improve natural-rate Top-1 accuracy over an unconditioned baseline of the same architecture? PointNet++ is evaluated on N-Caltech101; EventMamba on DVS128 Gesture. Bold marks the better (FiLM-Rate curriculum) value per row.

	N-Caltech101	DVS128 Gesture
$I(\log\hat{\lambda}_{E};y)/H(y)$	6.65%	1.75%
Per-class mean log-rate spread	1.43 nats	0.11 nats
Chance (most-frequent class)	9.18	21.79
Rate-only classifier (best)	16.48	21.79
gain over chance	$+7.30$	$\mathbf{+0.00}$
FiLM-Rate curriculum gain over baseline	$+6.3$ pp	$\mathbf{+14.0}$ pp

Table S6 : Rate/class leakage versus accuracy gain, on the two datasets where the recipe helps. If the gain were a rate-label shortcut, it would grow with leakage. It does the opposite: on DVS128 Gesture a rate-only classifier is exactly at chance, and that is where the recipe gains most.

Lighting condition	Baseline	FiLM-Rate (curr.)	$\Delta$	$n$
lab	55.87	65.47	+9.60	12,905
LED	88.71	91.38	+2.67	3,375
fluorescent	80.67	83.15	+2.48	5,809
natural ∗	76.92	87.18	+10.26	39
∗ Sample count too small for a reliable estimate; reported for completeness only.

Table S7 : Per-lighting-condition Top-1 accuracy on DVS128 Gesture (EventMamba backbone), stratified by the recording’s native illumination tag. The curriculum improves over the baseline under every condition, with the largest gain in the hardest one. Conditions are strongly imbalanced and we report

n

accordingly; the natural row is shown for completeness but is too small (

n=39

) to support a conclusion, and no test recordings carry the fluorescent+LED tag Bold marks the better (FiLM-Rate curriculum) value per row.

Method	b0	b1	b2	b3	b4	b5	b6	Mean	Worst
Baseline	65.1	65.9	68.3	59.8	60.6	56.6	50.0	60.9	50.0
FiLM-Rate (no aug)	65.9	72.3	69.1	67.9	62.3	57.8	55.7	64.4	55.7
FiLM-Rate (curr.)	67.5	76.3	72.7	70.3	67.1	63.1	53.6	67.2	53.6
$\Delta$ vs. baseline	+2.4	+10.4	+4.4	+10.5	+6.5	+6.5	+3.6	+6.3	+3.6

Table S8 : Per-bucket natural-rate accuracy (%) of the top three methods (b0 = lowest event rate, b6 = highest). The curriculum improves over FiLM (no aug) in 6 of 7 buckets; the single regression (b6) is discussed in Sec. 8 . Bold marks the best value per column.

Figure S3 : Left: mean learned scale parameter,

|\gamma|

(solid) and learned shift parameter,

|\beta|

(dashed) per FiLM layer across natural-rate buckets. The FiLM modulation is non-negligible at every layer and, for most channels, increases slightly with event rate. The per-node layers (SA1–SA3) carry signal that the variant which only modulates the pooled feature representation

f_{E}

( i.e . , after global pooling) cannot represent. Right: cosine-silhouette of the class structure in the pooled feature representation

f_{E}

, per bucket: the curriculum’s features are more class-separable everywhere, with the largest gaps in the buckets where its accuracy gains are largest ( Tab. S8 ).

Dataset	Between-class variance share	$\Delta$ curr. vs. baseline (pp)
N-Cars	0.0003	$-0.5$
CIFAR10-DVS	0.133	$+5.2$
N-Caltech101	0.374	$+6.3$
DVS128 Gesture	0.438	$+14.0$

Table S9 : Between-class share of the per-recording log-rate variance, and the curriculum’s accuracy gain over the baseline, across four datasets. The gain tracks the fraction of rate variance that separates classes, not the size of the rate spread itself. For N-Cars, of a total log-rate variance of

0.9062

, the within-class component is

0.9060

and the between-class component only

0.0003

Event-Based Vision

Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.

See all 232 papers

Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision
Jun 18, 2026Roberto Pellerito, Daniel Gehrig, Shintaro Shiba +1Event-Based Vision Representation Learning
Robotics and Perception Group, University of Zurich · University of Pennsylvania · The University of Tokyo +1
Match-Any-Events: Zero-Shot Motion-Robust Feature Matching Across Wide Baselines for Event Cameras
Apr 20, 2026Ruijun Zhang, Hang Su, Kostas Daniilidis +1Event-Based Vision Domain Generalization
Johns Hopkins University, Baltimore MD 21218, USA · ShanghaiTech University, Shanghai, China · University of Pennsylvania, Philadelphia PA 19104, USA +1
FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection
Jun 15, 2026Md Tawheedul Islam Bhuian, Kyoung-Don KangEvent-Based Vision Frequency-Domain Feature Learning
School of Computing State University of New York at Binghamton

Jun 18, 2026J/K move · Enter open · S save

Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision

Roberto Pellerito, Daniel Gehrig, Shintaro Shiba, Davide Scaramuzza

Robotics and Perception Group, University of Zurich · University of Pennsylvania · The University of Tokyo+1

Event-Based Vision Representation Learning Autoencoders Event-Based Object Detection

Open paper AskPDF arXiv

Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution events. Each individual event, however, only carries minimal semantic value, merely signaling a localized brightness change. To derive meaningful signals, downstream algorithms need to quickly integrate cues from a potentially massive torrent of low-information events. Current architectures, however, are easily overwhelmed, struggling to balance capturing fine-grained temporal dynamics and maintaining a manageable data throughput. This paper proposes a framework to re-tokenize event streams into a small set of highly informative neural events, each representing a local spatio-temporal context window with a discrete learnable code. Every time this code flips, a neural event is triggered, yielding a highly compressed data stream. We demonstrate that, across object detection and classification, networks trained on neural events are on par or surpass the performance of state-of-the-art approaches while reducing the event rate by a factor of 2.0.

Abstract

Event-Based Vision

Explore similar work

Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision

Match-Any-Events: Zero-Shot Motion-Robust Feature Matching Across Wide Baselines for Event Cameras

FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection