Sensor Generalization for Adaptive Sensing in Event-based Object Detection via Joint Distribution Training

Aheli Saha; René Schuster; Didier Stricker

cs.CVFeb 26, 2026

Sensor Generalization for Adaptive Sensing in Event-based Object Detection via Joint Distribution Training

Authors: Aheli Saha, René Schuster, Didier Stricker

Organizations: Deutsches Forschungszentrum für Künstliche Intelligenz, Kaiserslautern, Germany

Abstract

Bio-inspired event cameras have recently attracted significant research due to their asynchronous and low-latency capabilities. These features provide a high dynamic range and significantly reduce motion blur. However, because of the novelty in the nature of their output signals, there is a gap in the variability of available data and a lack of extensive analysis of the parameters characterizing their signals. This paper addresses these issues by providing readers with an in-depth understanding of how intrinsic parameters affect the performance of a model trained on event data, specifically for object detection. We also use our findings to expand the capabilities of the downstream model towards sensor-agnostic robustness.

Figures & tables

Figure 1: Overview of a sensor control loop. We aim to dynamically adapt the sensor characteristics based on energy utilization and task performance feedback. In this work, we investigate and mitigate the undesired effects of dynamically varying spike transduction on task performance, i.e . , object detection.

Panel of Figure 2 — Figure 2: Examples of two scenes with varying configurations which produce high and low event densities, respectively. The black and white points represent negative and positive events, respectively, accumulated over a 50-ms time period. The gray spaces indicate the occurrence of no events within this timeframe at the corresponding pixel locations. The corresponding instance segmentation map is provided for reference to the actual vehicular locations.

Config. ( ${\mathbfcal{E}}_{\bm{i}}$ )	Pos. Thr. ( $\bm{th_{p}}$ )	Neg. Thr. ( $\bm{th_{n}}$ )	Ref. Period ( $\bm{T_{r}}$ ) [ms]	Field of View ( $\bm{F_{v}}$ )
$\mathbfcal{E}_{\bm{base}}$	0.5	0.5	10	$90^{\circ}$
$\mathbfcal{E}_{\bm{1}}$	0.25	0.25	0.01	$90^{\circ}$
$\mathbfcal{E}_{\bm{2}}$	0.75	0.75	0.01	$90^{\circ}$
$\mathbfcal{E}_{\bm{3}}$	1.0	1.0	0.01	$90^{\circ}$
$\mathbfcal{E}_{\bm{4}}$	0.5	0.5	10	$90^{\circ}$
$\mathcal{E}_{\bm{5}}$	0.5	0.5	25	$90^{\circ}$

Table 1: Data Configurations.

Figure 3: Simplified overview of the design and positioning of the distinct test sets in the parameter space of two parameters - the field of view and negative threshold. The training set and the first test set consist of distinct configurations varying along each dimension. The second test set consists of configurations obtained by interpolating along each dimension, whereas the third set is obtained by combining seen parameters into unique configurations. The fourth set represents the unseen parameter space for each parameter.

Panel of Figure 4 — Figure 4: Comparison of results between the different models - RVT-B and SSMS-B trained on $\mathcal{E}_{base}$ and $S_{train}$ . The blue boxes indicate the predicted bounding boxes, and the red boxes represent the ground truth. Each of the three samples represents different sensor configurations to illustrate the performance across varying scenarios.

Train/Test		RVT-B		SSMS-B
Config.		$\mathbfcal{E}_{\bm{base}}$	$\bm{S}_{\bm{train}}$	$\mathbfcal{E}_{\bm{base}}$	$\bm{S}_{\bm{train}}$
$\bm{S}_{\bm{test}}^{\bm{1}}$	$\mathbfcal{E}_{\bm{base}}$	45.63	44.12	49.10	50.58
	$\mathbfcal{E}_{\bm{1}}$	44.49	45.94	51.99	53.90
	$\mathbfcal{E}_{\bm{3}}$	23.74	30.23	26.72	33.01
	$\mathbfcal{E}_{\bm{4}}$	45.13	44.72	48.97	50.55
	$\mathbfcal{E}_{\bm{6}}$	44.61	44.13	48.73	50.05

Table 2: Comparison of performance (AP scores in

\%

) for each configuration.

${\text{{Score}}}_{\bm{k}}$ ( $\bm{f}$ )	$\bm{AP(m_{1})}$		$\bm{AP_{\bm{50}}(m_{2})}$		$\bm{AP_{\bm{75}}(m_{3})}$		$\bm{AP_{\bm{L}}(m_{4})}$		$\bm{AP_{\bm{M}}(m_{5})}$
	RVT-B	SSMS-B	RVT-B	SSMS-B	RVT-B	SSMS-B	RVT-B	SSMS-B	RVT-B	SSMS-B
$\text{{Score}}_{\bm{1}}$ $(\mathbfcal{E}_{\bm{base}})$	34.6 $\pm$ 14.7	38.2 $\pm$ 16.6	59.7 $\pm$ 17.9	60.9 $\pm$ 19.8	37.4 $\pm$ 19.7	42.8 $\pm$ 21.6	44.0 $\pm$ 18.0	49.6 $\pm$ 20.4	30.0 $\pm$ 13.4	32.0 $\pm$ 14.9
$\text{{Score}}_{\bm{1}}$ $(\bm{S}_{\bm{train}})$	37.4 $\pm$ 10.7	42.4 $\pm$ 13.1	63.4 $\pm$ 11.3	65.3 $\pm$ 14.2	40.7 $\pm$ 15.5	47.7 $\pm$ 17.5	47.4 $\pm$ 14.1	53.8 $\pm$ 17.4	32.3 $\pm$ 9.3	36.4 $\pm$ 11.4
$\text{{Score}}_{\bm{2}}$ $(\mathbfcal{E}_{\bm{base}})$	37.5 $\pm$ 7.3	39.9 $\pm$ 8.5	64.6 $\pm$ 6.3	65.3 $\pm$ 6.9	39.7 $\pm$ 12.4	44.0 $\pm$ 12.1	46.3 $\pm$ 12.8	46.9 $\pm$ 16.1	33.5 $\pm$ 6.0	35.9 $\pm$ 6.0
$\text{{Score}}_{\bm{2}}$ $(\bm{S}_{\bm{train}})$	39.4 $\pm$ 4.5	44.8 $\pm$ 5.4	66.7 $\pm$ 3.6	68.9 $\pm$ 4.4	43.1 $\pm$ 7.8	51.3 $\pm$ 7.1	47.4 $\pm$ 10.7	54.4 $\pm$ 12.3	35.6 $\pm$ 4.6	39.8 $\pm$ 5.6
$\text{{Score}}_{\bm{3}}$ $(\mathbfcal{E}_{\bm{base}})$	34.1 $\pm$ 0.2	40.6 $\pm$ 0.1	62.3 $\pm$ 4.5	66.7 $\pm$ 2.7	34.4 $\pm$ 3.7	45.2 $\pm$ 0.0	39.3 $\pm$ 3.3	54.9 $\pm$ 4.4	27.6 $\pm$ 1.5	31.9 $\pm$ 0.6

Table 3: Complete Performance Comparison on the RVT-B Model and the SSMS-B Model. We observe that the SSMS-B Model trained on

S_{train}

performs the best uniformly in each score metric, across all test sets.

Adaptive Sensing

See all 19 papers

FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection
Jun 15, 2026Md Tawheedul Islam Bhuian, Kyoung-Don KangEvent-Based Vision Frequency-Domain Feature Learning
School of Computing State University of New York at Binghamton
BiasBench: A reproducible benchmark for tuning the biases of event cameras
Apr 25, 2025Andreas Ziegler, David Joseph, Thomas Gossard +2Event-Based Vision Benchmark Design
Cognitive Systems Group, University of Tübingen, Germany
VibES: Induced Vibration for Persistent Event-Based Sensing
Aug 26, 2025Vincenzo Polizzi, Stephen Yang, Quentin Clark +3Event-Based Vision
University of Toronto, Robotics Institute · University of Toronto, Department of Mechanical and Industrial Engineering · University of Toronto +1

Jun 15, 2026J/K move · Enter open · S save

FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection

Md Tawheedul Islam Bhuian, Kyoung-Don Kang

School of Computing State University of New York at Binghamton

Event-Based Vision Frequency-Domain Feature Learning Event-Based Object Detection

Open paper AskPDF arXiv

Event cameras are bio-inspired sensors that asynchronously capture logarithmic intensity changes, offering inherent advantages in high-speed and high-dynamic-range scenarios. However, the sparse and asynchronous nature of event streams poses a fundamental challenge for modern deep learning architectures. To enable compatibility with standard models, most existing approaches partition the accumulation window into fixed temporal sub-bins. While effective for spatial processing, this internal discretization discards fine-grained temporal structure and constrains inference to the low temporal frequencies imposed by training supervision. To address this limitation, we propose FATE, a unified framework built upon a novel Pillar Encoding (PE). While operating over discrete macro-accumulation windows dictated by the target frequency, PE avoids internal temporal sub-binning. It organizes events into spatial pillars and approximates their intra-window evolution via projection onto a continuous-time orthogonal polynomial basis. This formulation yields an L2-optimal representation that retains rich temporal dynamics in a dense pseudo-image, mitigating information loss under sparse event conditions. To fully leverage this representation, we introduce Frequency-Aware Training (FAT), a soft mean-teacher curriculum that generates temporally dense pseudo-labels, effectively bridging the mismatch between low-frequency supervision and high-frequency inference. Extensive experiments demonstrate that FATE generalizes across architectural paradigms and consistently outperforms strong baselines. It enables robust object detection at high temporal resolutions up to 200 Hz, while incurring minimal overhead in parameter count and inference latency

Abstract

Adaptive Sensing

Explore similar work

FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection

BiasBench: A reproducible benchmark for tuning the biases of event cameras

VibES: Induced Vibration for Persistent Event-Based Sensing