Organizations: Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China.
Event cameras are neuromorphic vision sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, supporting state estimation under rapid motion, motion blur, and high dynamic range (HDR) illumination. This also enables capture of high-frequency blinking patterns from light-emitting diodes (LEDs). Based on this, we propose a real-time event-based framework for LED tracking and six-degree-of-freedom (6-DoF) relative pose estimation using pulse-width modulation (PWM)-encoded LED markers. Multiple LEDs share a common blinking frequency and encode identities through distinct PWM duty cycles, reducing frequency-resource requirements and computational cost. Our dual-rate asynchronous tracking framework combines low-rate LED detection with high-rate raw-event updates, maintaining robust LED identity association and accurate pixel-level tracking under rapid motion. LED identities and positions establish reliable 2D-3D correspondences for real-time metric 6-DoF relative pose estimation. Extensive evaluations on a self-collected dataset demonstrate robustness under large motion flow and transient image-plane LED overlap. Furthermore, we develop a coordinated robotic platform comprising a quadrotor and a Mecanum-wheeled mobile robot, demonstrating the proposed framework's potential for closed-loop drone guidance through hovering and target-following experiments under time-varying HDR illumination, with onboard deployment on an NVIDIA Jetson Orin NX. The dataset and implementation will be released to facilitate further research.
Figures & tables
Method
LED
Asynchronous
High-Speed
Real-Time
Identification
Tracking
Validation
( ≥0.7 M events/s)
ACE [ 25 ]
✗
✓
✗
✗
HASTE [ 26 ]
✗
✓
✗
✗
EKLT [ 27 ]
✗
✗
✗
✗
DeepEvT [ 28 ]
✗
✗
-
✓
jAER [ 29 ]
✗
✓
✓
✓
TABLE I: Comparison of Event-Based Tracking Methods
Fig. 0: Overview of the proposed event-based relative pose estimation framework. The system follows a detect-track-estimate architecture, consisting of PWM-encoded LED detection and identification, asynchronous Kalman filter tracking, and 6-DoF pose estimation.
Sequence
Motion Type
LEDs
Main Characteristics
single_led
General motion
1
Large optical flow
rotation_slow
Rotation
4
Slow rotational motion
rotation_normal
Rotation
4
Moderate rotational motion
rotation_fast
Rotation
4
Large optical flow
translation_z
Translation
4
Translation along the optical axis
6dof_normal
6-DoF
4
Moderate 6-DoF motion
TABLE II: Summary of the Self-Collected Indoor Dataset
Fig. 1: LED tracking results for the evaluated methods on part sequences. Each subplot shows the x -coordinate tracking results of the 60% duty-cycle LED.
Sequence
HASTE [ 26 ]
jAER [ 29 ]
AEB-Tracker [ 13 ]
Ours †
Ours
single_led
failed
14.39
15.59
148.81
13.37
rotation_slow
155.15
134.73
86.78
6.58
8.28
rotation_normal
156.69
8.64
59.08
32.72
7.93
rotation_fast
110.75
13.99
13.21
149.59
13.06
translation_z
failed
65.17
32.54
10.30
7.12
6dof_normal
120.01
17.16
6.93
16.29
6.66
TABLE III: Comparison of LED Tracking Results on the Self-Collected Dataset. [ RMSE : px] Best and second-best results are highlighted and underlined , respectively
Fig. 2: Experimental platforms for relative-pose-based drone guidance. (a) The mecanum-wheeled car carrying the PWM-encoded LED marker. (b) The quadrotor equipped with a Prophesee EVK4 event camera and an NVIDIA Jetson Orin NX.
Fig. 3: Drone relative-position control using the estimated relative pose under three marker motion conditions: stationary, linear, and circular motion.
Method
Asynchronous update
Time / event [ns]
HASTE [ 26 ]
Patch update
217.0
jAER [ 29 ]
Rectangular Cluster Tracker update
32.1
AEB-tracker [ 13 ]
8-state EKF update
2139.0
Ours
4-state KF update
15.3
TABLE V: Computational Cost of the Asynchronous Tracking Update for Different Event-Based Trackers
Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.
Zhichao Li, Chiara Bartolozzi, Lorenzo Natale +1
Event-driven Perception for Robotics, Istituto Italiano di Tecnologia, Italy · Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Italy · University of Genoa, Genoa, Italy
In many autonomous applications requiring real-time localization, active marker-based systems are preferred due to their low latency and ease of deployment compared to computationally demanding feature-based methods. Event~\mbox{cameras} offer high temporal resolution and minimal delay and are commonly used with active LED markers for robust real-time localization. Existing methods typically rely on Perspective-n-Point (PnP) solvers for pose estimation. However, structured marker layouts can be challenging to deploy in space-constrained scenarios, while partial self-motion information (e.g., gravity direction and altitude) is readily available from onboard sensors. We derive a robust and accurate minimal solver that estimates camera pose from only two LED markers by incorporating known tilt angle and camera height measured by an onboard sensor, such as an IMU or an altimeter. The proposed formulation uniquely determines the camera pose through both a closed-form and a linear least-squares solution. We further analyze degenerate configurations and characterize the conditions under which height information does not contribute to rotation estimation. For evaluation, we developed an event-based active marker system to collect real-world data with ground truth from a motion capture system. Experiments on both synthetic and real data demonstrate improved accuracy over the state-of-the-art P2P solver and competitive performance relative to P3P.
Runze Yuan, Alexander Kappler, Jun Zhang +5
Institute of Visual Computing, Graz University of Technology · MIS laboratory, University of Picardie Jules Verne · ICB laboratory, University of Burgundy Europe
Event cameras provide microsecond latency, making them suitable for 6D object pose tracking in fast, dynamic scenes where conventional RGB and depth pipelines suffer from motion blur and large pixel displacements. We introduce EventTrack6D, an event-depth tracking framework that generalizes to novel objects without object-specific training by reconstructing both intensity and depth at arbitrary timestamps between depth frames. Conditioned on the most recent depth measurement, our dual reconstruction recovers dense photometric and geometric cues from sparse event streams. Our EventTrack6D operates at over 120 FPS and maintains temporal consistency under rapid motion. To support training and evaluation, we introduce a comprehensive benchmark suite: a large-scale synthetic dataset for training and two complementary evaluation sets, including real and simulated event datasets. Trained exclusively on synthetic data, EventTrack6D generalizes effectively to real-world scenarios without fine-tuning, maintaining accurate tracking across diverse objects and motion patterns. Our method and datasets validate the effectiveness of event cameras for event-based 6D pose tracking of novel objects. Code and datasets are publicly available at https://chohoonhee.github.io/Event6D.