In multi-robot systems, traditional radio frequency (RF) communication struggles with contention and jamming. Optical communication offers a strong alternative. However, conventional frame-based cameras suffer from limited frame rates, motion blur, and reduced robustness under high dynamic range lighting. Event cameras support microsecond temporal resolution and high dynamic range, making them extremely sensitive to scene changes under fast relative motion with an optical transmitter. Leveraging these strengths, we develop a complete optical communication system capable of tracking moving transmitters and decoding messages in real time. Our system achieves over 95% decoding accuracy for text transmission during motion by implementing a Geometry-Aware Unscented Kalman Filter (GA-UKF), achieving 7x faster processing speed compared to the previous state-of-the-art method, while maintaining equivalent tracking accuracy at transmitting frequencies ≥ 1 kHz.
Figures & tables
Fig. 1 : Hardware experimental setup. Bottom: Prophesee EVK4 event camera capturing a strobing LED mounted on the drone. Inset: Close-up view of the drone with the LED.
Fig. 2 : Left: Estimated blob orientation using linear (Euclidean) mapping (red) versus an S1 -wrapped update (blue), with the ground-truth orientation shown in green. Right: Illustration of the corresponding rotation trajectories. Dashed curves denote intermediate states and solid ellipses denote final estimates. The Euclidean update follows a longer rotation path, whereas the S1 -wrapped update follows the shorter, geometrically consistent path. Dynamic visualization is presented in the supplementary video.
Fig. 3 : Left: GA-UKF tracks LED position and shape amid motion. The green ellipse represents all events in one batch update, the red circle represents true LED position, and the purple ellipse shows the estimated blob. Right: Continuous-time signal reconstruction using dual-threshold hysteresis, robustly recovering the binary sequence despite motion and noise.
Fig. 4 : Event pipeline: spatially filtered events are buffered, tracked with GA-UKF, and decoded via a shared buffer to reconstruct binary data in real time.
Fig. 5 : Processing time comparison between GA-UKF and EKF from [ 13 ] at varying signal frequencies.
Fig. 7 : Accuracy vs Distance between Camera and LED for different LED configurations and frequencies.
Speed (pixel/s)
Accuracy
Freq. 1 kHz
Freq. 5 kHz
Freq. 10 kHz
1500
94%
96%
97%
3000
92%
95%
96%
4500
91%
95%
95%
6000
73%
82%
86%
>10000
< 10%
< 10%
< 15%
TABLE I : Decoding accuracy with GA-UKF at different speeds and frequencies
Inter-robot communication is a key enabling technology in cooperative robotics applications. While wireless communication is ubiquitous, simple, and effective, it has inherent limitations: the receiving robot cannot easily localise the source of an incoming signal, clock synchronisation is challenging, and signals are broadcast to all nearby devices rather than targeted recipients. Optical communication provides a complementary channel that is inherently directional and spatially localised, directly addressing these limitations. Event cameras are bio-inspired dynamic vision sensors that respond to changes in image intensity with high temporal resolution, high dynamic range, and low latency, making them well-suited as receivers for high-rate optical communication in cooperative robotic systems. In this paper, we propose the Asynchronous Tracking and Optical Communication (ATOC) system, which integrates LED smart-beacon modulation, event-based detection, optical tracking, and event-data demodulation in a single pipeline. By simultaneously tracking and demodulating multiple LED smart beacons, ATOC transforms a conventional visual marker into a robust communication channel suitable for a wide range of high-impact robotics applications. We validate ATOC in a suite of laboratory studies and two 'applications': a smart-city demonstration and a 3D motion-capture system.
Ziwei Wang, Angus Apps, Holly Battisson +3
Systems Theory and Robotics Group, College of Engineering and Computer Science, Australian National University
We present a real-time upper-body human-to-humanoid motion imitation framework driven by neuromorphic event-based vision. This work addresses practical perceptual bottlenecks of conventional frame-based RGB sensors, specifically their difficulty in high dynamic range (HDR) scenes and rapid motions due to fixed integration times. By leveraging the Prophesee EVK4 event camera, which operates asynchronously with high temporal resolution and a dynamic range exceeding 120 dB, our system supports stable tracking in conditions where standard vision pipelines degrade, such as severe backlighting and very low light environments below 5 lux. The architecture integrates a low-latency Perception Module, utilizing optimized event accumulation and gravity-aligned inertial fusion, with a causal Motion Module (TWIST) that performs online kinematic retargeting. We validate the system on an embedded NVIDIA Booster T1 platform and an 18-DoF humanoid upper-body setup, demonstrating an end-to-end photon-to-action latency of 23-34 ms and advantages over RGB baselines under our experimental setup. The results indicate a practical trade-off: events can be preferable for fast or poorly lit upper-body teleoperation, whereas well-lit static scenes may favor RGB or hybrid sensing.
Haoyu Fu, Zhou Ge, Chengze Li +4
School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China. · SHU General Intelligent Robotics Research Institute, Baoshan District, Shanghai, 200444, China. · School of Physics, Changchun University of Science and Technology, Jilin, 130015, China.
Event cameras detect per-pixel brightness changes asynchronously on microsecond timescales, with high dynamic range and low power draw. These are desirable properties for robots that move fast or work in difficult lighting conditions. However, integrating an event camera into a real-world robotic pipeline still requires substantial effort: plug-and-play drivers do not exist, event streams are recorded in a variety of incompatible file formats, and most projects rely on custom research-grade code. Here, we present EventCV, an open-source and extensible Rust library with OpenCV-style Python bindings that lowers the entry barrier to working with event cameras. EventCV provides a wide range of features: denoising filters and geometric transforms, augmentations, corner detection and unsupervised feature learning, contrast-maximization motion estimation, a video-to-events simulator, and Open Neural Network Exchange (ONNX) inference for deployment in robotic stacks. EventCV integrates the Neuromorphic Drivers package, allowing an event camera stream to be processed directly in real time. No existing toolkit covers this range of operations in one package, and EventCV builds representations and decodes files 1.1x to 3.7x faster than the currently available libraries. We deploy EventCV on a Jetson Orin AGX and present three robotics case studies spanning object detection, on-device model inference, and localization. Project webpage: https://eventcv.net.
Adam D. Hines, Michael Milford, Tobias Fischer
QUT Centre for Robotics, School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, QLD 4000, Australia.