Bio-Inspired Event-Based Visual Servoing for Ground Robots

Maral Mordad; Kian Behzad; Debojyoti Biswas; Noah J. Cowan; Milad Siami

cs.ROMar 24, 2026

Bio-Inspired Event-Based Visual Servoing for Ground Robots

Authors: Maral Mordad, Kian Behzad, Debojyoti Biswas, Noah J. Cowan, Milad Siami

Organizations: Department of Electrical & Computer Engineering, Northeastern University, Boston, MA 02115 USA · Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21218 USA · Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21218 USA

Abstract

Biological sensory systems are inherently adaptive, filtering out constant stimuli and prioritizing relative changes, likely enhancing computational and metabolic efficiency. Inspired by active sensing behaviors across a wide range of animals, this paper introduces a principled 1D event-based visual servoing framework for ground robots operating in structured environments. Utilizing a Dynamic Vision Sensor (DVS), we demonstrate that by applying a fixed spatial kernel to the asynchronous event stream generated from structured logarithmic intensity-change patterns, the resulting net event flux analytically isolates specific combinations of kinematic states. We establish a generalized theoretical bound for this event rate estimator and show that linear and quadratic spatial profiles isolate the robot's velocity and position-velocity product, respectively. Leveraging these properties, we employ a multi-pattern stimulus to directly synthesize a nonlinear state feedback term entirely without traditional state estimation. To overcome the inescapable loss of linear observability at equilibrium inherent in event sensing, we propose a bio-inspired active sensing limit-cycle controller. Experimental validation on a 1/10-scale autonomous ground vehicle confirms the efficacy, extreme low-latency, and computational efficiency of the proposed direct-sensing approach.

Figures & tables

Fig. 1: Experimental setup and control architecture. (A) A ground vehicle moves parallel to a monitor displaying quadratic and linear intensity patterns for an event camera. RGB axes denote the $x,y,z$ coordinate frames for the robot, camera, and world (origin at the pattern’s stabilization target). (B) Closed-loop EBVS with active sensing: Net event counts from the stimuli regulate vehicle-driven camera motion to stabilize it at the desired center.

Fig. 2: (A) Dual-pattern display with quadratic (top) and linear (bottom) intensity profiles. (B) Corresponding accumulated event stream, where red and blue dots indicate positive and negative polarity events, respectively and white background shows idle pixels (no event). The robot’s motion to the left relative to the pattern generates the observed events. Green rectangles denote the kernels $\mathcal{K}_{1}$ and $\mathcal{K}_{2}$ . Frame reconstruction is not required in our method; this figure is provided solely for illustration.

Fig. 3: Event-based state estimation vs. ground truth. Top: $x(t)\dot{x}(t)$ from $\mathcal{K}_{1}$ . Bottom: $\dot{x}(t)$ from $\mathcal{K}_{2}$ . Shaded regions indicate theoretical bounds. Ground truth for the robot’s position is provided by an array of six overhead motion capture cameras, while ground truth velocity is measured directly via the robot’s onboard wheel encoders.

Trial	Parameters				Results
Trial	$\sigma$	$k(\times 10^{-3})$	$v_{\max}$	$a_{\max}$ ( $\approx$ )	$\\|e_{\mathrm{q}}\\|$	$\\|e_{\mathrm{l}}\\|$
1	$330$	$1.299$	$0.45$	$2$	$0.000807$	$0.004303$
2	$330$	$1.299$	$0.65$	$3$	$0.001573$	$0.006937$
3	$330$	$2.205$	$0.45$	$2$	$0.000649$	$0.003578$
4	$330$	$2.205$	$0.65$	$3$	$0.001657$	$0.007203$
5	$430$	$1.299$	$0.45$	$2$	$0.000994$	$0.004303$

TABLE I: Estimation Errors by Target Profile and Velocity

Fig. 4: Closed-loop EBVS response for (A) fixed and (B) time-varying stabilization points. In both cases, in the left panel, the top plot shows position and the bottom plot shows velocity, with the desired oscillation radius $a$ indicated by dashed black lines. The right panels present the phase portraits, where green and red markers denote the start and end of the trajectory, respectively.

Vision-Based Robot Control

Vision-based robot control uses camera images to guide robots’ navigation and manipulation through closed-loop perception and action.

See all 221 papers

An Event-Driven Framework for Fly-Inspired Visual Motion Detection
Jul 6, 2026Qinbing Fu, Jingyu Huang, Yan Xie +2Event-Based Vision Visual Attention
School of Mathematics and Information Science, Guangzhou University, China
VibES: Induced Vibration for Persistent Event-Based Sensing
Aug 26, 2025Vincenzo Polizzi, Stephen Yang, Quentin Clark +3Event-Based Vision
University of Toronto, Robotics Institute · University of Toronto, Department of Mechanical and Industrial Engineering · University of Toronto +1
Event-based SLAM Benchmark for High-Speed Maneuvers
Apr 27, 2026Sheng Zhong, Junkai Niu, Guillermo Gallego +7Event-Based Vision Visual Odometry
Neuro-morphic Automation and Intelligence Lab (NAIL) at the School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · Department of Electrical Engineering and Computer Science, Technische Universität Berlin, the Science of Intelligence Excellence Cluster, the Robotics Institute Germany and the Einstein Center Digital Future, Berlin, Germany. · College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China. +2

Jul 6, 2026J/K move · Enter open · S save

An Event-Driven Framework for Fly-Inspired Visual Motion Detection

Qinbing Fu, Jingyu Huang, Yan Xie, Jigen Peng+1

School of Mathematics and Information Science, Guangzhou University, China

Event-Based Vision Visual Attention Spatiotemporal Reasoning

Open paper AskPDF arXiv

Fast and reliable motion detection is essential for machine vision and autonomous systems operating in dynamic environments. This work integrates emerging event-based sensing with biologically structured neural computation to establish an efficient computational paradigm for visual motion detection. The proposed framework is built upon a recently developed fly-inspired neural network that emulates motion-processing circuits in the optic lobe. Owing to its feed-forward and training-free architecture, the neural model requires only a small number of interpretable parameters and is well suited for real-time implementation. Event cameras provide low-latency, low-power, and high-dynamic-range visual sensing by asynchronously transmitting brightness-change events. However, their performance can be degraded by event noise, including temporal noise and junction-leakage-induced activity, particularly under low-light conditions. Moreover, effective integration between event-based visual representations and biologically inspired neural processing remains under-explored. To address these challenges, we propose an event-driven computational framework that combines time-surface encoding for front-end event representation with a fly optic-lobe-inspired neural network for foreground motion-direction estimation. A bottom-up attention mechanism is further incorporated to suppress background motion and enhance the saliency of foreground targets. The proposed method is evaluated via real-world datasets of ground-vehicle detection and compared with a baseline frame-based model and an optimization-based approach. Experimental results demonstrate that the framework effectively combines the temporal advantages of event-driven vision with the efficiency and interpretability of bio-inspired neural processing.

Abstract

Vision-Based Robot Control

Explore similar work

An Event-Driven Framework for Fly-Inspired Visual Motion Detection

VibES: Induced Vibration for Persistent Event-Based Sensing

Event-based SLAM Benchmark for High-Speed Maneuvers