Real-time Motion Segmentation with Event-based Normal Flow

Sheng Zhong; Zhongyang Ren; Xiya Zhu; Dehao Yuan; Cornelia Fermuller; Yi Zhou

cs.CVFeb 24, 2026

Real-time Motion Segmentation with Event-based Normal Flow

Authors: Sheng Zhong, Zhongyang Ren, Xiya Zhu, Dehao Yuan, Cornelia Fermuller, Yi Zhou

Organizations: Neuromorphic Automation and Intelligence Lab (NAIL) at School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · Capital One. · University of Maryland, College Park.

Abstract

Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in challenging scenarios. However, due to the sparse information content in individual events, directly processing the raw event data to solve vision tasks is highly inefficient, which severely limits the applicability of state-of-the-art methods in real-time tasks, such as motion segmentation, a fundamental task for dynamic scene understanding. Incorporating normal flow as an intermediate representation to compress motion information from event clusters within a localized region provides a more effective solution. In this work, we propose a normal flow-based motion segmentation framework for event-based vision. Leveraging the dense normal flow directly learned from event neighborhoods as input, we formulate the motion segmentation task as an energy minimization problem solved via graph cuts, and optimize it iteratively with normal flow clustering and motion model fitting. By using a normal flow-based motion model initialization and fitting method, the proposed system is able to efficiently estimate the motion models of independently moving objects with only a limited number of candidate models, which significantly reduces the computational complexity and ensures real-time performance, achieving nearly a 800x speedup in comparison to the open-source state-of-the-art method. Extensive evaluations on multiple public datasets fully demonstrate the accuracy and efficiency of our framework.

Figures & tables

Panel of Figure 1 — Fig. 1: The proposed system takes the normal flow generated by VecKM_Flow [ 11 ] as input and performs motion segmentation in real-time based on the normal flow constraint. (a) and (b) show the orientation and magnitude of the normal flow, respectively, with a circular color diagram in (a) indicating the angle-color correspondence. (c) presents the motion segmentation results of our system, with different colors representing distinct motion models. (d) compares the average runtime between our system and EMSGC [ 8 ] under identical setup. The mean operating frequency of each method are displayed above the corresponding box, with our system achieving a speedup of nearly 800 $\times$ compared to EMSGC.

Fig. 2: Flowchart of the proposed system. The proposed system comprises two independently operating modules. The data pre-processing module downsamples the input dense normal flow and constructs a spatial graph via Delaunay triangulation [ 39 ] . The motion segmentation module iteratively alternates between normal flow clustering (Labeling) and motion model fitting to segment the normal flow associated with IMOs.

Panel of Figure 3 — Fig. 3: Procedure of motion prediction. (a) Segmentation result at $t-1$ , with the black solid box indicating the system-generated region containing an IMO. (b) Normal flow input at $t$ , with the blue dashed box indicating the predicted IMO region after motion prediction. Normal flow within this region is used to initialize a candidate motion model. (c) Segmentation result at $t$ , where the IMO primarily resides within the predicted box.

Panel of Figure 4 — Fig. 4: Segmentation results on the EED dataset [ 14 ] . Time runs from left to right. The ground truth bounding boxes are denoted by red rectangles. Since the boxes are manually annotated on the grayscale images, and the timestamps cannot be perfectly aligned with the segmentation results, offsets are witnessed, especially for fast-moving IMOs.

Panel of Figure 5 — Fig. 5: Segmentation results on the EVIMO dataset [ 8 ] , on sequences Table (rows 1-2) and Boxed (rows 3-4). Time runs from left to right. The grayscale images are used for visualization only. The color of each label is determined by the number of normal flow associated with it, meaning that the label color for the same IMO may vary at different time. The ground truth mask is not perfectly aligned with the IMOs in the grayscale image due to the presence of motion blur.

Sequence	EMSMC [ 15 ]	EMSGC [ 8 ]	Ours
Fast drone	96.30	96.30	96.30
Lighting variation	80.51	93.51	98.70
Occlusions	92.31	100.00	100.00
What is background	100.00	100.00	100.00
Average	92.28	97.45	98.75

TABLE II: Quantitative evaluation on the EED dataset, with the evaluation metric being detection rate (in %).

Sequence	EMSGC [ 8 ]	Ours
box	0.30	0.56
table	0.46	0.55
Average	0.38	0.55

TABLE III: Quantitative evaluation on the EVIMO dataset, with the evaluation metric being IoU.

Panel of Figure 8 — Fig. 6: Qualitative comparison on the EMSGC dataset [ 8 ] . Time runs from left to right. Different colors indicate different motion models.

Node	Function	EMSGC [ 8 ]	Ours
Data Pre-processing	Pre-processing	87.65	25.50
Motion Segmentation	Initialization	5575.97	0.25
	Labeling & Fitting	10892.85	21.73
	Subtotal	16468.82	21.98

TABLE IV: Computational performance [time: ms].

Explore similar work

Feb 26, 2026

Motion-aware Event Suppression for Event Cameras

Event cameras report asynchronously per-pixel brightness changes with microsecond latency, encoding dynamic visual information as a sparse stream of events. However, their extreme temporal resolution floods perception systems with entangled events from ego-motion and independently moving objects (IMOs), which existing solutions fail to efficiently decouple, relying instead on prohibitive dense 3D reconstructions or limited hand-tuned filters. In this work, we introduce the first framework for Motion-aware Event Suppression, which learns to filter events triggered by IMOs and ego-motion in real time. Our model jointly segments IMOs in the current event stream while predicting their future motion, enabling anticipatory suppression of dynamic events before they occur. Our lightweight architecture achieves 173 Hz inference on consumer-grade GPUs with less than 1 GB of memory usage, outperforming previous state-of-the-art methods on the challenging EVIMO benchmark by 67% in segmentation accuracy while operating at a 53% higher inference rate. Moreover, we demonstrate significant benefits for downstream applications: our method accelerates Vision Transformer inference by 83% via token pruning and improves event-based visual odometry accuracy, reducing Absolute Trajectory Error (ATE) by 13%.

Roberto Pellerito, Nico Messikommer, Giovanni Cioffi +2

Robotics and Perception Group, University of Zurich, Switzerland

Event-Based Vision

Ask PDF

Sep 18, 2026

Event-Frame Fusion for Inter-Frame Segmentation via Event-Guided Motion

Autonomous navigation requires precise and efficient semantic segmentation, yet existing frame-based approaches remain limited by motion blur, glare, latency, and the low temporal resolution (20-30 FPS) of conventional cameras, which leads to information loss between frames. Event cameras have emerged as an alternative sensing modality, capturing intensity changes asynchronously with high temporal resolution, high dynamic range, and sparse outputs. However, event-based algorithms still fall short of frame-based ones in accuracy, as most segmentation methods are designed for dense frame data. To overcome these limitations, we propose a hybrid vision architecture that combines conventional frame-based and event-based cameras. The system integrates two complementary components: (1) a compact Spiking Neural Network (SNN) with 42k parameters for motion estimation, and (2) a lightweight event-driven SNN with 0.84M parameters for frame-based semantic segmentation, which interpolates motion between frames to refine segmentation results. By predicting inter-frame segmentations, the framework achieves segmentation rates of up to 500 Hz with an energy consumption below 1.87 mJ per inference, while maintaining real-time GPU execution at frequencies up to 200 Hz. Additionally, our approach compensates for information loss in frames affected by blur or overexposure, enabling more robust perception in challenging conditions.

Dalia Hareb, Jean Martinet, Benoit Miramond +1

I3S, Côte d’Azur University, CNRS, France · LEAT, Côte d’Azur University, France · Zernike Institute, Groningen University, Netherlands

Semantic Segmentation Event-Based Vision

Ask PDF

Aug 11, 2026

Static in Frames, Dynamic in Events: Rethinking Features in Event Cameras as Motion Cues

Event cameras capture intensity changes asynchronously with high temporal resolution, requiring novel preprocessing methods for downstream tasks. Unlike static intensity snapshots, event data inherently encode information about scene dynamics and object motion, meaning that features derived from events can exhibit behaviors with no direct analogue in frame-based vision. In this paper, we analyze two features used in event-based corner detection---the eigenvalues of the structure tensor and the spatiotemporal density values---and show that they are motion cues. We hypothesize that these features, combined with local geometric information, can enhance motion estimation tasks. To validate this, we first theoretically analyze how the eigenvalues of the structure tensor at moving corner points relate to the direction of motion. We then design controlled experiments on a synthetic dataset, confirming that extending local geometric features with eigenvalues and density values provides complementary motion information and is robust to texture and shot noise. Finally, we integrate the proposed features into a state-of-the-art event-based optical flow network and evaluate on the real-world DSEC benchmark, where the added features consistently improve accuracy, with the largest gains in data-scarce scenarios and for lower-capacity models. The code for this paper can be found at: https://github.com/hesamaraghi/static-in-frames-dynamic-in-events.

Hesam Araghi, Jan van Gemert, Nergis Tomen

Computer Vision Lab, Delft University of Technology

Optical Flow Event-Based Vision Event-Based Optical Flow Feature Fusion

Ask PDF