LiFR-Seg: Anytime High-Frame-Rate Segmentation via Event-Guided Propagation

Xiaoshan Wu; Xiaoyang Lyu; Yifei Yu; Bo Wang; Zhongrui Wang; Xiaojuan Qi

cs.CVMar 22, 2026

LiFR-Seg: Anytime High-Frame-Rate Segmentation via Event-Guided Propagation

Authors: Xiaoshan Wu, Xiaoyang Lyu, Yifei Yu, Bo Wang, Zhongrui Wang, Xiaojuan Qi

Organizations: The University of Hong Kong · Southern University of Science and Technology

Abstract

Dense semantic segmentation in dynamic environments is fundamentally limited by the low-frame-rate (LFR) nature of standard cameras, which creates critical perceptual gaps between frames. To solve this, we introduce Anytime Interframe Semantic Segmentation: a new task for predicting segmentation at any arbitrary time using only a single past RGB frame and a stream of asynchronous event data. This task presents a core challenge: how to robustly propagate dense semantic features using a motion field derived from sparse and often noisy event data, all while mitigating feature degradation in highly dynamic scenes. We propose LiFR-Seg, a novel framework that directly addresses these challenges by propagating deep semantic features through time. The core of our method is an uncertainty-aware warping process, guided by an event-driven motion field and its learned, explicit confidence. A temporal memory attention module further ensures coherence in dynamic scenarios. We validate our method on the DSEC dataset and a new high-frequency synthetic benchmark (SHF-DSEC) we contribute. Remarkably, our LFR system achieves performance (73.82% mIoU on DSEC) that is statistically indistinguishable from an HFR upper-bound (within 0.09%) that has full access to the target frame. This work presents a new, efficient paradigm for achieving robust, high-frame-rate perception with low-frame-rate hardware. Project Page: https://candy-crusher.github.io/LiFR_Seg_Proj/#; Code: https://github.com/Candy-Crusher/LiFR-Seg.git.

Figures & tables

Figure 1: Bridging the Perceptual Gap in High-Speed Scenarios. A critical "Blind Time Interval" for LFR systems is illustrated: (a) During $t$ to $t+\Delta t$ , a pedestrian rapidly enters the vehicle’s path. (b) A standard LFR system, constrained by discrete frames, Too Late detects danger only by $t+\Delta t$ . (c) In stark contrast, our HFR Anytime System leverages continuous events to detect imminent danger at $t+\delta t$ , providing crucial early warning and bridging this gap.

Figure 2: Overview of our LiFR-Seg framework. (a) The overall architecture. (b) The Splatting Module performs uncertainty-guided feature propagation using an event-driven motion field ( $\hat{\mathbf{M}}$ ) and its learned confidence ( $S$ ). (Note that $E_{t+\Delta t}$ is used strictly for training supervision to generate $Seg_{t+\Delta t}$ .) (c) The Memory Attention module refines the propagated feature by integrating historical context for long-term consistency.

Figure 3: Perception Paradigm Comparison. Visual definition of the four experimental settings: (a) The LFR (Baseline) , which is causal but not anytime-capable. (b) Interpolation-based methods, which are non-causal . (c) The original Event-Image Fusion paradigm. (d) Our (LiFR-Seg) framework, which is the only one that is both causal and anytime-capable .

Method	Input	CS	AT	DSEC	SHF	M3ED-D	M3ED-Q	D-Night
HFR (Ideal)	$I_{t+\delta t}$	✗	✗	73.91	65.40	64.57	69.27	41.83
LFR (Baseline)	$I_{t}$	✓	✗	67.67	61.73	55.23	63.20	37.44
LFR + Interpolation
TLX + Seg.	$I_{t},\mathbf{I_{t+\Delta t}},E...$	✗	✓	68.17	55.89	60.60	62.92	NaN
LFR + Fusion
EISNet	$I_{t},E_{t\to t+\delta t}$	✓	✓	68.11	61.28	58.34	62.98	37.28

Table 1: Comprehensive performance comparison across five diverse benchmarks. Our method is the only one satisfying the crucial causal (CS) and anytime (AT) constraints for real-world prediction. It not only bridges the perceptual gap by matching HFR performance on standard datasets but also demonstrates superior robustness in high-speed (M3ED) and low-light (DSEC-Night) scenarios. Results are reported at

\delta t=50

ms (DSEC, SHF, Night) and

\delta t=40

ms (M3ED).

Figure 4: Qualitative comparison of anytime interframe segmentation. The top row establishes the visual context, displaying the input RGB frame at time $t$ , the event stream from $t$ to $t+\delta t$ , and the target Ground Truth (GT) segmentation at time $t+\delta t$ . The bottom row presents a zoomed-in comparison of the GT against the outputs of all evaluated methods.

Figure 5: Anytime performance on SHF-DSEC. Our method (solid blue) remains stable, while baselines degrade as the temporal gap

\delta t

increases.

Method / Paradigm	mIoU (%)
LFR + Interpolation (Fig. 3 b)
TLX + Seg.	68.17
LFR + Fusion (Fig. 3 c*)
CMNeXt	70.13
Ours (LFR + Propagation) (Fig. 3 d)
w/ RAFT	72.93

Table 2: Ablation of different paradigms and their specific implementations on DSEC-Semantic. *Indicates inputs adapted for the anytime task.

Figure 6: Visualization of Uncertainty Map behavior. (a) Sparsity: Flow unsupported by sparse events triggers high uncertainty, suppressing hallucinations. (b) Inaccuracy: Disagreement between event edges and inaccurate flow triggers high uncertainty, filtering errors. (c)(d) Alignment: Consistent flow-event alignment (e.g., pedestrians, riders) yields high confidence for propagation.

Method	DSEC	SHF	Night
w/o Score	72.74	63.31	41.46
Ours	73.82	64.80	41.86

Table 3: Impact of ScoreNet.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Map	Description	Sequences	Usage
Town01	Small town featuring a river and bridges	1	Training
Town02	Small town with a mix of residential and commercial buildings	2	Training
Town03	Larger urban setting with a roundabout and multiple junctions	2	Training
Town04	Mountainous town with an infinite highway	1	Training
Town05	Grid-based town with cross-junctions, a bridge, and multi-lane directions	1	Training
Town10HD_Opt	Downtown area with skyscrapers, residential buildings, and an ocean promenade	1	Testing

Appendix

Table 6: SHF-DSEC Dataset Structure

	Event Camera	High-Speed RGB
Camera Type	Prophesee EVK4 HD	Phantom MTX-7510
Resolution	$1280\times 720$	$1280\times 640$
Max FPS	>1M	94K
Price (USD)	$5K	$150K
Power (W)	1.5	>325
Dynamic Range (dB)	>120	51

Appendix

Table 7: Event camera vs. high-speed RGB camera.

Method	Params (M)	Avg Cost ( $N=1$ )		Avg Cost ( $N=10$ )
Method	Params (M)	GFLOPs	FPS	GFLOPs	FPS
HFR Upper Bound	25.8	42.04	72.8	42.04	72.8
LFR + Fusion (EISNet)	34.5	72.73	34.7	72.73	34.7
LFR + Fusion (CMNeXt)	58.7	68.12	29.1	68.12	29.1
LFR + Interp. (TLX)	33.2	200.77	29.5	248.51	26.7
Ours (LiFR-Seg-Lite)	30.7	40.43	65.6	38.89	60.3

Appendix

Table 8: Computational cost comparison on RTX 3090. LiFR-Seg-Lite achieves the lowest amortized FLOPs while maintaining real-time speeds.

Explore similar work

Sep 22, 2026

LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Prediction

High-rate dense perception in dynamic environments is limited by the low update rate of RGB cameras, as rapid scene changes can occur between frames. Event cameras offer temporally dense but spatially sparse measurements, complementary to spatially dense RGB observations. Direct fusion cannot fully exploit this complementarity, while event-guided propagation fails on newly appearing or disoccluded regions without valid RGB support. We present LiFR v2, a unified propagation-completion-memory framework for causal anytime and streaming dense prediction from an RGB keyframe and events. LiFR v2 introduces an Event-Guided Completion Module (EGCM) to recover task-relevant representations where propagation is unsupported, and a History Retrieval Module (HRM) to reuse completed representations across successive queries. The framework supports semantic segmentation, monocular depth estimation, and multi-task dense prediction, and we further introduce SHF-Emerge to evaluate rapid object emergence and disocclusion. LiFR v2 achieves 74.37% mIoU on DSEC and 56.13% on SHF-Emerge, improving LiFR-Seg by 1.85 percentage points on the latter, while reducing SHF-Emerge depth RMSE from 1.564 m to 1.118 m over the propagation baseline. It also exceeds 100 FPS for both segmentation and depth, demonstrating accurate and efficient high-rate perception beyond RGB frame rates.

Tao Wan, Xiaoshan Wu, Yifei Yu +6

School of Microelectronics, Southern University of Science and Technology, Shenzhen 518055, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong, China

Event-Based Vision Monocular Depth Estimation Dense Prediction Semantic Segmentation

Ask PDF

Sep 18, 2026

Event-Frame Fusion for Inter-Frame Segmentation via Event-Guided Motion

Autonomous navigation requires precise and efficient semantic segmentation, yet existing frame-based approaches remain limited by motion blur, glare, latency, and the low temporal resolution (20-30 FPS) of conventional cameras, which leads to information loss between frames. Event cameras have emerged as an alternative sensing modality, capturing intensity changes asynchronously with high temporal resolution, high dynamic range, and sparse outputs. However, event-based algorithms still fall short of frame-based ones in accuracy, as most segmentation methods are designed for dense frame data. To overcome these limitations, we propose a hybrid vision architecture that combines conventional frame-based and event-based cameras. The system integrates two complementary components: (1) a compact Spiking Neural Network (SNN) with 42k parameters for motion estimation, and (2) a lightweight event-driven SNN with 0.84M parameters for frame-based semantic segmentation, which interpolates motion between frames to refine segmentation results. By predicting inter-frame segmentations, the framework achieves segmentation rates of up to 500 Hz with an energy consumption below 1.87 mJ per inference, while maintaining real-time GPU execution at frequencies up to 200 Hz. Additionally, our approach compensates for information loss in frames affected by blur or overexposure, enabling more robust perception in challenging conditions.

Dalia Hareb, Jean Martinet, Benoit Miramond +1

I3S, Côte d’Azur University, CNRS, France · LEAT, Côte d’Azur University, France · Zernike Institute, Groningen University, Netherlands

Semantic Segmentation Event-Based Vision

Ask PDF

Feb 24, 2026

Real-time Motion Segmentation with Event-based Normal Flow

Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in challenging scenarios. However, due to the sparse information content in individual events, directly processing the raw event data to solve vision tasks is highly inefficient, which severely limits the applicability of state-of-the-art methods in real-time tasks, such as motion segmentation, a fundamental task for dynamic scene understanding. Incorporating normal flow as an intermediate representation to compress motion information from event clusters within a localized region provides a more effective solution. In this work, we propose a normal flow-based motion segmentation framework for event-based vision. Leveraging the dense normal flow directly learned from event neighborhoods as input, we formulate the motion segmentation task as an energy minimization problem solved via graph cuts, and optimize it iteratively with normal flow clustering and motion model fitting. By using a normal flow-based motion model initialization and fitting method, the proposed system is able to efficiently estimate the motion models of independently moving objects with only a limited number of candidate models, which significantly reduces the computational complexity and ensures real-time performance, achieving nearly a 800x speedup in comparison to the open-source state-of-the-art method. Extensive evaluations on multiple public datasets fully demonstrate the accuracy and efficiency of our framework.

Sheng Zhong, Zhongyang Ren, Xiya Zhu +3

Neuromorphic Automation and Intelligence Lab (NAIL) at School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · Capital One. · University of Maryland, College Park.

Event-Based Vision

Ask PDF