Organizations: Graduate School of Engineering, Tokyo University of Science, Tokyo, Japan · Faculty of Science and Technology, Sophia University, Tokyo, Japan
Event-based optical flow estimation using spatiotemporal planar approximation inherently assumes constant velocity and tends to average out temporal velocity variations. In this paper, we propose a flow estimation method that extends this planar approximation along the temporal axis. The resulting event-based optical flow constraint allows existing variational optical flow formulations to model temporal motion variation. Specifically, we partition the space-time volume into sub-intervals and apply a two-step planar approximation to track object edges consistently. Moreover, a spatial compensation mechanism and a temporal regularization term are incorporated to compensate for edge displacements and temporally stabilize the flow estimation, respectively. The experimental results show that modeling temporal motion variation improves flow estimation accuracy over conventional methods.
Figures & tables
Fig. 1: Concept of optical flow estimation based on spatiotemporal planar approximation. In subsequent figures, the distinction between positive and negative polarities is omitted, and only one polarity is illustrated.
Fig. 2: Relationship between edge-line translation and optical flow constraint for (u,v)⊤ at x∈(ℓ0∩R) : (left) constraint line from edge-line translation, (right) constraint line from brightness constancy equation.
Fig. 3: Overview of the proposed temporal division and sub-interval planar approximation.
Fig. 4: Spatial compensation of edge transitions of Fig. 3 . Gradients from sub-intervals (left) are spatially compensated to the reference position ℓ01 (right). This aggregates the spatiotemporal gradients of the same object at identical coordinates to maintain spatiotemporal consistency.
zurich_city_01_a
zurich_city_02_a
thun_00_a
Method
All
s-0-10
s-10-40
s-40+
All
s-0-10
s-10-40
s-40+
All
s-0-10
s-10-40
s-40+
Shiba et al. (Flow, L1 ) [ 8 ] a
6.17
3.16
10.64
41.76
7.60
5.75
10.57
35.81
3.73
3.02
4.96
27.40
Frame Flow b
8.37
3.90
15.28
46.16
14.87
5.07
18.34
44.89
7.89
3.40
16.67
44.62
Baseline [ 9 ]
3.38
2.27
4.84
20.55
4.24
2.37
4.85
22.44
4.10
2.45
7.32
28.60
+Time Div.
3.14
2.24
4.24
19.43
3.97
2.32
4.38
21.23
3.53
2.19
6.16
27.15
+Spatial Comp.
3.06
2.29
3.93
19.66
3.96
2.27
4.26
21.83
3.40
2.20
5.74
26.93
TABLE I: Optical Flow Estimation Results on the DSEC Dataset [ 17 ] (AEE [pixel] ↓ )
Fig. 5: Visual comparison of results. Rows from top to bottom correspond to the zurich_city_01_a , zurich_city_02_a , and thun_00_a sequences. Columns from left to right: Frame Image, Event Frame, Ground Truth (GT) Flow, GT Acceleration, Baseline Flow (without temporal division), Estimated Flow and Acceleration. Both estimated results are obtained by the proposed method with m=5 temporal divisions. The acceleration maps are shown only to illustrate the temporal variation captured by the proposed model; the primary evaluation criterion is the forward-flow AEE.
Estimating continuous optical flow is a fundamental yet challenging problem in dynamic visual perception. Event-based cameras, with microsecond latency and high dynamic range, capture brightness changes asynchronously, offering a unique opportunity to model motion with fine temporal precision. However, the scarcity of temporally dense ground-truth annotations limits the effectiveness of supervised learning, while contrast maximization (CM) frameworks, focused on sharpening the Image of Warped Events (IWE), often neglect temporal continuity and structural coherence, leading to distorted trajectories under complex motion. To overcome these challenges, we propose a hybrid-supervised framework for continuous-time optical flow estimation, grounded in the principle of Spatio-temporal Structural Consistency (STSC). This paradigm jointly enforces local structural stability and trajectory continuity, ensuring physically coherent motion across time. To further enhance representation and robustness, we design a bidirectionally complementary multi-scale architecture and employ a curriculum-guided hybrid training strategy, enabling a smooth transition from supervised point constraints to self-supervised manifold regularization. Comprehensive experiments across multiple benchmarks show that our method achieves state-of-the-art performance in both continuous-time and standard optical flow estimation, demonstrating the effectiveness of the proposed learning paradigm.
Temporally dense optical flow is essential for dynamic perception in immersive VR/AR systems, where rapid head, hand, and object motion must be continuously captured and tracked. Existing frame-based optical flow estimation methods are constrained by the tradeoff between temporal resolution and computational cost; while event cameras, with their high temporal resolution and energy efficiency, serve as a natural solution to the dilemma. However, event-based approaches commonly rely on correlation volumes to capture pairwise voxel correspondences, which incur substantial memory and computation overhead. We present E-WAVE, a correlation-free framework for high-temporal-resolution (HTR) optical flow estimation from event streams. Instead of constructing all-pairs correlation volumes, E-WAVE employs global attention mechanism to model long-range feature dependencies and performs trajectory guided feature warping using Bézier curve. Through iterative updates, it predicts trajectories that allow for querying at arbitrary timestamps without repeated inference. Experiments on MultiFlow and DSEC-Flow demonstrate a 25% lower trajectory error and comparable endpoint flow estimation accuracy relative to state-of-the art baselines. Additional evaluations on self-captured data using a head-mounted prototype validate that E-WAVE remains robust under challenging real-world conditions.
Jiale Wu, Xiaoyang Bai, Haoming Yu +3
Zhejiang University. · The University of Hong Kong.
Event cameras capture brightness changes asynchronously with microsecond resolution, yet existing optical flow methods fail to fully exploit this temporal continuity. Frame-based approaches impose artificial accumulation latency and suffer from domain overfitting, while model-based local methods operate statelessly, discarding temporal history between predictions and yielding inaccurate flows. We propose LC-Flow, the first temporally continuous, learning-based optical flow estimator that operates purely from local events. At its core, a Continuous Local Recurrent Network maintains persistent hidden states per spatial grid, incrementally accumulating temporal context as events arrive. Unlike frame-based methods constrained to fixed accumulation windows, and unlike stateless model-based methods that recompute motion from scratch at each step, LC-Flow produces sparse local flow estimates at arbitrary timestamps with full motion history. To address the inherent ambiguity of local observations, we jointly learn a confidence score that quantifies the reliability of each prediction, explicitly handling event sparsity and the aperture problem. This confidence serves a dual role: filtering unreliable estimates for downstream tasks such as visual odometry, and providing principled weights for a multi-scale confidence-guided aggregation that reconstructs globally consistent flow from the sparse local outputs. LC-Flow achieves state-of-the-art performance among local methods on both MVSEC and DSEC, while the confidence-guided aggregation establishes a new overall state-of-the-art on the MVSEC benchmark, surpassing heavy frame-based networks that rely on global spatial priors.