Overview

Optical flow estimates a motion vector for each image pixel, helping systems track objects, stabilize video, guide robots, and interpolate or compress frames. Methods range from classical local and variational estimation to deep correlation-based networks such as RAFT, global matching transformers, and models that use pretrained vision features; event-camera methods handle asynchronous motion cues. Training commonly uses synthetic labeled data, while self-supervised approaches exploit photometric, geometric, and temporal consistency when labels are scarce. A central challenge is making accurate estimates transfer to real scenes with occlusion, large motion, lighting changes, and image corruption without excessive computation.

Latest papers

64 papers9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

Mar 17, 2026

x2x^2-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

Estimating dense 2D optical flow and 3D scene flow is essential for dynamic scene understanding. Recent work combines images, LiDAR, and event data to jointly predict 2D and 3D motion, yet most approaches operate in separate heterogeneous feature spaces. Without a shared latent space that all modalities can align to, these systems rely on multiple modality-specific blocks, leaving cross-sensor mismatches unresolved and making fusion unnecessarily complex.Event cameras naturally provide a spatiotemporal edge signal, which we can treat as an intrinsic edge field to anchor a unified latent representation, termed the Event Edge Space. Building on this idea, we introduce x2x^2-Fusion, which reframes multimodal fusion as representation unification: event-derived spatiotemporal edges define an edge-centric homogeneous space, and image and LiDAR features are explicitly aligned in this shared representation.Within this space, we perform reliability-aware adaptive fusion to estimate modality reliability and emphasize stable cues under degradation. We further employ cross-dimension contrast learning to tightly couple 2D optical flow with 3D scene flow. Extensive experiments on both synthetic and real benchmarks show that x2x^2-Fusion achieves state-of-the-art accuracy under standard conditions and delivers substantial improvements in challenging scenarios.
Mar 6, 2026

An Extended Topological Model For High-Contrast Optical Flow

In this paper, we identify low-dimensional models for dense core subsets in the space of 3×33\times 3 high-contrast optical flow patches sampled from the Sintel dataset. In particular, we leverage the theory of approximate and discrete circle bundles to identify a 3-manifold whose boundary is a previously proposed optical flow torus, together with disjoint circles corresponding to pairs of binary step-edge range image patches. The 3-manifold model we introduce provides an explanation for why the previously-proposed torus model could not be verified with direct methods (e.g., a straightforward persistent homology computation). We also demonstrate that nearly all optical flow patches in the top 1 percent by contrast norm are found near the family of binary step-edge circles described above, rather than the optical flow torus, and that these frequently occurring patches are concentrated near motion boundaries (which are of particular importance for computer vision tasks such as object segmentation and tracking). Our findings offer insights on the subtle interplay between topology and geometry in inference for visual data.
Feb 23, 2026

Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning

Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision -- expensive to obtain at scale and particularly scarce for dynamic real-world scenes. We present Flow3r, a framework that augments visual geometry learning with dense 2D correspondences (`flow') as supervision, enabling scalable training from unlabeled monocular videos. Our key insight is that the flow prediction module should be factored: predicting flow between two images using geometry latents from one and pose latents from the other. This factorization directly guides the learning of both scene geometry and camera motion, and naturally extends to dynamic scenes. In controlled experiments, we show that factored flow prediction outperforms alternative designs and that performance scales consistently with unlabeled data. Integrating factored flow into existing visual geometry architectures and training with ∼800{\sim}800K unlabeled videos, Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with its largest gains on in-the-wild dynamic videos where labeled data is most scarce.
Nov 22, 2024

Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions

Optical flow estimation is a fundamental task in dynamic scene understanding and computer vision. While recent deep models achieve impressive accuracy on clean benchmarks, their robustness to common visual and temporal degradations remains insufficiently explored. To address this gap, we establish a real-video-based spatiotemporal corruption benchmark suite that complements rendered and mixed-source robustness benchmarks, systematically evaluating two complementary aspects of visual perception: domain robustness and motion-distribution robustness. We construct two targeted benchmarks: KITTI Flow Corruptions (KITTI-FC) for dynamic outdoor scenes and GoPro Flow Corruptions (GoPro-FC) for diverse first-person-view complex motions. Both benchmarks use captured source data and apply 24 controlled, predominantly algorithmic corruptions encompassing digital, illumination, weather, noise, blur, and video artifacts. To quantify robustness, we use two complementary axes: the GT-free output-stability error Rs and the GT-based accuracy degradation Rd, together with their clean-error-normalized variants. We systematically evaluate 33 model variants from 17 representative optical flow methods. Our results show that, under the evaluated corruption schedules, models exhibit greater vulnerability to local structural information loss (e.g., JPEG compression and spatter) than to global perceptual distortions (e.g., fog and saturation). We further find that in-domain fine-tuning does not guarantee relative robustness and that no single current method dominates across all tested source and motion settings. Our work provides a controlled diagnostic framework for assessing optical flow robustness and developing reliable perception systems in complex environments. Code and benchmark resources are publicly available at https://github.com/ZhonghuaYi/optical_flow_robustness_benchmark.