Optical flow estimation is a fundamental problem in computer vision, yet the reliance on expensive ground-truth annotations limits the scalability of supervised approaches. Although unsupervised and semi-supervised methods alleviate this issue, they often suffer from unreliable supervision signals based on brightness constancy and smoothness assumptions, leading to inaccurate motion estimation in complex real-world scenarios. To overcome these limitations, we introduce \modelname, a novel framework that synthesizes large-scale, perfectly aligned frame--flow data pairs for supervised optical flow training without human annotations. Specifically, our method leverages a pre-trained depth estimation network to generate pseudo optical flows, which serve as conditioning inputs for a next-frame generation model trained to produce high-fidelity, pixel-aligned subsequent frames. This process enables the creation of abundant, high-quality synthetic data with precise motion correspondence. Furthermore, we propose an inconsistent pixel filtering strategy that identifies and removes unreliable pixels in generated frames, effectively enhancing fine-tuning performance on real-world datasets. Extensive experiments on KITTI2012, KITTI2015, and Sintel demonstrate that \modelname achieves competitive or superior results compared to existing unsupervised and semi-supervised approaches, highlighting its potential as a scalable and annotation-free solution for optical flow learning. We will release our code upon acceptance.
Figures & tables
Figure 1: Visualization of the synthesized data triplet on KITTI2012, including the reference frame It , the artificially generated optical flow F~t→t+1 , and the conditioned next-frame generation prediction result I~t+1 .
Figure 2: Overview of the conditioned next-frame framework and artificial optical flow generation. Given an input view and its corresponding optical flow, our framework constructs two types of embeddings: a 2D coordinate embedding of the input view and a warped coordinate embedding of the target view derived from the optical flow. A semantic preserver network extracts high-level semantic features from the input view, while a diffusion model conditioned on these embeddings learns the geometric warping necessary to generate the novel view and accurately align pixel-level motion. To further enhance spatial correspondence, we augment the self-attention mechanism with cross-view attention and jointly aggregate features across views. Notably, ground-truth optical flow is used only during pre-training, while synthetic datasets are constructed using artificial optical flow produced via a geometry-based camera-motion model.
Methods
Pretrained
Middlebury 2014
KITTI 2015
Model
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Leave blank
-
11.328
0.315
0.450
12.980
0.374
0.313
Stretch
-
14.842
0.432
0.285
14.757
0.429
0.212
3D Photography [ 44 ]
-
14.190
0.427
0.275
14.540
0.398
0.210
RePaint [ 30 ]
-
15.102
0.462
0.311
15.056
0.462
0.251
SD-Inpainting [ 47 ]
-
15.740
0.412
0.311
9.792
0.230
0.652
Table 1: Quantitative comparison of motion-conditioned image generation methods on the Middlebury 2014 and KITTI 2015 datasets. Evaluation metrics include PSNR ( ↑ ), SSIM ( ↑ ), and LPIPS ( ↓ ). For the Middlebury 2014 dataset, we report results using optical flow estimated by RAFT and WAFT, while the KITTI 2015 dataset provides ground-truth optical flow for evaluation. Qualitative results can be found in Supp A
Method
Without GenOpticalFlow
With GenOpticalFlow
EPE ↓
Fl-all ↓
EPE ↓
Fl-all ↓
RAFT-chairs [ 52 ]
9.82
37.55
8.18
20.59
GMFlowNet-things [ 71 ]
4.43
16.70
4.24
16.68
FlowFormer-chairs [ 15 ]
10.14
34.46
7.45
21.19
Flowformer++-chairs [ 43 ]
10.45
36.97
7.98
22.01
FlowDiffuser-things [ 31 ]
3.61
11.8
3.52
10.69
Table 2: Quantitative zero-shot evaluation on the KITTI 2015 benchmark. We report the EPE and Fl-all metrics for six representative optical flow models, both with and without integrating our proposed GenOpticalFlow. Across all methods, incorporating GenOpticalFlowconsistently reduces error, demonstrating its strong generalization ability and compatibility with diverse architectures. Lower values indicate better performance.
Middlebury
KITTI 2015
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
w/o CVA
17.93
.417
.314
15.36
.291
.290
w/o Coord
17.15
.508
.291
14.63
.384
.314
Ours
19.24
.653
.217
17.85
.552
.268
Table 3: Ablation Study.
Table 6
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Qualitative results of optical flow generation on the KITTI (top) and Sintel (bottom) datasets. In each panel, the first row displays the conditioning input frame, while the second row visualizes a corresponding optical flow map randomly sampled from our model. The results highlight the model’s ability to generate dense, structurally aligned flow predictions across varying scene complexities.
Figure 4: Visualization of next-frame generation results. The figure demonstrates the temporal consistency of our method on the KITTI and Sintel benchmarks. The top rows show the reference input frames, while the bottom rows display the synthesized next frames. Our framework effectively preserves texture details and object geometry during the generation process.
Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U2Flow, the first recurrent unsupervised framework that jointly estimates optical flow and per-pixel uncertainty. The core innovation is a decoupled learning strategy that derives uncertainty supervision from augmentation consistency via a Laplace-based maximum likelihood objective, enabling stable training without ground truth. The predicted uncertainty is further integrated into the network to guide adaptive flow refinement and dynamically modulate the regional smoothness loss. Furthermore, we introduce an uncertainty-guided bidirectional flow fusion mechanism that enhances robustness in challenging regions. Extensive experiments on KITTI and Sintel demonstrate that U2Flow achieves state-of-the-art performance among unsupervised methods while producing highly reliable uncertainty maps, validating the effectiveness of our joint estimation paradigm. The code is available at https://github.com/sunzunyi/U2FLOW.
Xunpei Sun, Wenwei Lin, Yi Chang +1
Sun Yat-sen University, Guangzhou, China · Huazhong University of Science and Technology, Wuhan, China
Accurate estimation of large displacement optical flow remains a critical challenge. Existing methods typically rely on iterative local search or/and domain-specific fine-tuning, which severely limits their performance in large displacement and zero-shot generalization scenarios. To overcome this, we introduce MegaFlow, a simple yet powerful model for zero-shot large displacement optical flow. Rather than relying on highly complex, task-specific architectural designs, MegaFlow adapts powerful pre-trained vision priors to produce temporally consistent motion fields. In particular, we formulate flow estimation as a global matching problem by leveraging pre-trained global Vision Transformer features, which naturally capture large displacements. This is followed by a few lightweight iterative refinements to further improve the sub-pixel accuracy. Extensive experiments demonstrate that MegaFlow achieves state-of-the-art zero-shot performance across multiple optical flow benchmarks. Moreover, our model also delivers highly competitive zero-shot performance on long-range point tracking benchmarks, demonstrating its robust transferability and suggesting a unified paradigm for generalizable motion estimation. Our project page is at: https://kristen-z.github.io/projects/megaflow.
Dingxi Zhang, Fangjinhua Wang, Marc Pollefeys +1
ETH Zurich, Switzerland · Microsoft, Switzerland · University of Tübingen, Tübingen AI Center, Germany
We present FlowIt, a novel architecture for optical flow estimation that combines global matching with confidence and occlusion-guided refinement. At its core, FlowIt leverages a hierarchical transformer architecture that captures extensive global context, enabling the model to effectively model long-range correspondences. To overcome the limitations of localized matching, we formulate the flow initialization as an optimal transport problem. This formulation yields a highly robust initial flow field, alongside explicitly derived occlusion and confidence maps. These cues are then seamlessly integrated into a guided refinement stage, where the network actively propagates reliable motion estimates from high-confidence regions into ambiguous, low-confidence areas. Extensive experiments across the Sintel, KITTI, Spring, and LayeredFlow datasets validate the effectiveness of our approach. FlowIt achieves state-of-the-art results on the competitive Sintel benchmark and establishes new state-of-the-art cross-dataset zero-shot generalization performance on Sintel, Spring, and LayeredFlow, while also delivering competitive performance on both the KITTI benchmark and KITTI zero-shot generalization settings.
Sadra Safadoust, Fabio Tosi, Matteo Poggi +1
Department of Computer Engineering and KUIS AI Center, Koç University, Istanbul, Turkey · Department of Computer Science and Engineering (DISI), University of Bologna, Italy