U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation

Xunpei Sun; Wenwei Lin; Yi Chang; Gang Chen

cs.CVApr 11, 2026

U $^{2}$ Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation

Authors: Xunpei Sun, Wenwei Lin, Yi Chang, Gang Chen

Organizations: Sun Yat-sen University, Guangzhou, China · Huazhong University of Science and Technology, Wuhan, China

Abstract

Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U $^{2}$ Flow, the first recurrent unsupervised framework that jointly estimates optical flow and per-pixel uncertainty. The core innovation is a decoupled learning strategy that derives uncertainty supervision from augmentation consistency via a Laplace-based maximum likelihood objective, enabling stable training without ground truth. The predicted uncertainty is further integrated into the network to guide adaptive flow refinement and dynamically modulate the regional smoothness loss. Furthermore, we introduce an uncertainty-guided bidirectional flow fusion mechanism that enhances robustness in challenging regions. Extensive experiments on KITTI and Sintel demonstrate that U $^{2}$ Flow achieves state-of-the-art performance among unsupervised methods while producing highly reliable uncertainty maps, validating the effectiveness of our joint estimation paradigm. The code is available at https://github.com/sunzunyi/U2FLOW.

Figures & tables

Figure 1: Comparison between previous optical flow estimation methods and our approach. (Left) Previous methods estimate only optical flow. (Right) Our proposed U 2 Flow framework jointly estimates optical flow and its uncertainty, and further leverages the predicted uncertainty to refine the flow estimation.

Panel of Figure 2 — Figure 2: Overview of U 2 Flow architecture. (a) The overall recurrent structure follows RAFT [ 50 ] . (b) The uncertainty-aware refinement module and the uncertainty estimation head respectively predict optical flow and per-pixel uncertainty in the recurrent update block.

Figure 3: Comparison of different masks for indicating flow errors. The masks are visualized as translucent green overlays on the optical flow error maps, where correct estimations are shown in blue , incorrect ones in red , and black pixels denote regions without ground truth. As shown in (c), some high-error regions remain unmasked, while certain low-error regions are incorrectly masked. In contrast, our method (d) accurately identifies high-error regions, providing more reliable cues for subsequent homography smoothness (Sec. 3.3 ), flow fusion (Sec. 3.4 ), and other downstream tasks.

[1.25pt]	Method	Sintel Clean		Sintel Final		KITTI 2012				KITTI 2015
		train	test	train	test	train	test			train	test
		EPE	EPE	EPE	EPE	EPE	Fl-all	Fl-noc	EPE	EPE	Fl-all	Fl-noc
Supervised	PWC-Net+ [ 48 ]	(1.71)	3.45	(2.34)	4.60	(0.99)	6.72	3.36	1.4	(1.47)	7.72	4.91
	RAFT [ 50 ]	(0.77)	1.61	(1.27)	2.86	–	–	–	–	(0.63)	5.10	3.07
	FlowFormer [ 11 ]	(0.48)	1.16	(0.74)	2.09	–	–	–	–	(0.53)	4.68	2.69
	VideoFlow [ 41 ] MF	(0.46)	0.99	(0.66)	1.62	–	–	–	–	(0.56)	3.65	–

Table 1: Quantitative results on Sintel and KITTI online benchmarks. Metrics evaluated at “all” (all pixels), “noc” (non-occlusions). MF denotes methods trained using multi-frame data.

{\dagger}

denotes models with semantic inputs. “+FF” denotes our bidirectional flow fusion module. Missing entries (–) denote unreported results. Parentheses indicate that training and testing are conducted on the same dataset.

[1.25pt] Method	Sintel		KITTI
	AUSE $\downarrow$	CC $\uparrow$	AUSE $\downarrow$	CC $\uparrow$
FB Check [ 34 ]	0.19	0.57	0.21	0.53
PDC-Net+ [ 51 ]	0.18	0.45	0.16	0.50
Smoothness [ 1 ]	0.26	0.43	0.23	0.45
U 2 Flow (Ours)	0.11	0.66	0.12	0.64
[1.25pt]

Table 2: Comparison of uncertainty estimation performance on the Sintel (final, clean) and KITTI (2012, 2015) training sets.

Figure 4: Sparsification curves for uncertainty evaluation. Lower AUSE (shown in parentheses) is better.

Figure 5: Qualitative results on the KITTI test set (sample frames #5 and #9) and the Sintel (final pass) test set (samples: ambush_3 , frame 23; cave_3 , frame 16), compared with SMURF [ 44 ] . Additional examples can be found on the official benchmark website.

[1.25pt] UE	Dec.	FR	$\ell_{\mathrm{hg}}$	FF	Sintel		KITTI 2015
					Final	Clean	EPE	Fl-all
-	-	-	-	-	2.46	1.56	1.98	6.82
$\checkmark$	-	-	-	-	2.57	1.65	2.18	7.72
$\checkmark$	$\checkmark$	-	-	-	2.41	1.44	1.98	6.87
$\checkmark$	$\checkmark$	$\checkmark$	-	-	2.32	1.42	1.95	6.85
$\checkmark$	$\checkmark$	-	$\checkmark$	-	2.52	1.48	1.87	6.69

Table 3: Ablation study on key components. We evaluate the impact of Uncertainty Estimation (UE), Decoupling of flow and uncertainty learning (Dec.), Flow Refinement (FR), uncertainty-enhanced

\ell_{\mathrm{hg}}

, and Flow Fusion (FF). The

\bigcirc

symbol indicates that

\ell_{\mathrm{hg}}

is applied only on the KITTI dataset.

[1.25pt] Model Variant	Sintel		KITTI 2015
	Final	Clean	EPE	Fl-all
w/o Refinement	2.41	1.44	1.87	6.69
Refinement w/o Uncertainty	2.42	1.51	1.93	6.81
\SetRow bg=gray!20 Refinement w/ Uncertainty	2.32	1.42	1.83	6.59
[1.25pt]

Table 4: Ablation study on the flow refinement module.

[1.25pt] Method	Sintel		KITTI 2015
	Final	Clean	Fl-all	Fl-noc	Fl-occ
w/o Fusion	2.32	1.42	6.59	5.10	16.01
Occ Mask	2.34	1.42	8.17	5.07	27.73
\SetRow bg=gray!20 Unc Mask (Ours)	2.29	1.36	6.30	5.05	14.17
[1.25pt]

Table 5: Ablation study on the bidirectional flow fusion module.

[1.25pt] Method	KITTI $\rightarrow$ Sintel		Sintel $\rightarrow$ KITTI
	Final	Clean	EPE	Fl-all
Ours (w/o Fusion)	5.80	4.67	5.10	17.05
Ours (w/ Fusion)	5.49	4.23	4.77	16.58
[1.25pt]

Table 6: Generalization ability. Training on one dataset and testing directly on the other dataset.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Detailed illustration of the recurrent update block. The diagram depicts the process flow for the $k$ -th iteration. Components to the left of the dashed line represent the original RAFT [ 50 ] architecture, while components to the right constitute our proposed flow refinement module and uncertainty estimation head.

Figure 7: Schematic diagram of the self-supervision process. (1) An initial flow $\mathbf{F}_{1\rightarrow 2}$ is predicted from the original image pair. (2) Strong augmentations are applied to both the images and the flow, creating an augmented image pair and a transformed pseudo-ground-truth flow $\hat{\mathbf{F}}_{1\rightarrow 2}$ . (3) A new flow prediction $\hat{\mathbf{F}}^{\prime}_{1\rightarrow 2}$ is generated from the augmented images. (4) The inconsistency $\hat{D}$ between the pseudo-ground-truth and the new prediction serves as the target signal for both the augmentation loss ( $\ell_{\mathrm{ar}}$ ) and the uncertainty supervision loss ( $\ell_{\mathrm{unc}}$ ).

[1.25pt] Iter. Num.	Sintel Final				KITTI 2015
	EPE $\downarrow$	AUSE $\downarrow$	CC $\uparrow$	EPE $\downarrow$	Fl-all $\downarrow$	AUSE $\downarrow$	CC $\uparrow$	Runtime
2	3.52	0.11	0.69	3.71	12.31	0.20	0.53	18 ms
4	2.75	0.10	0.69	2.32	8.23	0.21	0.50	35 ms
8	2.41	0.11	0.68	1.90	6.75	0.22	0.48	52 ms
12	2.32	0.11	0.67	1.83	6.59	0.22	0.48	66 ms
16	2.30	0.11	0.67	1.82	6.53	0.22	0.47	97 ms

Appendix

Table 7: Ablation study on the number of recurrent iterations (

K

). Flow accuracy improves and saturates with more iterations, while uncertainty estimation (AUSE) remains robustly stable. Runtime is measured on KITTI-2015.

[1.25pt] Method	Sintel		KITTI
	AUSE $\downarrow$	CC $\uparrow$	AUSE $\downarrow$	CC $\uparrow$
Appearance Aug. Only	0.46	0.25	0.25	0.51
Spatial Aug. Only	0.12	0.65	0.12	0.64
Full (Ours)	0.11	0.66	0.12	0.64
[1.25pt]

Appendix

Table 8: Ablation study of augmentation strategies for uncertainty.

Figure 8: Detailed test results of our model on Sintel [ 4 ] .

Panel of Figure 9 — Figure 9: Detailed test results of our final model on KITTI [ 9 , 35 ] .

[1.25pt] Model Variant	Sintel		KITTI 2015
	Final	Clean	EPE	Fl-all
w/o Uncertainty-Scaled	2.40	1.52	2.04	7.24
w/ Uncertainty-Scaled (Ours)	2.32	1.42	1.83	6.59
[1.25pt]

Appendix

Table 9: Ablation study on the uncertainty scaling mechanism. Explicitly modulating flow representations with the scaled feature

\tilde{\mathbf{f}}^{(k)}

yields better performance than implicit direct concatenation.

Figure 10: Visualization of selected regions for homography smoothness on Sintel and KITTI. The top two rows show samples from the Sintel dataset, while the bottom two rows are from KITTI. The uncertainty-based reliability mask successfully identifies regions with high flow error in both datasets. However, for Sintel, these high-error regions are typically non-rigid characters for which the planar homography assumption does not hold. In contrast, for KITTI, high-error regions often correspond to rigid vehicles and planar road surfaces, which are well-suited for homography regularization. This illustrates why the uncertainty-enhanced $\ell_{\mathrm{hg}}$ is effective on KITTI but not on Sintel.

Figure 11: Qualitative results on the Spring benchmark [ 33 ] without fine-tuning.

[1.25pt] Model	1px												EPE	Fl	WAUC
	total	low-det.	high-det.	matched	unmat.	rigid	non-rig.	not sky	sky	s0-10	s10-40	s40+
RAFT [ 50 ]	6.79	6.43	64.09	6.00	39.48	4.11	27.09	5.25	30.18	3.13	5.30	41.40	1.476	3.20	90.92
GMA [ 15 ]	7.07	6.70	66.20	6.28	39.89	4.28	28.25	5.61	29.26	3.65	5.39	40.33	0.914	3.08	90.72
GMFlow [ 55 ]	10.36	9.93	76.61	9.06	63.95	6.80	37.26	8.95	31.68	5.41	9.90	52.94	0.945	2.95	82.34
FlowFormer [ 11 ]	6.51	6.14	64.22	5.77	37.29	3.53	29.08	5.50	21.86	3.38	5.53	35.34	0.723	2.38	91.68
[dashed] SMURF [ 44 ]	6.43	6.04	66.52	5.49	45.12	3.58	27.95	5.53	20.07	3.44	6.59	30.99	0.659	2.10	90.45

Appendix

Table 10: Optical flow generalization results on the Spring benchmark [ 33 ] without fine-tuning. We report the 1px outlier rate across low/high-detail, (un)matched, (non-)rigid, and (non-)sky regions, along with the EPE, Fl error, and WAUC [ 38 ] metrics. The best and second-best results are highlighted in bold and underline, respectively, while key metrics are emphasized in blue.

Figure 12: Additional qualitative results on the KITTI-2015 test set.

Figure 13: Additional qualitative results on the Sintel (final pass) test set.

Explore similar work

GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning
Mar 23, 2026Yixuan Luo, Feng Qiao, Zhexiao Xiong +2Optical Flow Synthetic Data Generation
University of Chicago · Washington University in St. Louis
FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion
Jul 11, 2026Yuang Meng, Chenyang Wu, Xianshun Liu +7Optical Flow Diffusion Model Guidance
VCIP, CS, Nankai University · OPPO Research Institute · The Hong Kong Polytechnic University
FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow
Mar 30, 2026Sadra Safadoust, Fabio Tosi, Matteo Poggi +1Optical Flow
Department of Computer Engineering and KUIS AI Center, Koç University, Istanbul, Turkey · Department of Computer Science and Engineering (DISI), University of Bologna, Italy

Mar 23, 2026J/K move · Enter open · S save

GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning

Yixuan Luo, Feng Qiao, Zhexiao Xiong, Yanjing Li+1

University of Chicago · Washington University in St. Louis

Optical Flow Synthetic Data Generation

Open paper AskPDF arXiv

Optical flow estimation is a fundamental problem in computer vision, yet the reliance on expensive ground-truth annotations limits the scalability of supervised approaches. Although unsupervised and semi-supervised methods alleviate this issue, they often suffer from unreliable supervision signals based on brightness constancy and smoothness assumptions, leading to inaccurate motion estimation in complex real-world scenarios. To overcome these limitations, we introduce \modelname, a novel framework that synthesizes large-scale, perfectly aligned frame--flow data pairs for supervised optical flow training without human annotations. Specifically, our method leverages a pre-trained depth estimation network to generate pseudo optical flows, which serve as conditioning inputs for a next-frame generation model trained to produce high-fidelity, pixel-aligned subsequent frames. This process enables the creation of abundant, high-quality synthetic data with precise motion correspondence. Furthermore, we propose an inconsistent pixel filtering strategy that identifies and removes unreliable pixels in generated frames, effectively enhancing fine-tuning performance on real-world datasets. Extensive experiments on KITTI2012, KITTI2015, and Sintel demonstrate that \modelname achieves competitive or superior results compared to existing unsupervised and semi-supervised approaches, highlighting its potential as a scalable and annotation-free solution for optical flow learning. We will release our code upon acceptance.

Abstract

Explore similar work

GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning

FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion

FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow