Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions

Zhonghua Yi; Hao Shi; Qi Jiang; Yao Gao; Ze Wang; Yufan Zhang; Kailun Yang; Kaiwei Wang

eess.IVNov 22, 2024

Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions

Authors: Zhonghua Yi, Hao Shi, Qi Jiang, Yao Gao, Ze Wang, Yufan Zhang, Kailun Yang, Kaiwei Wang

Organizations: Zhejiang University · Hunan University

Abstract

Optical flow estimation is a fundamental task in dynamic scene understanding and computer vision. While recent deep models achieve impressive accuracy on clean benchmarks, their robustness to common visual and temporal degradations remains insufficiently explored. To address this gap, we establish a real-video-based spatiotemporal corruption benchmark suite that complements rendered and mixed-source robustness benchmarks, systematically evaluating two complementary aspects of visual perception: domain robustness and motion-distribution robustness. We construct two targeted benchmarks: KITTI Flow Corruptions (KITTI-FC) for dynamic outdoor scenes and GoPro Flow Corruptions (GoPro-FC) for diverse first-person-view complex motions. Both benchmarks use captured source data and apply 24 controlled, predominantly algorithmic corruptions encompassing digital, illumination, weather, noise, blur, and video artifacts. To quantify robustness, we use two complementary axes: the GT-free output-stability error Rs and the GT-based accuracy degradation Rd, together with their clean-error-normalized variants. We systematically evaluate 33 model variants from 17 representative optical flow methods. Our results show that, under the evaluated corruption schedules, models exhibit greater vulnerability to local structural information loss (e.g., JPEG compression and spatter) than to global perceptual distortions (e.g., fog and saturation). We further find that in-domain fine-tuning does not guarantee relative robustness and that no single current method dominates across all tested source and motion settings. Our work provides a controlled diagnostic framework for assessing optical flow robustness and developing reliable perception systems in complex environments. Code and benchmark resources are publicly available at https://github.com/ZhonghuaYi/optical_flow_robustness_benchmark.

Figures & tables

Aspect	Ours	RobustSpring	FlowBench
Data source	Captured KITTI + GoPro	Rendered Spring	Mixed datasets
Asymmetric frame pairs	$\checkmark$	$\checkmark$	$\times$
Video pipeline (H.264/bit error)	$\checkmark$	$\times$	$\times$
Motion control	10/30 FPS	$\times$	$\times$
Main severity reporting	Five-level average	One level	Level 3
GT-free output stability	$R^{s}$ : prediction shift	$R^{C}_{\mathrm{EPE}}$ : prediction shift	$\times$

Table I: Comparison with closely related optical-flow robustness benchmarks. A check mark indicates that the capability is explicitly included in the reported benchmark protocol.

Figure 1: Effects of the 24 corruptions at severity 5. For temporal corruptions, we display the previous and next frames (left/right) to reveal inter-frame differences, which are crucial for diagnosing temporal robustness in optical flow. Examples are from GoPro-FC.

Figure 2: PSFs at severity levels 1–5 (top to bottom). Four field positions show spatial variation in blur strength and shape; enlarged PSF details appear on the right.

Metric	GT	Clean-EPE norm.	Interpretation
$R^{s}$	No	No	Output shift; lower is more stable
$R^{d}$	Yes	No	Signed accuracy degradation
$R^{s}_{\mathrm{rel}}$	Yes	Yes	Relative output-stability error
$R^{d}_{\mathrm{rel}}$	Yes	Yes	Relative accuracy degradation

Table II: Interpretation of the diagnostic metric suite.

Figure 3: Complementary robustness axes. $R^{s}$ compares corrupted and clean predictions and does not require ground truth. $R^{d}$ uses flow GT to measure the signed EPE change from the clean baseline.

Figure 4: Optical flow amplitude statistics of KITTI-FC and GoPro-FC. GoPro-FC-30FPS exhibits more small displacements, while GoPro-FC-10FPS has displacements similar to KITTI-FC.

Model	Type	Encoder	Decoder
Farnebäck [ 27 ]	Knowledge-driven	-	-
EpicFlow [ 28 ]	Knowledge-driven	-	-
DIS [ 61 ]	Knowledge-driven	-	-
RAFT [ 3 ]	Supervised	CNN	GRU
GMA [ 62 ]	Supervised	CNN	GRU
CSFlow [ 63 ]	Supervised	CNN	GRU

Table III: Evaluated methods. Encoder and decoder architectures are listed where applicable.

Figure 5: Relationships among clean EPE, output stability, and accuracy degradation. The plots compare EPE, $R^{s}$ , $R^{d}$ , $R^{s}_{\mathrm{rel}}$ , and $R^{d}_{\mathrm{rel}}$ after reference normalization. The observed associations are descriptive and do not establish architectural causality.

Model	Clean	Digital				Illumination				Weather				Noise			Blur					RANK
Model	Clean	JPEG	Pixelate	Contrast	Saturate	HL *	LL *	OE *	UE *	Spatter	Fog	Frost	Snow	Gaussian	Shot	Impulse	Gaussian	Defocus	Glass	Camera	PSF	RANK
Knowledge-driven models
Farnebäck	25.6	2.3	2.4	10.2	3.3	4.0	4.5	7.5	6.0	6.2	8.0	8.1	9.7	5.5	5.5	6.0	8.1	8.7	5.8	6.6	7.8	3
EpicFlow	9.4	7.4	2.3	5.4	4.0	3.2	6.4	8.4	5.6	16.8	5.0	21.6	13.0	20.1	26.2	18.5	3.2	3.5	5.2	2.8	3.1	2
DIS	20.6	1.9	0.9	1.9	2.3	2.7	3.7	8.0	10.8	5.4	5.8	8.9	6.5	2.5	2.3	2.7	1.6	1.8	2.0	2.3	2.0	1
Out-Of-Domain (OOD) Supervised Models

Table IV:

R^{s}

results on KITTI-FC. HL: High-light , LL: Low-light , OE: Over Exposure , UE: Under Exposure .

Model	Clean	Digital				Illumination				Weather				Noise			Blur					RANK
Model	Clean	JPEG	Pixelate	Contrast	Saturate	HL *	LL *	OE *	UE *	Spatter	Fog	Frost	Snow	Gaussian	Shot	Impulse	Gaussian	Defocus	Glass	Camera	PSF	RANK
Knowledge-driven models
Farnebäck	25.6	0.4	1.1	7.2	-0.3	0.2	1.9	2.0	3.0	2.2	5.6	4.1	2.0	-1.3	-1.3	-1.4	5.4	5.8	3.2	3.2	5.0	2
EpicFlow	9.4	4.8	0.5	2.7	2.2	1.5	4.1	6.3	3.2	15.2	3.0	19.8	10.6	17.8	23.9	16.3	0.7	1.0	3.1	0.5	0.5	1
DIS	20.6	0.5	-0.1	0.3	-0.2	-0.5	1.4	3.8	6.7	3.8	4.0	6.7	2.6	0.8	0.5	0.9	-0.4	-0.4	-0.2	-0.2	-0.4	3
Out-Of-Domain (OOD) Supervised Models

Table V:

R^{d}

results on KITTI-FC. HL: High-light , LL: Low-light , OE: Over Exposure , UE: Under Exposure .

Corruption		RAFT	GMA	CSFlow	SKFlow	GMFlowNet	CRAFT	FF **	FF++ **	SAMFlow-H
Digital	JPEG	1.3 / 3.1	1.2 / 2.3	1.4 / 2.8	0.8 / 1.7	1.4 / 3.0	0.8 / 1.7	1.6 / 2.7	1.3 / 2.9	1.3 / 2.7
	Pixelate	0.2 / 0.6	0.2 / 0.6	0.2 / 0.5	0.2 / 0.5	0.2 / 0.6	0.2 / 0.6	0.3 / 0.7	0.4 / 0.9	0.4 / 0.7
	Contrast	0.3 / 0.9	0.3 / 0.8	0.3 / 0.8	0.2 / 0.7	0.2 / 0.7	0.3 / 1.0	0.3 / 0.9	0.4 / 0.9	0.4 / 0.8
	Saturate	0.4 / 1.4	0.3 / 0.8	0.5 / 1.3	0.3 / 0.9	0.4 / 1.2	0.3 / 0.9	0.4 / 1.0	0.4 / 1.0	0.4 / 1.4
Illu.	HL *	0.2 / 0.8	0.2 / 0.6	0.2 / 0.7	0.2 / 0.6	0.2 / 0.7	0.2 / 0.6	0.2 / 0.6	0.2 / 0.6	0.2 / 0.6
	LL *	0.4 / 1.3	0.3 / 1.0	0.4 / 1.5	0.3 / 1.0	0.5 / 1.7	0.4 / 1.1	0.4 / 1.1	0.4 / 1.2	0.4 / 1.0

Table VI:

R^{s}

results on GoPro-FC at 30/10 FPS. FF: FlowFormer, FF++: FlowFormer++.

Corruption	PSNR	RAFT-OOD
Corruption	PSNR	EPE	$R^{s}$	$R^{d}$
High-Light	13.97	5.2	2.01	0.90
Low-Light	14.53	7.25	4.16	2.95
Over Exposure	13.32	6.74	3.30	2.45
Under Exposure	12.54	4.34	0.43	0.05

Table VII: PSNR of four illumination corruptions. EPE,

R^{s}

, and

R^{d}

for RAFT-OOD are also reported.

Figure 6: JPEG, Saturate, Spatter, and Fog samples from KITTI-FC. JPEG removes local high-frequency cues, while Saturate modifies color distribution. Spatter introduces occlusions, and Fog lowers contrast while retaining local structure.

Corruption	PSNR	RAFT-OOD
Corruption	PSNR	EPE	$R^{s}$	$R^{d}$
JPEG	25.76	10.28	7.57	5.99
Saturate	19.29	5.73	2.41	1.44
Spatter	24.78	21.49	18.59	17.2
Fog	11.69	6.34	3.14	2.05

Table VIII: PSNR of four representative corruptions. JPEG and Spatter have higher PSNR but cause larger optical-flow errors because they remove local information or introduce occlusion. EPE,

R^{s}

, and

R^{d}

for RAFT-OOD are also reported.

Setting	Method	UE					Pixelate
Setting	Method	1	2	3	4	5	1	2	3	4	5
OOD	RAFT	-0.002	0.010	0.095	0.140	0.011	0.104	0.055	0.262	0.411	0.363
OOD	GMA	-0.044	-0.001	0.010	0.087	-0.033	-0.061	0.009	0.175	1.263	1.147
ID	RAFT	0.059	0.520	1.092	2.569	4.173	-0.028	-0.026	0.168	0.447	0.628
ID	GMA	-0.138	-0.207	-0.054	-0.295	-0.041	-0.190	0.187	0.723	1.482	1.846

Table IX:

R^{d}

across severity levels for Under Exposure and Pixelate.

Figure 7: Clean EPE, reference-normalized $R^{s}$ , and $R^{s}_{\mathrm{rel}}$ on KITTI-FC. Bubble size represents $R^{s}_{\mathrm{rel}}$ , with the corresponding value shown below each model name.

Model	Digital	Illum.	Weather	Noise	Blur	Avg.
SKFlow	0.8298	0.7801	0.9034	1.1370	0.9657	0.9146
GMFlowNet	0.9232	1.0818	0.9237	0.9129	0.8603	0.9378

Table X: OOD stability of spatial modeling designs on KITTI-FC. Category-average

R^{s}

is normalized corruption-wise by RAFT-OOD (lower is better).

Figure 8: Reference-normalized $R^{s}$ rank across benchmark settings. Among the matched methods shown, none maintains consistent superiority across KITTI-FC ID/OOD and GoPro-FC 10/30 FPS.

	Over Exposure		Snow
Level	One frame	Both frames	Independent	Shared
1	0.60/0.09	0.66/0.09	5.16/3.68	21.46/19.90
2	1.59/0.77	1.63/0.74	11.77/9.63	26.48/24.77
3	2.73/1.92	2.04/1.04	11.30/8.80	26.71/24.96
4	4.66/3.69	2.88/1.62	18.33/15.24	29.32/27.57
5	6.90/5.73	3.40/1.80	13.35/10.62	26.20/24.45

Table XI: Paired-frame controls on KITTI-FC. Asymmetric exposure is more harmful at high severity, while shared snow is more harmful across all levels.

Corruption	$R^{s}$ (10/30 FPS)	$Q$ (10/30 FPS)
JPEG	3.113/1.302	0.0952/0.1441
Snow	5.479/3.664	0.1593/0.2643

Table XII: GoPro-FC motion-scale analysis. Larger absolute prediction changes at 10 FPS coexist with smaller motion-normalized changes.

Method	LR	Step	Batch	Crop ( $H\!\times\!W$ )
RAFT, CSFlow, GMFlowNet, FlowDiffuser	$10^{-4}$	50k	6	$288\!\times\!960$
GMA, CRAFT	$1.25\!\times\!10^{-4}$	30k	6	$288\!\times\!960$
SKFlow	$1.75\!\times\!10^{-4}$	30k	6	$288\!\times\!960$
FlowFormer, FlowFormer++	$1.25\!\times\!10^{-4}$	50k	6	$432\!\times\!960$
SAMFlow-T/B/H	$1.25\!\times\!10^{-4}$	50k	6	$400\!\times\!720$
DPFlow	$10^{-4}$	50k	3	$288\!\times\!960$

Table XIII: Supervised ID fine-tuning settings. All methods use clean KITTI-FC training pairs with standard augmentations.

	JPEG / Saturate	Spatter / Saturate
Eligible pairs	33	14
Mean PSNR (dB)	26.25 / 25.95	18.27 / 18.06
Mean SSIM	0.945 / 0.960	0.756 / 0.759
Mean $R^{s}$ (px)	8.00 / 1.37	21.73 / 2.42
Mean $R^{d}$ (px)	5.71 / 0.52	19.46 / 1.33
$\Delta R^{s}$ (px)	+6.63	+19.31

Table XIV: Quality-controlled comparison using RAFT-OOD. JPEG and Spatter produce larger stability errors and accuracy degradation than Saturate at comparable image quality.

	KITTI-FC		RobustSpring
Corruption	12 iter.	32 iter.	Published
JPEG, level 5	10.639	10.299	3.190
Spatter, level 4	17.197	17.019	7.980
Glass Blur, level 1	1.078	1.057	1.050

Table XV: Official RAFT-Sintel checkpoint comparison (prediction shift in pixels). Larger shifts persist for JPEG and Spatter, while Glass Blur becomes comparable.

Optical Flow

Optical flow estimates the apparent motion of pixels between images, supporting video understanding, tracking, robotics, and image synthesis.

See all 58 papers

DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models
Mar 24, 2026Jaewon Min, Jaeeun Lee, Yeji Choi +7Optical Flow Image Corruption Robustness
KAIST AI · Hanwha Systems
On the Real-World Generalisability of Optical Flow Models
Jul 11, 2026Petter Reijalt, Sander Gielisse, Rickard Karlsson +1Optical Flow
Computer Vision Lab, TU Delft, Delft, the Netherlands
FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion
Jul 11, 2026Yuang Meng, Chenyang Wu, Xianshun Liu +7Optical Flow Diffusion Model Guidance
VCIP, CS, Nankai University · OPPO Research Institute · The Hong Kong Polytechnic University

Mar 24, 2026J/K move · Enter open · S save

DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models

Jaewon Min, Jaeeun Lee, Yeji Choi, Paul Hyunbin Cho+6

KAIST AI · Hanwha Systems

Optical Flow Image Corruption Robustness

Open paper AskPDF arXiv

Optical flow models trained on high-quality data often degrade severely when confronted with real-world corruptions such as blur, noise, and compression artifacts. To overcome this limitation, we formulate Degradation-Aware Optical Flow, a new task targeting accurate dense correspondence estimation from real-world corrupted videos. Our key insight is that the intermediate representations of image restoration diffusion models are inherently corruption-aware but lack temporal awareness. To address this limitation, we lift the model to attend across adjacent frames via full spatio-temporal attention, and empirically demonstrate that the resulting features exhibit zero-shot correspondence capabilities. Based on this finding, we present DA-Flow, a hybrid architecture that fuses these diffusion features with convolutional features within an iterative refinement framework. DA-Flow substantially outperforms existing optical flow methods under severe degradation across multiple benchmarks.

Abstract

Optical Flow

Explore similar work

DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models

On the Real-World Generalisability of Optical Flow Models

FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion