An Extended Topological Model For High-Contrast Optical Flow

Brad Turow; Jose A. Perea

cs.CVMar 6, 2026

An Extended Topological Model For High-Contrast Optical Flow

Authors: Brad Turow, Jose A. Perea

Abstract

In this paper, we identify low-dimensional models for dense core subsets in the space of $3\times 3$ high-contrast optical flow patches sampled from the Sintel dataset. In particular, we leverage the theory of approximate and discrete circle bundles to identify a 3-manifold whose boundary is a previously proposed optical flow torus, together with disjoint circles corresponding to pairs of binary step-edge range image patches. The 3-manifold model we introduce provides an explanation for why the previously-proposed torus model could not be verified with direct methods (e.g., a straightforward persistent homology computation). We also demonstrate that nearly all optical flow patches in the top 1 percent by contrast norm are found near the family of binary step-edge circles described above, rather than the optical flow torus, and that these frequently occurring patches are concentrated near motion boundaries (which are of particular importance for computer vision tasks such as object segmentation and tracking). Our findings offer insights on the subtle interplay between topology and geometry in inference for visual data.

Figures & tables

Panel of Figure 1 — Figure 1: Frames from the Sintel video labeled with sample (scaled) optical flow vectors

Figure 2: The linear step-edge annulus model for high-contrast $3\times 3$ optical image patches from [ 16 ] . Some binary step-edge patches are shown on the boundary circles. The central circle is the primary circle described in [ 9 ]

Figure 3: The mean-centered contrast-normalized DCT basis for $3\times 3$ image patches

Figure 5: The proposed optical flow torus model. Each column captures patches with the same flow direction. The horizontal flow circle is shown in the first and last column; note however that the two columns differ by a shift by $\pi$ . This reflects the fact that $(\theta,\alpha)$ is not a global coordinate system for $\mathcal{T}$ . Compare with Figure 7 in [ 2 ] .

Figure 6: Persistence diagrams for

X(1500,50)

X(1500,30)

and several samples from the idealized torus model

\mathcal{T}

with various levels of Gaussian noise

Figure 7: A sample of high-contrast optical flow patches labeled by predominant flow direction

Figure 8: Plane projections of the sets $\{X_{\frac{k\pi}{6}}(1500,50)\}_{k=1}^{6}$

Panel of Figure 9 — Figure 9: A sample of high-contrast optical flow patches from $X(1500,50)$ for which the predominant direction is essentially arbitrary

Figure 10:

Figure 11: $F(\alpha,\theta)$ (top) and $F^{\perp}(\alpha,\theta)$ (bottom) for various $(\alpha,\theta)$ pairs.

Panel of Figure 12 — Figure 12: Model flow patches of the form $\widetilde{F}(r,\alpha_{0},\theta_{0})$ for decreasing values of $r$ . Note that $\widetilde{F}(1,\alpha_{0},\theta_{0})=F(\alpha_{0},\theta_{0})$ is a flow torus patch, and the limiting patch as $r\to 0$ depends only on the sum of $\alpha_{0}$ and $\theta_{0}$ .

Figure 13: The limiting circle of patches with low directionality in the proposed extended model

Panel of Figure 14 — Figure 14: (Left) Correlations on intersections $X_{j}\cap X_{k}$ between the discrete approximate local trivializations $\{f_{j}:X_{j}\to\mathbb{S}^{1}\}_{j=1}^{16}$ computed for the data. (Right) The nerve of the open cover $\mathcal{U}=\{U_{j}\}_{j=1}^{16}$ of $\mathbb{RP}^{1}$ . Each edge $(jk)\in\mathcal{N(U)}$ is labeled by the average angular alignment error $\frac{1}{\#(X_{j}\cap X_{k})}\sum\limits_{x\in X_{j}\cap X_{k}}|f_{j}(x)-\Omega_{jk}f_{k}(x)|^{2}$ of the local trivializations on the corresponding intersection. The components $\omega_{jk}$ of the Stiefel-Whitney class representative $\omega=\text{det}_{*}(\Omega)$ are shown in blue, and a potential $\tau=\{\tau_{j}\}_{j=1}^{N}$ for $\omega$ is shown in red.

Figure 15: A sample of coordinatized flow patches from $X(1500,50)$ with high directionality, arranged according to predominant direction (x-axis) and assigned fiber coordinate (y-axis). Observe that the assigned fiber coordinate is strongly correlated with $\theta-\alpha$ in Figure 5 .

Figure 16: A sample of coordinatized flow patches from $X(1500,50)$ with directionality $r<0.7$ , arranged according to predominant direction (x-axis) and assigned fiber coordinate (y-axis). Observe that each column is roughly a copy of the low-directionality circle shown in Figure 13 .

Panel of Figure 17 — Figure 17: (a) PCA projections of several fibers of $X(50,60)$ and (b) Persistence diagrams of the fibers.

Figure 18: Results of the clustering experiment described above. Note: the cardinality of the largest component

G_{0}

G

was omitted from the rightmost plot for readability.

Panel of Figure 20 — Figure 20: Two ways to identify binary step-edge flow patches in the fiber at $\theta=\frac{\pi}{6}$ with the corresponding binary step-edge range patches

Figure 21: A visualization of a connected component of $G$ . Each node is labeled by a patch from the corresponding local cluster; edges representing intersections between local clusters are shown. Patches in the same column are represent local clusters over the same open set in the cover $\{U_{j}\}_{j=1}^{16}$ of $\mathbb{RP}^{1}$ .

Figure 22: Exemplary patches from three global connected components of $G$ , placed in $\mathbb{S}^{1}$ according the computed circular coordinate.

Figure 23:

Panel of Figure 24 — Figure 24: (a) A summary of the connected components of $G^{(0.07)}$ . (b) A visualization of the largest connected component of $G$ . Each node represents a local cluster, and each edge represents an intersection. The nodes are colored according to their connected components in the complete subgraph $G^{(0.07)}$ . Nodes corresponding to data from the (extended) optical flow torus are shown in black. Edges which do not appear in $G^{(0.07)}$ are shown in red.

Panel of Figure 25 — Figure 25: (a) The persistence diagrams for the unions of component pairs $\{2,38\}$ and $\{39,40\}$ of $G^{(0.07)}$ , respectively. Each union is disconnected until the components combine to form a single loop at a length scale of approximately $0.3$ . (b) Sample patches from each component pair arranged by assigned circular coordinate.

Figure 27: PCA projections of fibers of $\tilde{X}(30,80)$ (where $\tilde{X}$ denotes the top 5 percent of patches in our high-contrast sample $X$ . Hence $\tilde{X}$ represents the top 1 percent of all optical flow patches by contrast norm).

Panel of Figure 28 — Figure 28: Three frames from the Sintel video with dots labeling the original locations of high contrast patches from our sample. Dark purple dots indicate patches in the top 20 percent by contrast norm; light blue dots indicate patches in the top 1 percent.

Figure 30: A sample of binary step-edge patches from $X_{\text{step}}$ arranged according to lifted predominant direction ( $x$ -axis) and assigned fiber coordinate ( $y$ -axis).

Figure 31: A list of the 56 binary step-edge range image patches

Optical Flow

Optical flow estimates the apparent motion of pixels between images, supporting video understanding, tracking, robotics, and image synthesis.

See all 58 papers

On the Real-World Generalisability of Optical Flow Models
Jul 11, 2026Petter Reijalt, Sander Gielisse, Rickard Karlsson +1Optical Flow
Computer Vision Lab, TU Delft, Delft, the Netherlands
FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow
Mar 30, 2026Sadra Safadoust, Fabio Tosi, Matteo Poggi +1Optical Flow
Department of Computer Engineering and KUIS AI Center, Koç University, Istanbul, Turkey · Department of Computer Science and Engineering (DISI), University of Bologna, Italy
Triangular Consistency as a Universal Constraint for Learning Optical Flow
Jun 18, 2026Yi Xiao, Carlos Rodriguez Coronel, Jing Zhan +3Optical Flow Consistency Regularization
Louisiana State University, Baton Rouge, LA 70803, USA · University of California, Los Angeles, Los Angeles, CA 90025, USA · Yale University, New Haven, CT 06520, USA

Jul 11, 2026J/K move · Enter open · S save

On the Real-World Generalisability of Optical Flow Models

Petter Reijalt, Sander Gielisse, Rickard Karlsson, Jan van Gemert

Computer Vision Lab, TU Delft, Delft, the Netherlands

Optical Flow

Open paper AskPDF arXiv

Real-world deployment of vision models to broadly benefit society is arguably a main research objective. In optical flow, however, the difficulty to obtain the ground truth has focused research mainly on synthetic data and domain-specific benchmarks. Here, we investigate the severity of this mismatch. We study how well modern optical flow estimation models generalise to real-world video and question if accuracy on synthetic benchmark proxies actually predicts accuracy on real-world optical flow. To address this, we build a real-world evaluation benchmark and evaluate the real-world generalisability of a broad set of recent optical flow models using standard checkpoints. Our benchmark contains 8,204 frame pairs across TAP-Flow, Slow Flow, and our own dataset FlowFactor. FlowFactor is a manually annotated real-world benchmark of 1,000 HD frame pairs organised into four confounding factors: large displacements, repetitive textures, occlusions, and lighting variation. Each setting mainly varies only one factor, enabling diagnostic, confounder-specific analysis. Using FlowFactor, we reveal that performance on varying lighting and large displacements correlates most strongly with real-world accuracy, and that improvements on large-motion regimes can trade off against robustness in small-motion, stationary scenes. Our experiments show that progress on Sintel, KITTI and Spring only weakly predicts accuracy on real-world data, highlighting the need for a broad real-world optical flow benchmark. Interestingly, scaling up the amount of training data does not necessarily resolve the gap, calling for new innovative research instead of simply scaling data and compute.

Abstract

Optical Flow

Explore similar work

On the Real-World Generalisability of Optical Flow Models

FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow

Triangular Consistency as a Universal Constraint for Learning Optical Flow