VisFlow: Interpretable Flow Matching for Visibility Reconstruction, Characterization, and Recognition

Rahul Moorthy; Volkan Isler

cs.CGOct 22, 2026

VisFlow: Interpretable Flow Matching for Visibility Reconstruction, Characterization, and Recognition

Authors: Rahul Moorthy, Volkan Isler

Abstract

Methods for learning continuous representations, such as latent manifolds, are fundamental and ubiquitous in modern machine learning. Yet many geometric objects combine continuous shape information with discrete combinatorial structure. This combination presents a fundamental challenge for representation learning because small geometric perturbations can induce abrupt changes in the underlying discrete representation. As a result, the smoothness assumptions that make continuous representations effective no longer hold. Polygons provide a canonical example. Their vertex locations vary continuously, but infinitesimal changes in geometry can dramatically alter the associated visibility graph. Existing methods address the inverse mapping from visibility graphs to polygon geometry using vertex-, triangulation-, or signed-distance-function (SDF)-based representations. Vertex- and triangulation-based approaches construct polygons through explicit geometric operations. However, they are less effective at modeling the relationship between geometry and visibility. SDF-based approaches capture this relationship more naturally. Their intermediate states, however, do not correspond to explicit stages of polygon construction. We present VisFlow, a novel, interpretable flow formulation that combines continuous SDF representations with explicit polygon evolution. We evaluate VisFlow on Visibility Reconstruction and Visibility Characterization. Visibility Reconstruction generates one polygon consistent with a visibility graph (G), while Visibility Characterization characterizes all such polygons. VisFlow maintains 20% more visibility relations on Visibility Reconstruction and improves coverage by 30% on Visibility Characterization compared to state-of-the-art methods.

Figures & tables

Figure 1: Rectified Flow vs polygon evolution. Top: SDF rectified flow starts from Gaussian noise and produces intermediate grid states that are not explicitly tied to stages of polygon formation. Bottom: Our formulation starts from a triangle and evolves through intermediate polygon states toward the final polygon.

Figure 2: Visibility graph problems. Reconstruction finds a polygon consistent with a given visibility graph, characterization finds all polygons corresponding to the same visibility graph (The red circle shows the perturbations but still the same visibility graph), and recognition determines whether an arbitrary graph is a valid visibility graph.

Figure 3: Overview of VisFlow. Left: The Flow Network evolves an initial triangle latent into a target polygon latent using a time-conditioned U-Net conditioned on the target visibility graph. Right: The SDF Latent Generator encodes a continuous SDF into spatially localized latent features and decodes polygon vertices conditioned on the visibility graph. The Visibility Prediction Network provides differentiable visibility supervision. Inference: Given a target visibility graph $G$ , we sample and encode an initial triangle, evolve its latent using the learned flow, decode the final latent into a polygon, and refine its vertices using the Visibility Prediction Network to better match $G$ .

Panel of Figure 4 — Figure 4: Coverage: (a) A learned visibility-graph reconstruction network generates polygon $P_{i}$ from source state $S_{i}$ . Child states are formed as $S_{\mathrm{child}}=S_{\mathrm{parent}}+\epsilon$ , where $\epsilon\sim\mathcal{N}(0,\sigma^{2}I)$ . (b) A candidate polygon $P_{j}$ is accepted if $F_{1}(P_{j})>T$ and $D(P_{j},P_{i})>T_{d}$ for every $i<j$ . Here, $F_{1}(P_{j})$ is the F1 score between the visibility graph of $P_{j}$ and the target visibility graph, and $D(P_{j},P_{i})$ is the Chamfer distance between the two polygons. Coverage is the ratio of expanded nodes to total nodes, $N_{\mathrm{expanded}}/N_{\mathrm{total}}$ .

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) C-VAE ( Harvey et al., 2022 )	0.84 $\pm$ 0.04	0.78 $\pm$ 0.04	0.69 $\pm$ 0.03	0.72 $\pm$ 0.03
(b) GNN ( Cui and Wei, 2023 )	0.86 $\pm$ 0.06	0.86 $\pm$ 0.06	0.69 $\pm$ 0.05	0.73 $\pm$ 0.06
(c) Mesh ( Lionar et al., 2025 )	0.84 $\pm$ 0.06	0.82 $\pm$ 0.11	0.69 $\pm$ 0.05	0.71 $\pm$ 0.05
(d) VisDiff ( Mahesh et al., 2026 )	0.86 $\pm$ 0.04	0.80 $\pm$ 0.04	0.74 $\pm$ 0.03	0.76 $\pm$ 0.03
(e) VisFlow + Optimization (Ours)	0.94 $\pm$ 0.03	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05

Table 1: Baseline comparison: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

F1 Threshold $T$	Distance Threshold $T_{d}$	VisDiff ( Mahesh et al., 2026 )	VisFlow + Optimization
0.70	0.01	0.89 $\pm$ 0.28	1.00 $\pm$ 0.00
0.70	0.05	0.68 $\pm$ 0.29	1.00 $\pm$ 0.01
0.80	0.01	0.03 $\pm$ 0.16	0.95 $\pm$ 0.17
0.80	0.05	0.00 $\pm$ 0.00	0.95 $\pm$ 0.19

Table 2: Coverage performance (

\uparrow

) under varying F1 threshold

T

and distance threshold

T_{d}

: each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.

Figure 5: Recognition Performance: Accuracy vs F1 Threshold

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) C-VAE ( Harvey et al., 2022 )	0.79 $\pm$ 0.05	0.05 $\pm$ 0.02	0.32 $\pm$ 0.08	0.08 $\pm$ 0.03
(b) GNN ( Cui and Wei, 2023 )	0.49 $\pm$ 0.13	0.06 $\pm$ 0.03	0.96 $\pm$ 0.03	0.11 $\pm$ 0.05
(c) VisFlow(Ours)	0.99 $\pm$ 0.00	0.83 $\pm$ 0.08	0.89 $\pm$ 0.05	0.86 $\pm$ 0.06

Table 3: Contact-Graph Reconstruction Performance Fragfold 150 Dataset: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

Panel of Figure 6 — Figure 6: Contact-Graph Reconstruction. Ground-truth protein structures are shown in (a) and (c), with their corresponding VisFlow reconstructions shown in (b) and (d), respectively. The labels report the contact-graph similarity between each ground-truth structure and its reconstruction.

Figure 7: Grassfire trajectory for polygon formation. Ground-truth trajectories are constructed by applying grassfire flow to collapse the polygon along local SDF-normal directions until only a triangle remains. The arrows indicate the direction of vertex motion. The yellow vertices mark an edge-collapse event, in which two neighboring vertices meet as the edge between them shrinks to zero length, after which the polygon continues with one fewer vertex. For training, we reverse the trajectory so that polygon formation proceeds from the triangle to the complete polygon. The intermediate grassfire states supervise the corresponding points along the latent trajectory.

Panel of Figure 8 — Figure 8: Visibility predicates. (a) Two polygon vertices can be visible only if their connecting segment lies within the in-cone region at both endpoints. At each endpoint, the two incident polygon edges define an angular region, and the portion of this region lying inside the polygon forms the local in-cone region, shown in blue. Green arrows indicate that the segment from the other vertex enters through the in-cone region, while red arrows indicate that it enters from outside. Thus, $\overline{BH}$ fails at both endpoints, $\overline{HE}$ satisfies the condition only at $H$ , and $\overline{CE}$ satisfies it at both endpoints. (b) The intersection predicate determines whether a candidate segment intersects a non-incident polygon edge.

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) Visibility Verifier Network	1.00 $\pm$ 0.00	1.00 $\pm$ 0.00	1.00 $\pm$ 0.00	1.00 $\pm$ 0.00
(b) Transformer Architecture	0.91 $\pm$ 0.02	0.89 $\pm$ 0.03	0.76 $\pm$ 0.05	0.82 $\pm$ 0.04

Table 4: Vertex Prediction comparison: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

Figure 9: Architecture of the Visibility Prediction Network. Polygon vertices are first mapped to positional embeddings by a shared vertex encoder. For a candidate segment $(v_{i},v_{j})$ , the In-Cone Branch combines the candidate-segment representation with the neighboring polygon edges at each endpoint and predicts an in-cone probability for both endpoints. The two endpoint probabilities are multiplied to obtain $P_{\mathrm{InCone}}$ . The Intersection Branch compares the candidate segment with each non-incident polygon boundary edge and predicts pairwise intersection probabilities, which are aggregated using a max operation to obtain $P_{\mathrm{Intersect}}$ . The branch outputs are combined as $P_{\mathrm{Vis}}=P_{\mathrm{InCone}}(1-P_{\mathrm{Intersect}})$ to obtain the final visibility probability for $(v_{i},v_{j})$ .

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) C-VAE ( Harvey et al., 2022 )	0.74	0.72	0.70	0.70
(b) GNN ( Cui and Wei, 2023 )	0.73	0.79	0.69	0.67
(c) Mesh ( Lionar et al., 2025 )	0.75	0.74	0.72	0.71
(d) VisDiff ( Mahesh et al., 2026 )	0.92	0.91	0.91	0.91
(e) VisFlow + Optimization (Ours)	0.97	0.98	0.96	0.97

Table 5: Baseline comparison VisDiff Dataset: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

Panel of Figure 10 — Figure 10: Visibility Reconstruction Qualitative Results: (a) GT, (b)VisFlow + Optimization, (c) VisDiff, (d) GNN, (e) Mesh, (f) C-VAE. The labels below show the F-1 Score comparison with the GT Visibility Graph.

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) MSE Loss	0.87 $\pm$ 0.05	0.81 $\pm$ 0.04	0.75 $\pm$ 0.03	0.77 $\pm$ 0.03
(b) MSE + Visibility Loss	0.89 $\pm$ 0.02	0.84 $\pm$ 0.04	0.82 $\pm$ 0.04	0.83 $\pm$ 0.04

Table 6: Loss Ablation Study: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) VisFlow	0.89 $\pm$ 0.02	0.84 $\pm$ 0.04	0.82 $\pm$ 0.04	0.83 $\pm$ 0.04
(b) VisFlow + Optimization	0.94 $\pm$ 0.03	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05
(c) Optimization	0.88 $\pm$ 0.04	0.81 $\pm$ 0.07	0.88 $\pm$ 0.04	0.83 $\pm$ 0.06

Table 7: Approach Ablation Study Reconstruction Problem: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

F1 Threshold $T$	Distance Threshold $T_{d}$	VisFlow + Optimization	VisFlow
0.70	0.01	1.00 $\pm$ 0.00	1.00 $\pm$ 0.00
0.70	0.05	1.00 $\pm$ 0.01	0.76 $\pm$ 0.21
0.80	0.01	0.95 $\pm$ 0.17	0.51 $\pm$ 0.47
0.80	0.05	0.95 $\pm$ 0.19	0.36 $\pm$ 0.35

Table 8: Approach Ablation Study Characterization Problem: each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) Cont-VisDiff	0.88 $\pm$ 0.02	0.82 $\pm$ 0.03	0.83 $\pm$ 0.03	0.82 $\pm$ 0.03
(b) VisFlow	0.89 $\pm$ 0.02	0.84 $\pm$ 0.04	0.82 $\pm$ 0.04	0.83 $\pm$ 0.04
(c) Cont-VisDiff + Optimization	0.94 $\pm$ 0.02	0.91 $\pm$ 0.03	0.91 $\pm$ 0.03	0.91 $\pm$ 0.03
(d) VisFlow + Optimization	0.94 $\pm$ 0.03	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05	0.91 $\pm$ 0.05

Table 9: Approach Ablation Study: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

F1 Threshold $T$	Distance Threshold $T_{d}$	Cont-VisDiff	VisFlow
0.70	0.01	0.05 $\pm$ 0.10	1.00 $\pm$ 0.00
0.70	0.05	0.01 $\pm$ 0.00	0.76 $\pm$ 0.21
0.80	0.01	0.05 $\pm$ 0.10	0.51 $\pm$ 0.47
0.80	0.05	0.01 $\pm$ 0.00	0.36 $\pm$ 0.35

Table 10: Coverage performance (

\uparrow

) under varying F1 threshold

T

and distance threshold

T_{d}

: each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.

	Acc $\uparrow$	Prec $\uparrow$	Rec $\uparrow$	F1 $\uparrow$
(a) VisDiff ( Mahesh et al., 2026 )	0.84 $\pm$ 0.05	0.78 $\pm$ 0.04	0.72 $\pm$ 0.03	0.74 $\pm$ 0.03
(b) Cont-VisDiff	0.88 $\pm$ 0.02	0.81 $\pm$ 0.04	0.82 $\pm$ 0.03	0.82 $\pm$ 0.03
(c) VisFlow	0.89 $\pm$ 0.02	0.82 $\pm$ 0.04	0.83 $\pm$ 0.04	0.83 $\pm$ 0.04

Table 11: Vertex Prediction comparison: Acc : Accuracy, Prec : Precision, Rec : Recall, F1 : F1-Score.

Panel of Figure 11 — Figure 11: Dataset Statistics: Distribution of (a) polygon link diameter and (b) visibility-graph edge density across polygons in the VisFlow after resampling based on link diameter.

Baselines	Link Diameter (Avg / Std)	Min	Max
MNIST ( LeCun, 1998 )	1.83 / 0.50	1.0	4.1
COCO 2017 ( Lin et al., 2014 )	1.32 / 0.25	1.0	3.823
VisDiff ( Mahesh et al., 2026 )	4.4 / 2.2	1.0	9.0
VisFlow	6.5 / 2.32	1.0	11.0

Table 12: Real-world dataset comparison showing the diversity (in link diameter) between our dataset and standard benchmarks.

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 663 papers

Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning
Feb 23, 2026Zhongxiao Cong, Qitao Zhao, Minsik Jeon +13D Reconstruction Feed-Forward 3D Reconstruction
Carnegie Mellon University
FUSE: A Flow-based Mapping Between Shapes
Nov 17, 2025Lorenzo Olearo, Giulio Viganò, Daniele Baieri +2Flow Matching Non-Rigid Shape Correspondence
University of Milano-Bicocca, Milan, Italy · University of Bonn, Bonn, Germany · Lamarr Institute, Germany +1
Latent attention on masked patches for flow reconstruction
Mar 2, 2026Ben Eze, Luca Magri, Andrea NóvoaComputational Fluid Dynamics Vision Transformer
Aeronautics Dept., Imperial College London, Exhibition Rd, London SW7 2BX, UK · DIMEAS, Politecnico di Torino, Corso Duca degli Abruzzi, 24, Torino, 10129, Italy · I-X, Imperial College London, 84 Wood Lane, London W12 0BZ, UK

Feb 23, 2026J/K move · Enter open · S save

Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning

Zhongxiao Cong, Qitao Zhao, Minsik Jeon, Shubham Tulsiani

Carnegie Mellon University

3D Reconstruction Feed-Forward 3D Reconstruction Dynamic Scene Reconstruction Optical Flow

Open paper AskPDF arXiv

Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision -- expensive to obtain at scale and particularly scarce for dynamic real-world scenes. We present Flow3r, a framework that augments visual geometry learning with dense 2D correspondences (`flow') as supervision, enabling scalable training from unlabeled monocular videos. Our key insight is that the flow prediction module should be factored: predicting flow between two images using geometry latents from one and pose latents from the other. This factorization directly guides the learning of both scene geometry and camera motion, and naturally extends to dynamic scenes. In controlled experiments, we show that factored flow prediction outperforms alternative designs and that performance scales consistently with unlabeled data. Integrating factored flow into existing visual geometry architectures and training with

{\sim}800

K unlabeled videos, Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with its largest gains on in-the-wild dynamic videos where labeled data is most scarce.

Abstract

Flow Matching

Explore similar work

Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning

FUSE: A Flow-based Mapping Between Shapes

Latent attention on masked patches for flow reconstruction