Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning

Zhongxiao Cong; Qitao Zhao; Minsik Jeon; Shubham Tulsiani

cs.CVFeb 23, 2026

Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning

Authors: Zhongxiao Cong, Qitao Zhao, Minsik Jeon, Shubham Tulsiani

Organizations: Carnegie Mellon University

Abstract

Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision -- expensive to obtain at scale and particularly scarce for dynamic real-world scenes. We present Flow3r, a framework that augments visual geometry learning with dense 2D correspondences (`flow') as supervision, enabling scalable training from unlabeled monocular videos. Our key insight is that the flow prediction module should be factored: predicting flow between two images using geometry latents from one and pose latents from the other. This factorization directly guides the learning of both scene geometry and camera motion, and naturally extends to dynamic scenes. In controlled experiments, we show that factored flow prediction outperforms alternative designs and that performance scales consistently with unlabeled data. Integrating factored flow into existing visual geometry architectures and training with ${\sim}800$ K unlabeled videos, Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with its largest gains on in-the-wild dynamic videos where labeled data is most scarce.

Figures & tables

Figure 2: Mechanisms for flow prediction. (a) The visual geometry backbone first extracts the camera token and patch tokens of each input. (b) Existing correspondence heads [ 39 ] predict flow directly from local features via matching. (c) Flow can also be obtained by explicitly projecting predicted 3D points into another view via decoded camera parameters. However, this projection-based formulation is restricted to static scenes and sensitive to geometric errors. (d) Our factored flow approach conditions source-view geometry latents on the target-view camera latent and decodes dense correspondences directly in latent space, providing a geometry-aware and robust flow prediction mechanism that naturally extends to dynamic scenes.

Figure 3: Overview of Flow3r . Flow3r predicts visual geometry using factored flow supervision, enabling scalable geometry learning from unlabeled videos. Each input image is encoded and processed by the multi-view transformer to produce camera tokens and patch tokens. For data with dense geometry and pose labels, we directly supervise the patch tokens and camera tokens with the corresponding labels. For unlabeled datasets without geometry and pose supervision, we predict flow between two frames in a factorized manner, supervised by the pseudo labels from an off-the-shelf 2D flow prediction model [ 46 ] . To obtain the factored flow, we fuse the patch features of one frame with the camera features of another, and decode the fused representation through the DPT head to produce dense flow predictions.

Model Variant	Data (# Seqs)	RRA@30 $\uparrow$	RTA@30 $\uparrow$	CD $\downarrow$	MSE $\downarrow$	Data (# Seqs)	RRA@30 $\uparrow$	RTA@30 $\uparrow$	CD $\downarrow$	MSE $\downarrow$
	Static Scenes					Dynamic Scenes
3d-sup	ScanNet++ ( $1$ K)	0.7500	0.6929	0.030	0.088	OmniWorld ( $1$ K)	66.01	62.37	0.105	0.637
flow-projective	+ ScanNet++ ( $1$ K)	0.6700	0.4572	0.033	0.088	+ SpatialVID ( $3$ K)	61.23	56.12	0.158	0.710
flow-tracking	+ ScanNet++ ( $1$ K)	0.7438	0.7021	0.030	0.089	+ SpatialVID ( $3$ K)	68.56	62.95	0.107	0.628
flow-factored	+ ScanNet++ ( $1$ K)	0.7700	0.7366	0.026	0.078	+ SpatialVID ( $3$ K)	76.26	68.84	0.103	0.598

Table 1: Does factored flow prediction help visual geometry learning? We evaluate the camera and geometry prediction of a model trained with only 3D labeled data and variants that use additional unlabeled data via flow supervision. We find that on both static and dynamic scenes, our proposed factored flow prediction ( flow-factored ) yields consistent gains over the model variant without additional unlabeled data ( 3d-sup ) as well as other flow-supervised alternatives.

Figure 4: Factored flow prediction aids visual geometry learning. Compared with the baseline ( 3d-sup ) and alternative formulations that use flow supervision ( flow-projective , flow-tracking ), Flow3r ( flow-factored ) yields more accurate scene reconstruction, highlighting the benefits of unlabeled data and our factored flow formulation.

Figure 5: Scaling with unlabeled videos. With factored flow supervision, increasing the amount of unlabeled SpatialVID data progressively improves dynamic-scene reconstruction quality, even surpassing the model with additional 3D-labeled training data.

Figure 6: Qualitative results on in-the-wild data. We compare the reconstructions from Flow3r with representative feedforward reconstruction methods on in-the-wild multi-view images from static scenes, dynamic scenes, and interaction videos. We observe that Flow3r infers cleaner and more accurate scene structure than baselines that often yield misaligned outputs or infer inaccurate motion.

Methods	Kinetics700				EPIC-KITCHENS				Sintel				Bonn
Methods	RPE trans $\downarrow$	RPE rot $\downarrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	MSE $\downarrow$	f-score@5 $\uparrow$
DUSt3R	0.063	9.343	0.366	0.533	0.110	8.492	0.312	0.528	0.179	15.166	0.622	0.271	0.113	6.384	0.116	0.800
CUT3R	0.027	1.988	0.303	0.573	0.081	4.709	0.338	0.493	0.128	1.998	0.676	0.217	0.095	6.349	0.088	0.899
VGGT	0.038	1.392	0.347	0.479	0.049	3.025	0.220	0.617	0.086	1.220	0.595	0.311	0.096	6.021	0.082	0.884
$\pi^{3}$	0.023	1.006	0.267	0.585	0.043	3.025	0.200	0.620	0.066	1.122	0.523	0.317	0.095	6.240	0.076	0.905
Flow3r	0.018	0.830	0.256	0.599	0.037	2.729	0.199	0.622	0.058	0.920	0.426	0.404	0.094	6.262	0.052	0.954

Table 2: Comparison on dynamic datasets. We evaluate the camera (RPE rot and trans) and geometry prediction (MSE and f-score) for dynamic scene reconstruction across different datasets. Best, second and third results are highlighted in light red, orange, and yellow, respectively. Flow3r consistently outperforms state-of-the-art methods in both camera pose estimation and scene reconstruction, demonstrating the effectiveness of leveraging large-scale unlabeled videos for visual geometry learning via factored-flow supervision.

Methods	Co3Dv2				Scannet				NRGBD				7-scenes
Methods	RTA@30 $\uparrow$	AUC@30 $\uparrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	MSE $\downarrow$	f-score@5 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	MSE $\downarrow$	f-score@5 $\uparrow$
DUSt3R	90.68	68.11	0.223	0.783	57.14	31.56	0.092	0.831	93.25	76.04	0.063	0.865	76.59	55.07	0.170	0.714
CUT3R	90.68	68.11	0.278	0.737	71.39	42.30	0.132	0.740	95.63	76.90	0.075	0.808	81.15	59.72	0.183	0.695
VGGT	97.27	87.62	0.151	0.874	93.71	71.37	0.053	0.931	99.21	93.13	0.032	0.964	86.51	69.77	0.182	0.665
$\pi^{3}$	97.49	90.53	0.151	0.874	91.14	69.39	0.058	0.930	99.20	90.40	0.021	0.983	87.69	71.80	0.169	0.737
Flow3r	97.62	90.41	0.150	0.876	92.89	71.00	0.053	0.943	99.60	94.40	0.018	0.992	91.66	75.76	0.102	0.807

Table 3: Comparison on static datasets. We evaluate the camera (RTA and AUC) and geometry prediction (MSE and f-score) for static scene reconstruction from the multi-view input. Best, second and third results are highlighted in light red, orange, and yellow, respectively. Flow3r outperforms state-of-the-art methods, indicating that the gains from incorporating unlabeled dynamic videos for visual geometry learning also transfer to static scenes.

Data		Metrics
Labeled (# Seqs)	Unlabeled (# Seqs)	RRA@30 $\uparrow$	RTA@30 $\uparrow$	CD $\downarrow$	MSE $\downarrow$
OmniWorld ( $1$ K)	-	66.01	62.37	0.105	0.637
OmniWorld ( $1$ K)	SpatialVID ( $3$ K)	76.26	68.84	0.103	0.598
OmniWorld ( $1$ K)	SpatialVID ( $10$ K)	78.45	68.82	0.077	0.560
OmniWorld ( $1$ K)	SpatialVID ( $20$ K)	81.12	71.21	0.075	0.532
OmniWorld ( $4$ K)	-	78.68	70.26	0.080	0.565

Table 4: Scaling with unlabeled videos for dynamic visual geometry learning. We train model variants on OmniWorld [ 50 ] (labeled 3D supervision) together with increasing amounts of SpatialVID [ 38 ] (unlabeled videos with flow supervision). Performance improves consistently as we scale up the unlabeled data.

Base	3D Sup.	Flow Loss		EPIC-KITCHENS				7-scenes
Base	3D Sup.	Labeled	Unlabeled	trans $\downarrow$	rot $\downarrow$	MSE $\downarrow$	f-score $\uparrow$	RTA $\uparrow$	AUC $\uparrow$	MSE $\downarrow$	f-score $\uparrow$
VGGT	N/A (base model)			0.049	3.025	0.220	0.617	86.51	69.77	0.182	0.665
	✓	✗	✗	0.050	3.005	0.218	0.530	88.10	68.23	0.167	0.750
	✓	✓	✗	0.045	3.005	0.203	0.530	89.17	68.22	0.167	0.749
	✓	✓	✓	0.039	2.808	0.202	0.599	90.67	71.75	0.166	0.750
$\pi^{3}$	N/A (base model)			0.043	3.025	0.200	0.620	87.69	71.80	0.169	0.737

Table 5: Effect of adding unlabeled videos via flow supervision. For each backbone (VGGT [ 39 ] and

\pi^{3}

[ 42 ] ), we compare a 3D supervision only baseline, a labeled multi-task model (geometry + flow), and the full model incorporating additional unlabeled data via flow supervision. This demonstrates the effectiveness of unlabeled data via flow supervision for visual geometry learning.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model Variant	Data (# Seqs)	RRA $\uparrow$	RTA $\uparrow$	CD $\downarrow$	MSE $\downarrow$	AEPE $\downarrow$	EPE@5px $\uparrow$
flow-projective	+ SpatialVID ( $3$ K)	61.23	56.12	0.158	0.710	30.24	24.15
flow-tracking	+ SpatialVID ( $3$ K)	68.56	62.95	0.107	0.628	18.82	43.26
flow-factored	+ SpatialVID ( $3$ K)	76.26	68.84	0.103	0.598	23.29	36.42

Appendix

Table 6: Pose, geometry, and flow evaluation.

Data		Metrics
Labeled (# Seqs)	Unlabeled (# Seqs)	RRA $\uparrow$	RTA $\uparrow$	CD $\downarrow$	MSE $\downarrow$
OmniWorld ( $1$ K)	-	66.01	62.37	0.105	0.637
OmniWorld ( $1$ K)	SpatialVID ( $3$ K)	76.26	68.84	0.103	0.598
OmniWorld ( $1$ K)+SpatialVID ( $3$ K)	-	75.99	65.30	0.108	0.625
OmniWorld ( $4$ K)	-	78.68	70.26	0.080	0.565

Appendix

Table 7: Ablating pseudo-labeled 3D as supervision.

Methods	Kinetics700
Methods	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	0.045	0.063	9.343	0.083	0.106	0.095	0.366	0.317	0.533
CUT3R	0.019	0.027	1.988	0.070	0.076	0.073	0.303	0.352	0.573
VGGT	0.027	0.038	1.392	0.088	0.120	0.104	0.347	0.258	0.479
$\pi^{3}$	0.016	0.023	1.006	0.059	0.097	0.078	0.267	0.347	0.585
Flow3r	0.013	0.018	0.830	0.062	0.092	0.077	0.256	0.371	0.599

Appendix

Table 8: Comparison on Kinetics700.

Methods	EPIC-KITCHENS
Methods	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	0.077	0.110	8.492	0.100	0.092	0.096	0.312	0.385	0.528
CUT3R	0.056	0.081	4.709	0.085	0.095	0.090	0.338	0.297	0.493
VGGT	0.032	0.049	3.025	0.061	0.069	0.065	0.220	0.415	0.617
$\pi^{3}$	0.032	0.043	3.025	0.069	0.058	0.059	0.200	0.459	0.620
Flow3r	0.032	0.037	2.729	0.066	0.056	0.059	0.199	0.339	0.622

Appendix

Table 9: Comparison on EPIC-KITCHENS.

Methods	Sintel
Methods	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	0.152	0.179	15.166	0.255	0.224	0.240	0.622	0.124	0.271
CUT3R	0.111	0.128	1.998	0.221	0.269	0.245	0.676	0.111	0.217
VGGT	0.090	0.086	1.220	0.217	0.227	0.222	0.595	0.105	0.311
$\pi^{3}$	0.060	0.066	1.122	0.189	0.191	0.190	0.523	0.141	0.317
Flow3r	0.048	0.058	0.920	0.153	0.104	0.129	0.426	0.201	0.404

Appendix

Table 10: Comparison on Sintel.

Methods	Bonn
Methods	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	0.055	0.113	6.384	0.029	0.037	0.033	0.116	0.546	0.800
CUT3R	0.021	0.095	6.349	0.018	0.023	0.021	0.088	0.658	0.899
VGGT	0.011	0.096	6.021	0.016	0.028	0.021	0.082	0.644	0.884
$\pi^{3}$	0.016	0.095	6.240	0.014	0.025	0.019	0.076	0.669	0.905
Flow3r	0.009	0.094	6.262	0.009	0.018	0.013	0.052	0.801	0.954

Appendix

Table 11: Comparison on Bonn.

Methods	7-scenes
Methods	RRA@30 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	100.0	76.59	55.07	0.047	0.053	0.050	0.170	0.457	0.714
CUT3R	100.0	81.15	59.72	0.055	0.049	0.052	0.183	0.457	0.695
VGGT	100.0	86.51	69.77	0.052	0.056	0.054	0.182	0.395	0.665
$\pi^{3}$	100.0	87.69	71.80	0.039	0.045	0.042	0.169	0.473	0.737
Flow3r	100.0	91.66	75.76	0.025	0.016	0.020	0.102	0.729	0.807

Appendix

Table 12: Comparison on 7-scenes.

Methods	NRGBD
Methods	RRA@30 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	100.0	93.25	76.04	0.031	0.024	0.027	0.063	0.662	0.865
CUT3R	100.0	95.63	76.90	0.042	0.019	0.030	0.075	0.549	0.808
VGGT	100.0	99.21	93.13	0.015	0.010	0.012	0.032	0.833	0.964
$\pi^{3}$	100.0	99.20	90.40	0.010	0.007	0.008	0.021	0.910	0.983
Flow3r	100.0	99.60	94.40	0.008	0.005	0.007	0.018	0.938	0.992

Appendix

Table 13: Comparison on NRGBD.

Methods	Scannet
Methods	RRA@30 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	100.0	57.14	31.56	0.031	0.032	0.032	0.092	0.591	0.831
CUT3R	99.39	71.39	42.30	0.053	0.034	0.043	0.132	0.499	0.740
VGGT	100.0	93.71	71.37	0.015	0.019	0.017	0.053	0.769	0.931
$\pi^{3}$	99.75	91.14	69.39	0.016	0.022	0.019	0.058	0.762	0.930
Flow3r	100.0	92.89	71.00	0.015	0.018	0.016	0.053	0.780	0.943

Appendix

Table 14: Comparison on Scannet.

Methods	Co3Dv2
Methods	RRA@30 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$	Acc. $\downarrow$	Comp. $\downarrow$	CD $\downarrow$	MSE $\downarrow$	f-score@2% $\uparrow$	f-score@5% $\uparrow$
DUSt3R	97.07	90.91	66.83	0.036	0.069	0.033	0.223	0.567	0.783
CUT3R	93.85	90.68	68.11	0.047	0.082	0.064	0.278	0.507	0.737
VGGT	98.41	97.27	87.62	0.022	0.051	0.036	0.151	0.707	0.874
$\pi^{3}$	98.82	97.49	90.53	0.022	0.052	0.037	0.151	0.707	0.874
Flow3r	98.84	97.62	90.41	0.020	0.052	0.037	0.150	0.709	0.876

Appendix

Table 15: Comparison on Co3Dv2.

Figure 7: More qualitative results on dynamic scenes. Flow3r effectively handles dynamic components, yielding cleaner geometry than prior feed-forward methods.

Figure 8: More qualitative results on static scenes and interaction videos. The first six examples show static scenes and the last two examples feature interaction videos. Flow3r tends to produce more stable geometry under motion, though some challenging cases still show noticeable artifacts.

Explore similar work

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
Oct 12, 2026Junhyeong Go, Mingyu Jung, Woobin Im +1Vision-Language-Action Models Robotic Manipulation
Ajou University · Samsung Electronics
PURE: Refining Feed-forward 3D Reconstruction with Unlabeled Post-training
Oct 10, 2026Yaolong Zhu, Peng Li, Jiahao Lu +83D Reconstruction Feed-Forward 3D Reconstruction
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · The Hong Kong University of Science and Technology · Pengcheng Laboratory
SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data
Apr 10, 2026Qingwen Zhang, Xiaomeng Zhu, Chenhan Jiang +1Synthetic Data Pretraining Synthetic Data Generation
KTH Royal Institute of Technology, Stockholm, Sweden · Hong Kong University of Science and Technology, Hong Kong, China

Oct 12, 2026J/K move · Enter open · S save

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Junhyeong Go, Mingyu Jung, Woobin Im, Jongbin Ryu

Ajou University · Samsung Electronics

Vision-Language-Action Models Robotic Manipulation Motion Representation Learning 3D Scene Understanding

Open paper AskPDF arXiv

Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.

Abstract

Explore similar work

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

PURE: Refining Feed-forward 3D Reconstruction with Unlabeled Post-training

SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data