From Feed-Forward to Flow: Unifying Reconstruction and Generation Is Easier Than You Think

Haoru Wang; Qianfan Shen; Kai Ye; Wenzheng Chen; Baoquan Chen

cs.CVSep 26, 2026

From Feed-Forward to Flow: Unifying Reconstruction and Generation Is Easier Than You Think

Authors: Haoru Wang, Qianfan Shen, Kai Ye, Wenzheng Chen, Baoquan Chen

Organizations: Peking University

Abstract

Reconstruct where the images provide evidence, and generate where they do not: recent success of spatial world models such as Atlas (World Labs Team, 2026) highlights the value of unifying reconstruction and generation in one model. Yet the two have long lived in separate paradigms with distinctive failure modes: feed-forward reconstruction averages ambiguity into blur, while conditional generation invents plausible but scene-inconsistent detail. In this work, we present a unified flow-based formulation for reconstruction and generation, where a shared clean-target predictor performs direct reconstruction at its single-step endpoint and unfolds conditional generation through multi-step flow. A controlled toy study reveals the mechanism: with a single step, the predictor collapses to the conditional mean just like feed-forward methods, favoring consistency over diversity. With multi-step inference, the fidelity of generated details grows with context richness: closer observations reduce ambiguity and yield better-matched details. We further instantiate the formulation in appearance and geometry 3D tasks. JiT-LVSM improves perceptual and distributional quality in novel view synthesis, while JUSt3R retains competitive single-step geometry prediction with additional multi-step inference capabilities that reduces veil and flying-pixel artifacts, producing cleaner surface structure with greater test-time compute. Together, they show that reconstruction and generation can share both a formulation and a backbone, with their behavior governed by denoising configuration---making unification surprisingly simple.

Figures & tables

Panel of Figure 1 — Figure 1: Controlled toy experiment. Rows increase context reliability $r$ . Left blocks show all modes, while right blocks fix one hidden mode for every sample, so wrong-mode samples appear near an incorrect center. Columns: ground truth, the analytic conditional mean, feed-forward prediction, and one flow model evaluated with one and 15 Euler steps. One step tracks the mean, which can lie between modes; 15 steps from the same model sample the posterior.

Figure 2: From feed-forward to flow. Both models keep the original backbone and add only a noisy target

x_{t}

and flow time

t

; one checkpoint is both a one-step predictor and a multi-step generator.

Method	RealEstate10K ( Zhou et al., 2018 )								DL3DV ( Ling et al., 2024 )
	Sparse Context				Rich Context				Sparse Context				Rich Context
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
PixelSplat	17.18	0.566	0.496	101.2	25.51	0.867	0.126	3.102	16.78	0.512	0.511	127.1	21.02	0.725	0.220	5.276
NoPoSplat	16.02	0.511	0.588	98.23	25.46	0.854	0.137	3.017	15.17	0.497	0.602	172.3	21.50	0.751	0.237	5.145
LVSM	19.78	0.601	0.331	32.84	27.14	0.875	0.118	2.979	16.81	0.410	0.482	108.7	22.29	0.727	0.201	3.996
JiT-LVSM	17.94	0.593	0.275	21.45	27.77	0.872	0.117	2.977	15.13	0.446	0.421	52.02	22.93	0.769	0.179	3.292

Table 1: Novel view synthesis on RealEstate10K ( Zhou et al., 2018 ) and DL3DV ( Ling et al., 2024 ) . PSNR and SSIM measure agreement with the reference view; LPIPS and FID measure perceptual and distributional quality. Bold : best; underline : second best.

Panel of Figure 3 — Figure 3: Sparse-context comparison on DL3DV ( Ling et al., 2024 ) . LVSM blurs regions the inputs do not constrain; JiT-LVSM commits to sharp detail. This visual improvement need not raise PSNR, as the aggregate sparse-context results in Table 1 also show.

Panel of Figure 4 — Figure 4: Rich-context comparison on RealEstate10K ( Zhou et al., 2018 ) . With more constraining observations, JiT-LVSM’s detail agrees more closely with the reference.

Metric	DL3DV	RealEstate10K
# scenes	$140$	$6424$
RoMA mean	$0.111$	$0.249$
RoMA median	$0.056$	$0.237$
Conf. $>0.5$ mean	$0.049$	$0.142$
Grid cov. $>0.5$ med.	$0.188$	$0.688$

Table 2: Source-view overlap measured with RoMA on the two evaluation sets.

	AbsRel $\downarrow$				$\delta_{1.25}$ $\uparrow$				BF1 $\uparrow$ ( $\times 100$ )				veil $\downarrow$				flying $\downarrow$
Benchmark	DUSt3R ∗	DUSt3R	JUSt3R 1	JUSt3R 10	DUSt3R ∗	DUSt3R	JUSt3R 1	JUSt3R 10	DUSt3R ∗	DUSt3R	JUSt3R 1	JUSt3R 10	DUSt3R ∗	DUSt3R	JUSt3R 1	JUSt3R 10	DUSt3R ∗	DUSt3R	JUSt3R 1	JUSt3R 10
NYU-v2	.080	.075	.070	.071	.907	.915	.925	.925	5.17	7.10	8.03	8.84	.220	.215	.219	.186	.041	.047	.037	.023
TUM	.180	.177	.165	.178	.774	.750	.782	.749	2.29	2.09	3.34	4.16	.271	.273	.231	.169	.027	.040	.038	.022
Bonn	.141	.095	.096	.105	.825	.904	.902	.885	1.88	2.71	2.83	2.57	.275	.230	.220	.144	.016	.024	.021	.014
Sintel	.413	.418	.400	.402	.560	.533	.560	.558	.36	.16	.36	1.24	.191	.210	.188	.185	.023	.030	.030	.017
KITTI	.112	.098	.096	.109	.863	.897	.899	.873	sparse GT				sparse GT				.109	.107	.106	.089

Table 3: Zero-shot depth on NYU-v2 ( Silberman et al., 2012 ) , TUM ( Sturm et al., 2012 ) , Bonn ( Palazzolo et al., 2019 ) , Sintel ( Butler et al., 2012 ) , and KITTI ( Geiger et al., 2013 ) . DUSt3R is retrained with our pipeline and is the controlled baseline; JUSt3R 1 and JUSt3R 10 are the same checkpoint with one and ten steps. Bold : best; underline : second best, ranked among these three. The released DUSt3R 512-DPT checkpoint (DUSt3R ∗ , gray) is shown for reference. Veil rate and BF1 require dense ground truth and are omitted on KITTI.

Panel of Figure 5 — Figure 5: Point-cloud comparison on two NYU-v2 scenes and one BlendedMVS scene (top to bottom), rendered from a $14^{\circ}$ offset view. DUSt3R and one-step JUSt3R exhibit streaks and stretched surfaces near depth boundaries, where the conditional mean falls between surfaces; multi-step inference from the same JUSt3R checkpoint separates them.

Panel of Figure 6 — Figure 6: Pointmaps from JUSt3R 50 on object, indoor, and outdoor inputs.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Reliability	Method	CM error $\downarrow$	Mode dist. $\downarrow$	Mode cons. $\uparrow$	Distortion $\downarrow$	Diversity
$r=0.0$	exact	$1.98173{\pm}0.00244$	$0.18787{\pm}0.00069$	$0.14550{\pm}0.00360$	$3.95202{\pm}0.02895$	$3.95706{\pm}0.00373$
$r=0.0$	mean	$0.00000{\pm}0.00000$	$1.75274{\pm}0.00107$	$0.16716{\pm}0.00050$	$1.97979{\pm}0.00252$	–
$r=0.0$	feed-forward	$0.00136{\pm}0.00036$	$1.74957{\pm}0.01735$	$0.16368{\pm}0.00349$	$1.98199{\pm}0.00303$	–
$r=0.0$	flow-1	$0.00256{\pm}0.00084$	$1.76830{\pm}0.01112$	$0.16583{\pm}0.00163$	$1.98313{\pm}0.00390$	$0.00091{\pm}0.00031$
$r=0.0$	flow-15	$1.86915{\pm}0.05846$	$0.19175{\pm}0.01137$	$0.14235{\pm}0.00379$	$3.85215{\pm}0.05699$	$3.72978{\pm}0.11580$
$r=0.4$	exact	$1.90867{\pm}0.00603$	$0.18853{\pm}0.00102$	$0.17390{\pm}0.00179$	$3.83301{\pm}0.02287$	$3.80880{\pm}0.00182$

Appendix

Table 4: Toy metrics, mean

\pm

std over five seeds. One flow checkpoint per

(r,\text{seed})

is evaluated with one Euler step ( flow-1 ) and with 15 ( flow-15 ). mean is the analytic

\mathbb{E}[x_{0}\mid c]

and exact the exact posterior sampler. One-step predictors match mean ; the 15-step sampler matches the mode selection of exact . CM error: per-coordinate squared error to

\mathbb{E}[x_{0}\mid c]

; mode dist.: distance to the nearest mode center; mode cons.: fraction of predictions nearest the true mode.

	$r=0.80$	$r=0.85$	$r=0.90$	$r=0.95$	$r=1.00$
exact	$0.510$	$0.689$	$0.898$	$0.998$	$1.000$
flow-15	$0.519$	$0.690$	$0.895$	$0.997$	$1.000$

Appendix

Table 5: Mode consistency near rich context (seed 0). flow-15 tracks the exact posterior sampler as the posterior contracts.

Figure 7: Noised-LVSM on RealEstate10K: extrapolation improves while interpolation remains comparable.

Panel of Figure 8 — Figure 8: Random DL3DV visual samples, page 1. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 9 — Figure 9: Random DL3DV visual samples, page 2. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 10 — Figure 10: Random DL3DV visual samples, page 3. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 11 — Figure 11: Random RealEstate10K visual samples, page 1. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 12 — Figure 12: Random RealEstate10K visual samples, page 2. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 13 — Figure 13: Random RealEstate10K visual samples, page 3. Each row shows two randomly selected scenes; each scene includes two input views and one target view, with columns ordered as LVSM, JiT-LVSM, and reference.

Panel of Figure 14 — Figure 14: Additional JUSt3R comparisons, page 1. Each case shows depth predictions followed immediately by its offset point-cloud views.

Panel of Figure 15 — Figure 15: Additional JUSt3R comparisons, page 2. Each case shows depth predictions followed immediately by its offset point-cloud views.

Panel of Figure 16 — Figure 16: Additional JUSt3R comparisons, page 3. Each case shows depth predictions followed immediately by its offset point-cloud views.

Panel of Figure 17 — Figure 17: Additional JUSt3R comparisons, page 4. Each case shows depth predictions followed immediately by its offset point-cloud views.

Panel of Figure 18 — Figure 18: Additional JUSt3R comparisons, page 5. Each case shows depth predictions followed immediately by its offset point-cloud views.

Explore similar work

GenRec: Knowing Where to Reconstruct and Where to Generate
Aug 18, 2026Ata Çelen, Jaewoo Jung, Federico Tombari +4Novel View Synthesis Sparse-View 3D Reconstruction
ETH Zürich · KAIST · Google +1
FlowObject: Flow Steering for Bridging Generative Priors and Reconstruction Fidelity
Jun 17, 2026Yuchen Rao, Xuqian Ren, Yinyu Nie +4Flow Matching 3D Reconstruction
Graz University of Technology Austria · Tampere University Finland · Technical University of Munich Germany +3
ReconPlusGen: Injecting Reconstruction Prior into Multi-view 3D Generation through Noise Inversion and Modulation
Sep 10, 2026Jiarui Liu, Heng Li, Weiyu Li +73D Shape Generation 3D Diffusion Models
HKUST · LightIllusions · BYD +1

Aug 18, 2026J/K move · Enter open · S save

GenRec: Knowing Where to Reconstruct and Where to Generate

Ata Çelen, Jaewoo Jung, Federico Tombari, Marc Pollefeys+3

ETH Zürich · KAIST · Google+1

Novel View Synthesis Sparse-View 3D Reconstruction Flow-Based Generative Modeling

Open paper AskPDF arXiv

Generative novel view synthesis from sparse input images is rarely all reconstruction or all generation: pixels visible in some source view have a unique correct value modulated only by view-dependent shading, while pixels in disocclusions or beyond the captured volume admit a distribution of plausible completions. Existing generative novel-view-synthesis methods conflate these regimes under a single uniform loss, blurring the line between geometric fidelity and creative hallucinations even when scene geometry is injected through warped point clouds or projected depth. We introduce GenRec, a multi-view flow matching model that builds the reconstruction--generation split directly into its architecture, supervision, and gradient flow. Guided by an observation mask derived from the source cameras and a monocular depth estimator, a flow matching backbone jointly denoises RGB and scene-coordinate maps across all target views, while a pixel-space refinement stage restores high-frequency detail on observed pixels; the same mask gates supervision so regression signals do not contaminate the generative prior. Across RealEstate10K, DL3DV-10K, and Mip-NeRF~360, in both single-view extrapolation and two-view interpolation, GenRec attains the best reconstruction fidelity in observed regions while also surpassing purely generative baselines on perceptual quality in unobserved ones, showing the effectiveness of our approach.

Abstract

Explore similar work

GenRec: Knowing Where to Reconstruct and Where to Generate

FlowObject: Flow Steering for Bridging Generative Priors and Reconstruction Fidelity

ReconPlusGen: Injecting Reconstruction Prior into Multi-view 3D Generation through Noise Inversion and Modulation