ReFlow: Self-correction Motion Learning for Dynamic Scene Reconstruction

Yanzhe Liang; Ruijie Zhu; Hanzhi Chang; Zhuoyuan Li; Jiahao Lu; Tianzhu Zhang

Figure 1: Typical Challenges in monocular dynamic scene reconstruction. Top: Incomplete initialization for dynamic regions: the initial 3D structure from SfM often misses dynamic components and initializes Gaussians without separating static points (green) from dynamic points (red) , leading to an entangled and incomplete representation. Bottom: To compensate, existing methods frequently resort to external dense motion guidance to constrain and stabilize the reconstruction of dynamic regions.

Figure 2: Motivation of Self-correction Flow Matching. (a) We start with a simple observation: 2D observations, such as the shifting balloon, are caused by 3D motion. Accurate reconstructed 3D Motion should naturally align with these visible changes. (b) Unlike previous methods that use external motion priors to supervise 3D motion, we instead uses raw video as motion supervision through a self-correction flow matching mechanism to directly align predicted 3D motion projections with 2D frame differences.

Figure 3: Overview of ReFlow . We start by constructing a complete canonical space(Sec. 3.2.1 ), which includes both static and dynamic components, ensuring a reliable 3D scene initialization. Next, we disentangle these elements using spatial and spatiotemporal feature planes(Sec. 3.2.2 ), providing a structured representation that separately handles static and dynamic regions. This preparation allows us to introduce targeted motion constraints(Sec. 3.3 ): Full Flow supervises motion across the entire scene, while Camera Flow enforces consistency in static regions, enabling the self-correction learning mechanism for accurate 3D motion reconstruction.

Figure 4: Self-correction flow matching mechanism. (a) Different Motion and Flow in the 4D Scene. Static areas move only due to camera motion (camera flow), while dynamic areas involve both camera and object motion (full flow). Accurate motion learning requires region-specific flow supervision. (b) Self-correction flow matching. We apply full flow to warp the entire image from state

t_{1}

to state

t_{2}

and compare with the real observation, validating overall motion. Camera flow is used similarly but only on static regions, ensuring their stability. Together, these provide a complementary self-correction signal for 3D motion learning.

Method	Balloon1			Balloon2			Jumping			Playground
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS [ 28 ]	17.71	0.407	0.473	20.62	0.500	0.431	14.45	0.394	0.385	15.75	0.341	0.493
STGS [ 34 ]	20.36	0.746	0.196	23.12	0.881	0.124	20.82	0.631	0.187	19.23	0.842	0.151
Ex4DGS [ 30 ]	14.69	0.351	0.503	16.29	0.471	0.457	18.93	0.661	0.321	14.16	0.442	0.437
Efficient-D3DGS [ 27 ]	23.32	0.864	0.140	26.20	0.906	0.099	21.54	0.784	0.253	24.08	0.892	0.098
Deformable-3DGS [ 63 ]	26.12	0.891	0.122	26.48	0.910	0.100	23.21	0.837	0.199	23.96	0.901	0.088

Table 1: Quantitative comparison on Nvidia Monocular dataset. We report PSNR/SSIM/LPIPS per scene; the last block shows the mean across all available scenes (including dynamicFace when available).

Figure 5: Qualitative comparison on Nvidia Monocular dataset [ 18 ] . Yellow boxes highlight zoomed-in regions for detail examination. Per-scene average PSNR values are provided.

Figure 6: Qualitative comparison on Nerfies-HyperNeRF dataset [ 45 , 46 ] . Yellow boxes highlight zoomed-in regions for detail examination. Per-scene average PSNR values are provided.

Methods	(a) Nerfies-HyperNeRF
Methods	PSNR $\uparrow$	SSIM $\uparrow$
T-NeRF [ 47 ]	20.43	0.508
NSFF [ 35 ]	19.41	0.423
Nerfies [ 45 ]	20.08	0.462
HyperNeRF [ 46 ]	20.40	0.467
Deformable-3DGS [ 63 ]	22.19	0.558

Table 2: Quantitative comparison on Nerfies-HyperNeRF dataset [ 45 , 46 ] . We highlight the best and second best results.

Figure 7: Visualization of self-correction flow matching effects. Top: Incorrect motion leads to erroneous warping results that do not align with the ground truth, highlighted by yellow boxes, which in turn drives motion optimization. Bottom: After flow matching, finer details are better reconstructed, demonstrating improved rendering quality.

#	CompInit.	Sep.	FullFM	CamFM	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Baseline					25.81	0.844	0.210
1			✓		26.45	0.865	0.172
2	w/o align				24.50	0.815	0.260
3	✓				26.60	0.870	0.167
4	✓	✓			27.00	0.880	0.159
5	✓	✓	✓		27.85	0.897	0.116

Table 3: Ablation study of key components in our method. CompInit.: Complete Canonical Space Initialization; Sep.: Separation-based Dynamic Scene Modeling; FullFM: Full Flow Matching; CamFM: Camera Flow Matching. We highlight best performance in bold .

Figure 8: Illustration of the computation of Full Flow (top-left) and Camera Flow (bottom-right).

Method	Balloon1	Balloon2	dynamicFace	Jumping
Baseline [ 55 ]	35m 32s / 6.30G	37m 49s / 16.88G	58m 23s / 6.24G	37m 56s / 3.20G
Ours	44m 52s / 7.53G	46m 4s / 18.33G	68m 28s / 7.34G	46m 41s / 4.53G
Method	Playground	Skating	Truck	Umbrella
Baseline [ 55 ]	51m 19s / 6.45G	37m 19s / 2.32G	34m 56s / 4.12G	44m 45s / 5.96G
Ours	61m 9s / 7.50G	47m 30s / 3.60G	42m 58s / 5.61G	54m 15s / 7.11G

Table 4: Training Time and Peak GPU Memory Usage. We compare our method with the baseline on the Nvidia Monocular dataset.

Method	Balloon1	Balloon2	dynamicFace	Jumping
Baseline [ 55 ]	20fps / 74M	21fps / 72M	15fps / 106M	26fps / 35M
Ours	18fps / 87M	17fps / 86M	13fps / 121M	23fps / 52M
Method	Playground	Skating	Truck	Umbrella
Baseline [ 55 ]	15fps / 93M	30fps / 33M	24fps / 45M	25fps / 54M
Ours	15fps / 105M	25fps / 47M	21fps / 65M	21fps / 72M

Table 5: Rendering Speed and Model Storage Size. We report the rendering frame rate (FPS) and the final model size (in MBs) for all scenes on the Nvidia Monocular dataset.

Figure 9: Failure case of the Utilized Geometry Foundation Model. Top Left: Aggregated point cloud exhibits noise and distortion from merging inconsistent frames. Top Right: Per-frame structure reconstructions show significant noise and fragmentation, demonstrating its difficulty in establishing reliable geometry.

Figure 10: Visualization of our self-correction flow matching progress across training iterations, using the DynamicFace sequence from the Nvidia Monocular dataset [ 18 ] . Each row shows results at different training iterations: 300 (top row), 1000 (second row), 5000 (third row), and 7000 (bottom row). The columns present: Left column (

I_{1}

): Ground truth image at time

t_{1}

. Middle column (

I_{1}^{warped}

): Ground truth image from

t_{1}

warped using our predicted flow field

\mathbf{F}

, demonstrating how content transforms to match the target frame. Right column (

I_{2}

): Ground truth image at time

t_{2}

, serving as the reference for evaluating warping accuracy. As training progresses,

I_{1}^{warped}

increasingly aligns with

I_{2}

, demonstrating that our self-correction flow matching effectively learns to model dynamic scene motion without requiring external motion guidances.

Method	Balloon1	Balloon2	Jumping	dynamicFace
Ours	27.65 / 0.903 / 0.092	29.01 / 0.908 / 0.098	24.85 / 0.855 / 0.151	30.98 / 0.973 / 0.036
Ours (External Flow)	27.35 / 0.899 / 0.095	27.74 / 0.843 / 0.179	24.92 / 0.853 / 0.162	31.10 / 0.975 / 0.034
Method	Playground	Skating	Truck	Umbrella
Ours	25.58 / 0.889 / 0.086	30.17 / 0.946 / 0.076	30.18 / 0.917 / 0.115	27.21 / 0.831 / 0.170
Ours (External Flow)	20.97 / 0.820 / 0.099	30.09 / 0.976 / 0.032	29.30 / 0.905 / 0.133	27.35 / 0.831 / 0.171

Table 6: Quantitative comparison using PSNR(

\uparrow

), SSIM(

\uparrow

), and LPIPS(

\downarrow

) metrics. Best results are in bold .

Figure 11: Qualitative comparison on all scenes from the Nvidia Monocular dataset [ 18 ] .

Figure 12: More qualitative comparisons from the Nerfies-HyperNeRF dataset [ 45 , 46 ] .

Figure 13: Novel time synthesis results with trajectory visualization across different dynamic scenes. Each column shows a different dataset: Nvidia-Playground, Nvidia-Umbrella, Nvidia-Skating, Hypernerf-broom, and Hypernerf-banana. The yellow boxes highlight dynamic/static regions with our tracked trajectories visualized using the Deform-GS approach. Rows represent different timesteps with fixed camera positions, demonstrating how our method correctly models temporal scene evolution. Note how static background elements remain perfectly stable across frames while dynamic components exhibit physically plausible motion paths. The Playground scene (leftmost column) particularly demonstrates our method’s capability to preserve fine structures like blue ribbons during motion, which are typically challenging to reconstruct accurately.

Method	Broom		Tail		Toby-sit
Method	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
T-NeRF [ 47 ]	20.17	0.257	22.11	0.385	18.53	0.330
NSFF [ 35 ]	20.46	0.247	21.72	0.388	18.65	0.329
Nerfies [ 45 ]	19.51	0.202	21.17	0.305	18.41	0.326
HyperNeRF [ 46 ]	19.23	0.197	21.13	0.301	18.33	0.324
Deformable-3DGS [ 63 ]	20.51	0.352	22.10	0.474	21.13	0.427

Table 8: Per-scene performance metrics on Nerfies-HyperNeRF [ 45 , 46 ] dataset. We highlight the best and second best results.

Method	3D Printer		Chicken		Broom		Banana
Method	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
T-NeRF [ 47 ]	18.60	0.591	24.41	0.764	20.17	0.257	22.07	0.721
NSFF [ 35 ]	16.26	0.426	20.72	0.619	20.46	0.247	18.62	0.530
Nerfies [ 45 ]	18.81	0.588	22.71	0.742	19.51	0.202	19.85	0.609
HyperNeRF [ 46 ]	18.73	0.583	23.88	0.753	19.23	0.197	21.08	0.641
Deformable-3DGS [ 63 ]	20.53	0.641	22.82	0.618	20.51	0.352	26.05	0.833

Table 9: Per-scene performance metrics on HyperNeRF [ 46 ] dataset. We highlight the best and second best results.

ReFlow: Self-correction Motion Learning for Dynamic Scene Reconstruction

Abstract

Figures & tables

Scene Flow Estimation

MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer

UniQuery4R: Unified 4D Scene Reconstruction from a Single Query

RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction

Abstract

Scene Flow Estimation

Explore similar work

MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer

UniQuery4R: Unified 4D Scene Reconstruction from a Single Query

RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction