Phase-DGS: Phase-guided Dynamic Gaussian Splatting from Unsynchronized Multi-view Video

Hosung Jeon; Jun Young Jeong; Sangwoon Kwak; Joonsoo Kim; Sangmin Kim; Jaesik Park; Won-Sik Cheong; Hyon-Gon Choo

cs.CVNov 3, 2026

Phase-DGS: Phase-guided Dynamic Gaussian Splatting from Unsynchronized Multi-view Video

Authors: Hosung Jeon, Jun Young Jeong, Sangwoon Kwak, Joonsoo Kim, Sangmin Kim, Jaesik Park, Won-Sik Cheong, Hyon-Gon Choo

Organizations: Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · Seoul National University, Seoul, Republic of Korea

Abstract

Spatiotemporal scene reconstruction with Dynamic Gaussian Splatting (DGS) fundamentally depends on perfect temporal synchronization across all cameras, an assumption rarely achievable in practice. Industry-standard synchronization requires expensive specialized hardware and complex setup, while data-driven alternatives including audio-based alignment, geometry matching, and pose tracking remain limited by restrictive environmental or scene-specific assumptions. To overcome these limitations, we propose Phase-DGS, a framework that replaces conventional timestamps with a semantic phase representing each frame's inherent state within the underlying motion cycle. Frames capturing the same motion state share the same phase regardless of which camera records them or when, naturally establishing spatiotemporal correspondence across unsynchronized cameras. We validate Phase-DGS under temporal offsets, frame drops, and camera freezes, achieving up to +8.5 dB PSNR improvement over baselines while enabling seamless integration with existing DGS backbones including RealTime-4DGS and FreeTimeGS.

Figures & tables

Figure 1: Conceptual diagram of Phase-DGS. (a) In an unsynchronized multi-view dataset, images sharing the frame index are actually captured at different time instants. The proposed phase extraction module estimates an inherent semantic phase from each input image, which is then used to train 4DGS. (b) The graph shows the estimated phases across cameras. When a sequence is offset, the phase shifts along the x-axis direction; when frozen, the phase is predicted parallel to the x-axis ; and when dropped, the phase is predicted parallel to the y-axis .

Figure 2: Training pipeline of Phase-DGS. Multi-view data provides camera parameters (view matrix, focal length, resolution) and an RGB image. The phase projector module predicts a phase value from the RGB image, replacing the timestamp input of the 4DGS backbone. The feature bank serves as a learnable reference repository that enforces cross-camera phase coherence, so that identical motion states share the same phase across all views. Training is guided by three losses: a Rendering loss between the rendered and input images, a monotonic loss, and a phase consistency loss.

			SelfCap			Nvidia
Condition	Backbone	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Synced	K-Planes	Sync-NeRF	-	-	-	28.889	0.942	0.048
	RealTime-4DGS	-	30.858	0.970	0.030	29.735	0.952	0.045
	RealTime-4DGS	Ours	32.145	0.973	0.026	28.938	0.946	0.061
	FreeTimeGS	-	34.181	0.984	0.010	31.167	0.963	0.022
		VisualSync	33.446	0.983	0.013	30.429	0.958	0.025

Table 1: Average quantitative comparison on the SelfCap and Nvidia Dynamic Scene datasets under synchronized (Synced) and unsynchronized (Shifted, Dropped) conditions.

\uparrow

: higher is better,

\downarrow

: lower is better. Green : best among methods sharing the same backbone and condition. Red : best across all methods under the same condition.

Panel of Figure 3 — Figure 3: Qualitative comparison under the synchronized condition (Synced) and the unsynchronized condition (Shifted and Dropped) on Nvidia Dynamic Scene dataset and SelfCap datasets . Phase-DGS consistently produces cleaner renderings while baselines exhibit ghosting and geometric distortion.

	Jumping			Bike
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Ours (default)	29.386	0.942	0.037	31.212	0.977	0.016
w/o Monotonic Loss	24.421	0.887	0.073	25.903	0.944	0.039
w/o Bank Learning	24.794	0.897	0.070	27.642	0.958	0.026
$k=1$	29.425	0.942	0.038	31.130	0.977	0.015
$k=4$	29.006	0.938	0.038	31.101	0.976	0.015

Table 2: Ablation study on the Jumping scene (Nvidia Dynamic Scene dataset) and Bike scene (SelfCap dataset) under the Dropped condition. FreeTimeGS is used as the DGS backbone.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Multi-camera frames at the same frame index from the Jumping sequence (Nvidia Dynamic Scene Dataset) under the Dropped condition. Despite sharing the same frame index, each camera captures a distinct motion state, illustrating the severity of temporal misalignment introduced by frame drops and freezes.

Figure A2: Interface of the ROI extraction tool used in our experiments. The tool supports manual bounding box selection, with options to use the central square region or the full frame. Different cameras in the same scene can have independent bounding boxes.

Configuration	GPU Memory	Training Throughput
FreeTimeGS (backbone only)	1686 MB	148 it/s
Phase-DGS (FreeTimeGS)	2200 MB (+514 MB)	70 it/s ( $2.11\times$ slower)

Appendix

Table A1: Training overhead of Phase-DGS with the FreeTimeGS backbone on a 23-camera, 50-frame sequence, measured on a single NVIDIA RTX 5090 GPU.

Figure A3: Phase estimation results on the Jumping scene under the Shifted (top) and Dropped (bottom) conditions. (Left) Ground-truth frame correspondences between each camera and the source video. (Center, Right) Per-frame phase estimates produced by Phase-DGS with the RealTime-4DGS and FreeTimeGS backbones, respectively.

Condition	Scene	$W{=}0$	$W{=}5$	$W{=}10$ (default)	$W{=}50$
Synced	Bike	31.355	31.376	31.376	31.376
Synced	Jumping	28.851	28.851	28.851	28.851
Shifted	Bike	31.110	31.299	31.299	31.299
Shifted	Jumping	27.354	28.090	28.103	28.103
Dropped	Bike	31.131	31.217	31.217	31.217
Dropped	Jumping	29.386	29.386	29.386	29.386

Appendix

Table A2: Sensitivity of PSNR to the search window

W

of the test-time phase search with the FreeTimeGS backbone.

W{=}0

denotes rendering at the reference phase without search, and

W{=}10

is the default setting.

Figure A4: To assess rendering quality in dynamic regions, we generate segmentation masks from ground-truth images and compute metrics exclusively over the masked areas. The mask is generated by object segmentation using SAM2 [ 26 ] , and a small padding margin is applied to the mask boundaries to assess the reconstruction quality of regions surrounding dynamic objects.

Figure A5: Per-frame phase estimates on the Bike scene (SelfCap) under the Dropped condition. (Top left) Ground-truth frame correspondences. Phase estimation results under (Top right) default setting, (Bottom left) without monotonic loss. (Bottom right) without feature bank learning.

Run	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
1	35.81	0.989	0.008
2	36.86	0.990	0.007
3	36.04	0.990	0.008
4	35.99	0.990	0.008
Mean	36.175	0.990	0.008
Std	0.466489	0.000450	0.000455

Appendix

Table A3: Numerical stability of Phase-DGS across four independent runs on the Corgi scene (SelfCap dataset) under the Dropped condition.

Figure A6: Failure Case of Phase-DGS on Balloon1, a Highly Periodic Scene.

			Bike			Corgi			Yoga
Condition	Backbone	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Synced	K-Planes	Sync-NeRF	-	-	-	-	-	-	-	-	-
	RealTime-4DGS	-	26.430	0.950	0.040	35.552	0.985	0.013	30.593	0.974	0.036
	RealTime-4DGS	Ours	28.565	0.960	0.032	34.907	0.982	0.019	32.963	0.978	0.028
	FreeTimeGS	-	30.888	0.977	0.014	35.677	0.990	0.006	35.977	0.987	0.011
		VisualSync	30.490	0.974	0.016	36.646	0.990	0.006	33.203	0.984	0.016

Appendix

Table A4: Quantitative comparison on the SelfCap dataset under synchronized (Synced) and unsynchronized (Shifted, Dropped) conditions.

\uparrow

: higher is better,

\downarrow

: lower is better. Green : best among methods sharing the same backbone and condition. Red : best across all methods under the same condition.

Dynamic Scene Reconstruction

Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.

See all 300 papers

ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment
Aug 3, 2026Junyu Zhu, Hao Zhu, Xinzhuo Zhang +33D Reconstruction Multi-View 3D Reconstruction
School of Electronic Science and Engineering, Nanjing University, Nanjing, 210023, China · College of Engineering, Computing and Cybernetics, Australian National University, Canberra, ACT 2601, Australia
GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis
Jul 23, 2026Jiahao He, Yihua Shao, Zhengkai Zhao +63D Gaussian Splatting Dynamic Scene Reconstruction
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Department of Computing, Hong Kong Polytechnic University, Hong Kong, China +4
FreeTimeGS++: Secrets of Dynamic Gaussian Splatting and Their Principles
May 5, 2026Lucas Yunkyu Lee, Soonho Kim, Youngwook Kim +23D Gaussian Splatting 4D Reconstruction
Seoul National University · POSTECH

Aug 3, 2026J/K move · Enter open · S save

ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment

Junyu Zhu, Hao Zhu, Xinzhuo Zhang, Hongdong Li+2

School of Electronic Science and Engineering, Nanjing University, Nanjing, 210023, China · College of Engineering, Computing and Cybernetics, Australian National University, Canberra, ACT 2601, Australia

3D Reconstruction Multi-View 3D Reconstruction Dynamic Scene Reconstruction

Open paper AskPDF arXiv

Dynamic 3D scene reconstruction has achieved remarkable success under the assumption of strictly synchronized multi-camera inputs. However, in real-world scenarios, temporal asynchrony among capturing devices remains a critical challenge, leading to severe motion blur and geometric artifacts. Existing asynchronous reconstruction methods typically estimate temporal offsets through photometric supervision, but appearance matching provides weak temporal cues under large offsets and complex motions. We attribute this limitation to two major bottlenecks: texture-induced collapse, where low-texture regions provide nearly vanishing alignment signals, and deformation-induced coupling, where temporal errors are absorbed into distorted geometry or motion rather than being explicitly corrected. To address these issues, we propose ASTRA (Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment), a framework that introduces 2D motion trajectories as explicit, texture-agnostic supervision for asynchronous dynamic reconstruction. Instead of synchronizing cameras solely through rendered color residuals, ASTRA jointly optimizes temporal offsets and dynamic 3D representations by aligning the projected motion of reconstructed 3D points with observed 2D trajectories, while using dynamic and certainty masking to suppress unreliable trajectory constraints. Extensive experiments on different dynamic Gaussian Splatting backbones show that ASTRA preserves high-frequency spatial details and sustains strong robustness even under severe asynchrony with up to 25-frame offsets, achieving approximately 1.4 dB PSNR improvement, reducing temporal-offset MAE by 54.0%, and nearly quadrupling the synchronization success rate.

Abstract

Dynamic Scene Reconstruction

Explore similar work

ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment

GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis

FreeTimeGS++: Secrets of Dynamic Gaussian Splatting and Their Principles