We present ProDyGS, a novel dynamic 3D Gaussian Splatting framework for high-quality novel view synthesis from videos captured by a single static camera. While existing methods rely on multi-view setups or significant camera motion for geometric constraints, our approach addresses the challenging scenario where multi-view supervision is completely absent. We overcome this limitation by generating synthetic multi-view supervision through depth-guided proxy image synthesis. Specifically, we estimate temporally consistent depth maps using foundational monocular depth networks, then construct 3D Gaussian representations that generate proxy images from arbitrary viewpoints. A deformation network learns temporal dynamics by warping canonical Gaussians using this augmented supervision. Experiments on the DyNeRF dataset demonstrate that our method achieves state-of-the-art performance while requiring only monocular depth estimation as external supervision, outperforming approaches that rely on stronger priors such as scene flow.
Figures & tables
Fig. 2: Proxy images generation. In ProDyGS for each frame Ii0 of the input sequence we initialize a lightweight 3DGS model out of each pair (Ii0,D˙i0) . We then use the 3DGS model to render the additional proxy images {{Iij}j=1M}i=0N from each of M viewpoints {Πj}j=1M , by means of differentiable splatting S(Πj,Gi) .
Fig. 3: Overview of our ProDyGS framework. The deformation network Ω consists of a spatio-temporal encoder, based on a multi-resolution HexPlane and a lightweight MLP, that extracts features from both the Gaussians and the timestamp. A multi-head decoder D={ϕx,ϕs,ϕr,ϕo,ϕc} then predicts per-Gaussian updates of position, scale, rotation, opacity, and color. These updates are applied to the canonical Gaussians G to obtain the deformed Gaussians Gi′
Method
cut_roasted_beef
sear_steak
coffee_martini
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
HexPlane [ 18 ]
16.76
0.5382
0.5054
16.89
0.5897
0.5049
13.26
0.4049
0.5835
SC-GS [ 38 ]
20.69
0.7414
0.2625
21.23
0.7870
0.2188
19.02
0.7124
0.2151
D-GS [ 46 ]
22.20
0.7808
0.1931
23.56
0.8101
0.1773
19.23
0.7013
0.2270
MoDGS [ 8 ]
23.98
0.8221
0.1438
23.53
0.8126
0.1642
21.37
0.7962
0.1473
ProDyGS (ours)
27.55
0.8986
0.1010
29.57
0.9220
0.1069
22.73
0.8201
0.1344
TABLE I: Per-scene results on the DyNeRF dataset [ 6 ] . We compare our method with HexPlane [ 18 ] , SC-GS [ 38 ] , Deformable GS (D-GS) [ 46 ] and MoDGS [ 8 ] in PSNR ↑ , SSIM ↑ , and LPIPS ↓ .
DyNeRF
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
HexPlane [ 18 ]
15.33
0.5593
0.4514
SC-GS [ 38 ]
18.77
0.7359
0.2310
D-GS [ 46 ]
19.55
0.7446
0.2171
MoDGS [ 8 ]
22.64
0.8042
0.1545
ProDyGS (ours)
26.64
0.8806
0.1110
TABLE II: Average results on the DyNeRF dataset [ 6 ] .
Fig. 4: Qualitative results on sequences from the Light Field Video dataset [ 91 ] . Each pair compares a frame rendered by ProDyGS (left) with the corresponding ground truth image from one of the dataset’s camera poses (right).
DyNeRF
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
(A)
2 Synthetic Cameras
23.77
0.8280
0.1921
(B)
4 Synthetic Cameras
26.08
0.8780
0.1166
(C)
8 Synthetic Cameras
26.15
0.8790
0.1141
(D)
18 Synthetic Cameras
26.64
0.8806
0.1110
(E)
w/o Refinement
26.58
0.8763
0.1165
TABLE III: Ablation Studies. Impact of the number of proxy images (A-D) and the depth refinement stage (E-F).
Fig. 5: Qualitative comparison on flame_salmon_1 sequence – w/o Refinement (top) vs w/ Refinement (bottom). Depth refinement allows for cleaner discontinuities and higher-frequency details (see the yellow boxes).
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Department of Computing, Hong Kong Polytechnic University, Hong Kong, China +4