Temporal retiming, the ability to reconstruct and render dynamic scenes at arbitrary timestamps, is crucial for applications such as slow-motion playback, temporal editing, and post-production. However, most existing 4D Gaussian Splatting (4DGS) methods overfit at discrete frame indices but struggle to represent continuous-time frames, leading to ghosting artifacts when interpolating between timestamps. We identify this limitation as a form of temporal aliasing and propose RetimeGS, a simple yet effective 4DGS representation that explicitly defines the temporal behavior of the 3D Gaussian and mitigates temporal aliasing. To achieve smooth and consistent interpolation, we incorporate optical flow-guided initialization and supervision, triple-rendering supervision, and other targeted strategies. Together, these components enable ghost-free, temporally coherent rendering even under large motions. Experiments on datasets featuring fast motion, non-rigid deformation, and severe occlusions demonstrate that RetimeGS achieves superior quality and coherence over state-of-the-art methods.
Figures & tables
Figure 2: Illustration of temporal overfitting to input frames t1 and t2 , causing ghosting at t1.5 in 4D primitive-based methods.
Figure 3: Pipeline Overview. We represent a dynamic scene using a novel 4D representation that combines regularized temporal opacity with smooth spline-based spatial positioning. By leveraging tailored training strategies using RGB images and bidirectional optical flow, our method can reconstruct arbitrary intermediate frames under sparse temporal sampling and large motion.
Figure 4: Qualitative comparison. Results on DNA-Rendering Dataset [ 4 ] (w/o GT held-out views, left) and Stage-Capture Dataset (w/ GT held-out views, right). The red boxes show the corresponding zoomed-in views for detailed comparison.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Deform-GS [ 39 ]
28.45
0.867
0.0272
STGS [ 18 ]
25.34
0.825
0.0357
GaussianFlow [ 6 ]
25.91
0.825
0.0339
2D Lifting [ 28 , 18 ]
28.79
0.886
0.0267
Ours
30.08
0.904
0.0225
Table 1: Quantitative comparisons on the Stage-Capture Dataset, focusing on the foreground region. Red and yellow cell colors indicate the best and second-best results, respectively.
Figure 5: ( 5(a) ) The texture is distorted when flow-related components are removed. ( 5(b) ) Without triple rendering, the two adjacent primitive groups each capture only part of the content in the input frames they jointly cover: the previous group reconstructs the right texture in the circled region, while the next group reconstructs the left texture.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
w/o flow initialization
29.69
0.899
0.0227
w/o flow supervision
27.24
0.861
0.0282
w/o triple-rendering
27.16
0.849
0.0319
w/o dynamic stretching
28.81
0.886
0.0247
linear trajectory
28.50
0.884
0.0243
Ours
30.08
0.904
0.0225
Table 2: Quantitative ablation on the Stage-Capture Dataset, focusing on the foreground region. Red and yellow cell colors indicate the best and second-best results, respectively.
Figure 6: Ablation on dynamic stretching. The magenta is rendered using static stretched primitives, and the teal is rendered using dynamic primitives (with static background removed).
Figure 7: Ablation on spline trajectory. Without spline trajectories, the error heatmap highlights larger errors along the bear’s edges due to imprecise piecewise linear motion.
Figure 8: Failure case under extremely low capture FPS. Our method struggles to interpolate intermediate frames when the inter-frame motion becomes too large due to low temporal sampling or large motion.
Scenes
Avg.
Method
Bear
Doll
Undress
Open Case
Pass Doll
Pickup Doll
Pack Computer
Stretch
Walk
PSNR ↑
Deform-GS [ 39 ]
29.91
27.21
29.46
29.99
27.87
28.30
29.96
25.81
27.56
28.45
STGS [ 18 ]
22.96
23.49
28.41
29.09
24.07
23.44
28.75
23.42
24.46
25.34
GaussianFlow [ 6 ]
24.53
24.48
28.58
28.87
25.11
24.67
28.71
23.88
24.38
25.91
2D Lifting [ 28 , 18 ]
28.60
26.60
30.23
31.16
27.97
28.37
30.09
27.21
28.82
28.79
Table 3: Per-scene quantitative comparisons on the Stage-Capture Dataset (foreground region). Higher PSNR/SSIM and lower LPIPS are better.
Figure 9: Qualitative comparison on Neural3DV: Scene with fire (rapid opacity changes) and non-stage-capture setting.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Deform-GS [ 39 ]
28.45
0.867
0.0272
STGS [ 18 ]
25.34
0.825
0.0357
GaussianFlow [ 6 ]
25.91
0.825
0.0339
Ex4DGS [ 15 ]
25.95
0.811
0.0379
2D Lifting [ 28 , 18 ]
28.79
0.886
0.0267
Ours
30.08
0.904
0.0225
Table 4: Quantitative results on the Stage-Capture dataset, including the Ex4DGS baseline.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Deform-GS [ 39 ]
31.79
0.952
0.081
STGS [ 18 ]
32.52
0.959
0.079
GaussianFlow [ 6 ]
31.89
0.957
0.082
Ex4DGS [ 15 ]
31.06
0.919
0.094
2D Lifting [ 28 , 18 ]
33.17
0.960
0.080
Ours
33.22
0.959
0.074
Table 5: Quantitative evaluation on the Flame Steak and Flame Salmon scenes from the Neural3DV dataset.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
WAFT [ 36 ]
30.08
0.904
0.0225
SEA-RAFT [ 37 ]
29.73
0.898
0.0253
Table 6: Ablation study on flow supervision: comparison of WAFT [ 36 ] and SEA-RAFT [ 37 ] as optical flow supervision modules.
Method
Training Time (s)
Peak Memory (GB)
STGS [ 18 ]
1406.8
2.47
Ours
3794.3
3.14
Table 7: Comparison of training efficiency and peak GPU memory under a shared budget of 1M primitives.