Reconstructing dynamic 3D scenes with photorealistic detail and strong temporal coherence remains a significant challenge. Existing Gaussian splatting approaches for dynamic scene modeling often rely on per-frame optimization, which can overfit to instantaneous states instead of capturing underlying motion dynamics. To address this, we present Mango-GS, a multi-frame, node-guided framework for high-fidelity 4D reconstruction. Mango-GS leverages a temporal Transformer to model motion dependencies within a short window of frames, producing temporally consistent deformations. For efficiency, temporal modeling is confined to a sparse set of control nodes. Each node is represented by a decoupled canonical position and a latent code, providing a stable semantic anchor for motion propagation and preventing correspondence drift under large motion. Our framework is trained end-to-end, enhanced by an input masking strategy and two multi-frame losses to improve robustness. Extensive experiments demonstrate that Mango-GS achieves state-of-the-art reconstruction quality and real-time rendering speed, enabling high-fidelity reconstruction and interactive rendering of dynamic scenes.
Figures & tables
Figure 1: An overview of the Mango-GS framework. Our method is driven by a set of decoupled control nodes, each comprising a canonical position and a feature code. A dense 3D Gaussian cloud is associated with these nodes via a learned k-NN relationship based on both position and features. A temporal attention network takes the canonical node positions and a time window [0,T] as input, processing them through MLP layers and temporal attention blocks to predict the nodes’ deformations over the entire time window. This learned motion is then propagated back to the Gaussian cloud to produce the final dynamic scene representation for each frame. The entire model is optimized end-to-end with a temporal input masking scheme and a composite loss, which includes a top- k hard-frame photometric loss, and a motion-aware temporal loss.
Figure 2: Position-only nodes versus decoupled nodes with position and code. We visualize 24 Gaussians (red) in high motion region and its three corresponding nodes (white). With the decoupled design, Gaussians attach to semantically consistent nodes rather than merely following spatial neighbors, which struggle under large motion.
Figure 3: The Architecture of the Temporal Deformation Network. For each of the N control nodes, features over a window of T frames are processed by an MLP backbone interleaved with temporal self-attention blocks. Attention operates along the time axis and is fused by a lightweight gate, then decoded to per-node translation, rotation, and scale for all T frames.
Figure 4: Visualization comparison between baselines and our methods from the Neural 3D Video dataset. The main differences are highlighted and zoomed in with boxes.
Method
Neural 3D Video
HyperNeRF-vrig
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
MS-SSIM ↑
tLPIPS ↓
FPS ↑
Storage ↓
D-3DGS
31.15
0.941
0.078
25.0
0.70
0.0234
14.2
172 MB
E-D3DGS
30.86
0.938
0.048
25.4
0.70
0.0257
45.2
64 MB
4DGS
31.58
0.942
0.055
25.2
0.68
0.0248
45.0
59 MB
SC-GS
30.20
0.935
0.067
23.6
0.66
0.0236
24.5
85 MB
GaGS
31.10
0.944
0.060
24.3
0.65
0.0233
12.0
48 MB
Table 1: Quantitative comparison on two datasets. Best is bold and second-best is underlined .
Figure 5: Qualitative comparison on HyperNeRf. Our method offers sharp results. Differences are highlighted with boxes.
(a) Temporal Window Size ( T )
(b) Number of Neighbors ( K )
T
PSNR ↑
SSIM ↑
tLPIPS ↓
FPS ↑
K
PSNR ↑
SSIM ↑
tLPIPS ↓
2
27.53
0.925
0.0225
87.9
2
27.41
0.920
0.0205
4
28.19
0.937
0.0203
117.8
3
28.39
0.942
0.0196
6
28.35
0.942
0.0196
149.5
4
28.26
0.938
0.0199
8
28.24
0.940
0.0197
156.2
5
27.90
0.931
0.0203
Table 2: Ablation on time window size T and number of neighbors K .
Step
Method
PSNR ↑
SSIM ↑
LPIPS ↓
tLPIPS ↓
1
Baseline (single frame)
25.15
0.875
0.139
0.0250
2
+ Nodes (w/o learned affinity)
24.52
0.868
0.142
0.0235
3
+ Decoupled nodes (with learned affinity)
25.31
0.892
0.118
0.0223
4
+ Multi-frame (w/o temporal attention)
27.30
0.928
0.096
0.0225
5
+ Multi-frame (with temporal attention)
27.78
0.937
0.084
0.0196
6
+ Top- k loss
28.05
0.941
0.077
0.0202
Table 3: Ablation on the core components of Mango-GS. We report results from progressively adding each component to a baseline.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Overview of multi-frame group sampling strategies. Colored blocks with number denote real frames, and gray blocks with F indicate placeholder timestamps. A row of blocks represents the T -frame training window.
Sparse ratio
Interpolation ratio
PSNR ↑
MS-SSIM ↑
tLPIPS ↓
1.0
0.0
25.4
0.74
0.0206
0.9
0.1
25.9
0.76
0.0199
0.7
0.3
26.2
0.78
0.0196
0.5
0.5
25.6
0.72
0.0210
Appendix
Table 4: Ablation on the ratio between sparse stride and interpolation samplers.
Initial Nodes
Final Nodes
PSNR ↑
SSIM ↑
LPIPS ↓
512
∼ 710
27.81
0.931
0.088
1024
∼ 960
28.14
0.938
0.079
2048
∼ 1420
28.32
0.942
0.071
4096
∼ 2550
28.25
0.941
0.073
Appendix
Table 5: Ablation on the number of control nodes N . We report initial and final node counts, along with reconstruction quality metrics.
Scene
PSNR ↑
MS-SSIM ↑
tLPIPS ↓
Broom
23.3
0.72
0.0253
Peel-Banana
27.1
0.76
0.0213
3DPrinter
26.3
0.79
0.0178
Chicken
27.7
0.85
0.0139
Mean
26.2
0.78
0.0196
Appendix
Table 6: Per-scene reconstruction quality of Mango-GS on the HyperNeRF-vrig dataset
Figure 7: Sequence of 12 consecutive inference results from Mango-GS ( T=6 ). Each group displays a complete inference output at specific timestamps.