Organizations: The University of Hong Kong Hong Kong, China · University of Newcastle Callaghan, NSW, Australia · Santa Clara University Santa Clara, CA, USA · Futurewei Technologies, Inc. San Jose, CA, USA
High-fidelity free-viewpoint video (FVV) and interactive rendering increasingly rely on explicit Gaussian representations, yet practical deployment remains constrained by representation size, dynamic updates, and computational cost. Existing multi-view video benchmarks provide valuable real-captured content, but they make it difficult to isolate the effects of controlled camera geometry, representation efficiency, and temporal redundancy. We introduce M3ISR, a controlled synthetic benchmark for 3D and 4D Gaussian Splatting (3DGS/4DGS). The benchmark contains 25 scenes from five indoor and outdoor scene groups, two camera/motion configurations, six synchronized 1080p views, and dense ground-truth annotations including RGB, camera parameters, depth, semantic and instance segmentation, and static--dynamic masks. The shared-center camera design intentionally isolates angular view variation and enables controlled evaluation of novel-view synthesis and representation efficiency. We organize M3ISR into five complementary tracks covering 3DGS synthesis, 4DGS synthesis, 4DGS streaming, 3DGS compression, and 4DGS compression. Representative baseline results show small differences in static reconstruction quality but substantial differences in representation storage, while the evaluated streaming methods exhibit substantially higher reported training or reconstruction cost than the corresponding offline dynamic reconstruction baselines. We further define feedforward compression tasks for 3DGS and 4DGS and provide reference rate--distortion formulations and preliminary baseline evaluations. The benchmark is intended as a controlled and complementary testbed for systematic study of Gaussian-based FVV reconstruction, compression, and streaming.
Figures & tables
Figure 1. M 3 ISR camera configuration. Six synchronized views share a common scene center and cover a 120∘ horizontal fan at yaw angles {−60∘,−40∘,−20∘,+20∘,+40∘,+60∘} . Each frame is paired with RGB and camera parameters.
Figure 2. Representative synchronized multi-view samples from M 3 ISR. The displayed figure shows four representative scene groups; the Outdoor row groups the two dataset categories Outdoor1 and Outdoor2 for readability. Each row contains six synchronized camera views.
Dataset
Year/Venue
Type
#Cams
Resolution
FPS
Scenes / Rig Geometry
N3DV ( 23 )
CVPR 2022
R
18–21
2028×2704
30
6 scenes; forward-facing planar
Meeting Room ( 22 )
NeurIPS 2022
R
13
1280×720
30
3 scenes; forward-facing planar
SelfCap ( 49 )
SIGGRAPH Asia 2024
R
22
3840×2160
–
3 sequences; inward-facing surround
Google Immersive Video ( 4 )
SIGGRAPH 2020
R
46
2560×1920
30
15 scenes; hemispherical dome
Ego-Exo4D ( 15 )
CVPR 2024
R
1–4 ego + exo
1440×1440
30
large-scale hybrid ego/exo capture
EPIC-KITCHENS-100 ( 10 )
PAMI 2020
R
ego (+ exo)
1920×1080
60
egocentric trajectories
Table 1. Representative datasets for 3DGS/4DGS streaming evaluation. R denotes real capture and S denotes synthetic. M 3 ISR provides six synchronized views with a shared-center camera layout.
Figure 3. Qualitative comparison of Track 1 sparse-view 3DGS reconstruction on M 3 ISR. Ground truth and results from 3DGS, CoR-GS, FSGS, and DropGS are shown for representative indoor and outdoor scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Storage (MB) ↓
3DGS
18.705
0.655
0.461
1331
CoR-GS
18.825
0.658
0.445
3174
FSGS
18.726
0.654
0.475
508
DropGaussian
18.864
0.657
0.466
992
Table 2. Average static 3DGS metrics on M 3 ISR. Storage is total disk usage over 25 scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Storage (MB) ↓
Train/Recon. Time (s) ↓
Render FPS ↑
QUEEN ( 14 )
15.93
0.5125
0.4786
1734
351.48
54.64
Octree-/LoD-structured ( 31 )
16.32
0.5336
0.4650
1452
356.90
60.10
Table 3. Average streaming metrics reported on M 3 ISR. Time refers to the training or reconstruction time reported by the reference implementation and should not be interpreted as end-to-end streaming latency.
Figure 4. Qualitative comparison of Track 3 4DGS streaming reconstruction on M 3 ISR. Ground truth and results from QUEEN and Octree-LOD are shown for representative dynamic scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Storage (MB) ↓
FPS ↑
Deformable 3DGS ( 51 )
22.79
0.7258
0.4024
257
12.84
4DGS ( 47 )
22.57
0.7323
0.3598
200
52.84
Table 4. Average dynamic Gaussian reconstruction results on M 3 ISR.
Figure 5. Qualitative comparison of Track 2 dynamic 4DGS reconstruction on M 3 ISR. Ground truth and results from Deformable 3DGS and 4DGS are shown for representative dynamic scenes.
Figure 6. The 3DGS compression performance of the FCGS method on our M 3 ISR benchmark.
Figure 7. The 3DGS compression performance of the HAC method on our M 3 ISR benchmark.
Figure 8. The 4DGS compression performance of the D-FCGS method on our M 3 ISR benchmark.
Feed-forward 3D Gaussian Splatting (3DGS) enables efficient novel-view synthesis from sparse multi-view images, yet its representations remain costly to store and transmit. Existing approaches compress either the input images, incurring heavy receiver-side reconstruction, or the reconstructed Gaussian primitives, which are difficult to compress due to their heterogeneous and irregular attributes. We instead compress compact intermediate features, providing a better balance between compression efficiency and receiver-side complexity. Based on this paradigm, we propose FeCoSplat, a feedback-guided compression framework for feed-forward 3DGS. FeCoSplat first compresses multi-view features to obtain an intermediate 3DGS, whose rendered views are used as feedback to guide a second-stage compression for further refinement. The resulting bitstreams are decoded into a compact implicit state, from which the final Gaussian primitives are reconstructed with a lightweight predictor. Experiments demonstrate that FeCoSplat achieves favorable rate--distortion performance, particularly at low bitrates, while requiring only 3.45M parameters for receiver-side Gaussian reconstruction. Code will be released soon.
Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally. We present \emph{StreamSplat}, a streaming feed-forward 3DGS framework that incrementally maintains a persistent geometry-grounded scene state and decodes it into renderable 3D Gaussians after each input chunk. StreamSplat centers on a \textbf{Voxel-Aligned Causal Cache (VACC)}, which stores historical 3D tokens in a memory-bounded voxel structure so that memory grows with explored scene geometry rather than stream length. To better reuse history during causal prediction, we introduce \textbf{History-Projected Depth Anchoring (HPDA)} to project cached geometry as depth guidance for current cost-volume estimation, and \textbf{Cache-Guided Feature Injection (CGFI)} to inject cached latent evidence into Gaussian-token regression. Experiments on DL3DV, RealEstate10K, and ScanNet show that StreamSplat remains competitive with state-of-the-art feed-forward 3DGS methods under sparse causal inputs, despite not using future views or full-scene context. More importantly, it scales to long input streams with 256, 512, and 1024 views where fixed-view baselines run out of memory, yielding sustained improvements in novel-view synthesis quality as more observations arrive. The code will be made publicly available upon acceptance.
We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scene reconstruction from only six cameras with wide baselines. To achieve robust dynamic reconstruction under such sparse views, our framework integrates three components. (1) Region-adaptive spatial priors: We use foreground masks to guide Gaussian initialization and mask voting to control densification separately for the dynamic foreground and static background. Background geometry is regularized using monocular depth aligned to metric scale. (2) Motion-consistent temporal priors: We provide supervision at intermediate times through frame interpolation and constrain projected Gaussian motion with estimated optical flow. (3) Generative assistance: We place virtual cameras in the widest angular gaps and restore their rendered images using a diffusion-based model conditioned on camera poses. The restored images are iteratively incorporated into training as pseudo-supervision. On the validation set, our framework improves full-frame PSNR from 25.60 dB for the baseline to 29.75 dB. On the official test benchmark, it achieves 30.04 dB full-frame PSNR and 27.88 dB foreground PSNR, ranking first overall in the Sparse-View Track.
Shengqi Wang, Zhengxian Yang, Kaiwen Tian +6
Department of Automation, Tsinghua University, Beijing, China · JD.com, Beijing, China · Beijing MEET YUAN Co., Ltd, Beijing, China