Feed-forward 3D Gaussian Splatting (3DGS) methods enable efficient reconstruction of 3D scenes from sparse multi-view images in a single forward pass, without scene-specific optimization. However, existing feed-forward scene-coding pipelines first generate a complete Gaussian representation and then encode its geometry and appearance attributes separately, often involving non-differentiable operations. The resulting 'generate-then-compress' pipeline requires complex attribute-specific codec designs and limits rate-distortion performance by restricting the exploitation of cross-attribute redundancy. We propose multi-view feature coding for Gaussian splatting (MFGS), an end-to-end feed-forward framework for scene coding. Instead of compressing an explicit Gaussian representation, MFGS extracts multi-view features from the input images and encodes them into a compact bitstream. Through end-to-end rate-distortion training, MFGS learns compact multi-view features that jointly represent geometry and appearance, enabling the receiver to predict Gaussian attributes from the decoded features. We further introduce importance-guided Gaussian selection to accommodate different rendering budgets, allowing the receiver to adjust the number of Gaussians used for rendering from the same bitstream. Across RE10K, ACID, and DL3DV, MFGS achieves 99.76-99.86% bitrate savings over the generate-then-compress pipeline, as measured by BD-rate, while reducing total GMACs by 52-55% and providing an approximately 12 times end-to-end speedup.
Figures & tables
Figure 1: Comparison of 3D scene coding pipelines. (a) Generate-then-compress predicts explicit Gaussians and encodes their attributes through specialized paths, limiting joint compression of geometry and appearance. Non-differentiable operations such as G-PCC also hinder end-to-end training. (b) The MFGS framework compresses multi-view features with an established learned image codec and predicts Gaussian attributes from the decoded features at the receiver. This unified path simplifies coding, exploits cross-attribute redundancy, and enables end-to-end rate–distortion training, while compact latent grids reduce computation.
Figure 2: Relative bitrate and complexity of MFGS-Base versus FCGS (generate-then-compress), using the same generator architecture ( Section 4.2 ). FCGS is normalized to 100% ; lower is better. (a) Bitrate at comparable quality; (b) runtime, (c) GMACs, and (d) peak GPU memory. Panels (b)–(d) distinguish encoding and decoding. Bars show means across datasets; whiskers span their minima and maxima.
Figure 3: Overview of the MFGS framework from multi-view feature extraction through Gaussian reconstruction. The sender extracts F from sparse input views and encodes it into a bitstream, while the receiver decodes F^ and predicts Gaussian primitives. The MFGS-Adaptive model additionally supports importance-guided bit allocation and receiver-side Gaussian selection. Green modules are specific to the MFGS-Adaptive model. AE and AD denote arithmetic encoding and decoding, respectively.
Figure 4: Rate–distortion curves for the MFGS-Base model, its frozen-generator variant, FCGS, and TMCV on RE10K, ACID, and DL3DV. The horizontal axis shows scene bitstream size on a logarithmic scale.
Method
RE10K
ACID
DL3DV
FCGS
0.00
0.00
0.00
TMCV
–
–
+14.59
MFGS-Base
−99.86
−99.80
−99.76
MFGS-Adaptive ( k=1.0 )
−99.84
−99.80
−99.77
Table 1: BD-rate (%) relative to FCGS (lower is better), using identical scene sets across all methods and rate points within each dataset ( Section A.3 ). The MFGS-Adaptive model is evaluated at the full Gaussian budget ( k=1 ).
Figure 5: Target-view reconstructions on ACID, RE10K, and DL3DV. FCGS and TMCV compress the output of the standalone MVSplat+ZPressor generator. Yellow boxes mark regions enlarged in the insets. Each panel reports target-view PSNR and the complete scene bitstream size.
Encoding
Decoding
Method
GMACs ↓
Peak (MiB) ↓
Params. (M)
Runtime (s) ↓
GMACs ↓
Peak (MiB) ↓
Params. (M)
Runtime (s) ↓
MVSplat+FCGS
1,601.30
8502.66
15.83
3.694
895.62
3971.57
2.19
2.683
MFGS-Base
868.89
3475.75
27.78
0.299
289.17
1279.27
13.75
0.192
Ratio
0.543×
0.409×
1.75×
0.081×
0.323×
0.322×
6.29×
0.072×
Table 2: Encoding and decoding complexity at 256×256 on RTX 3090 GPUs. Values are averaged across RE10K, ACID, and DL3DV. Measurements use 100 randomly sampled test scenes from each of RE10K and ACID and the full DL3DV test set. MVSplat+FCGS includes MVSplat+ZPressor generation. For MVSplat+FCGS, only runtime includes G-PCC execution; all other complexity metrics (GMACs, peak GPU memory, and parameters) exclude G-PCC. Parameters count learned weights only, in millions (M); G-PCC has no learned weights. Ratio denotes MFGS-Base divided by MVSplat+FCGS. Encoding and decoding exclude rendering. Operating points are λ=0.0256 for MFGS-Base and λ=0.0016 for MVSplat+FCGS.
Figure 6: Rate–distortion comparison of the MFGS-Base model and matched FCGS baselines on ACID with MonoSplat and TranSplat.
Figure 7: PSNR comparison between the MFGS-Adaptive model and post-hoc pruning baselines across Gaussian keep ratios on ACID and DL3DV at λ=0.0256 . LightGaussian and PUP-3DGS are applied to the Gaussians reconstructed by the MFGS-Base model.
Figure 8: Qualitative pruning comparison on DL3DV across Gaussian keep ratios. Rows show the MFGS-Adaptive model and post-hoc pruning baselines at λ=0.0256 , and columns show k decreasing from 1.0 to 0.2 . Each panel reports PSNR with respect to the ground-truth target image.
Figure 9: Scene-level Gaussian selection on ACID. The top row shows the ground-truth target image, reconstructions at k=0.8 and k=0.6 from the same bitstream, and their mean absolute RGB difference. The bottom row shows camera-aligned orthographic projections of excluded Gaussians, colored by decoder-side importance percentile within the full candidate set at each budget. Blue stars mark the reference input-camera center.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Block
Main operation
k/s
Channels
Output
Analysis ga
C + GDN
5/2
133→192
192×H/2×W/2
[ C + GDN] ( ×2 )
5/2
192→192
192×H/8×W/8
C
5/2
192→320
yv:320×H/16×W/16
Hyper-analysis ha
C + LReLU
3/1
320→192
192×H/16×W/16
C + LReLU
5/2
192→192
192×H/32×W/32
C
5/2
192→192
zv:192×H/64×W/64
Appendix
Table 3: Multi-view feature-codec architecture. C and TC denote convolution and transposed convolution, respectively; k/s gives the kernel size and stride. Latent tensor shapes are reported per ZPressor plane and omit the batch dimension.
average-pool Senc to the latent resolution elementwise modulation before quantization
gated y
Keep-ratio embedding
0<k≤1
sinusoidal encoding with 128 frequencies (256 dimensions) 256→256→256 MLP with SiLU FiLM conditioning of gs , ψ , and pdec
e(k):256
Decoder importance pdec
F^k , detached density and geometry descriptors, e(k)
two-layer convolutional trunk FiLM conditioning and Sigmoid head
Sdec(k):V×H×W
Gaussian selection
G^max,Sdec(k),k
rank the N candidates by Sdec(k) temperature-controlled top- K relaxation during training hard top- K selection at inference
G^ with K=⌊kN⌋ Gaussians
Appendix
Table 4: Additional modules used in the MFGS-Adaptive model. FiLM denotes feature-wise linear modulation.
(a) Training schedule
Stage
Configuration
Steps
Stage 1: Generator pretraining
Train the feed-forward Gaussian generator from a pretrained UniMatch initialization
100k
Stage 2: Distortion warm-up
Initialize from Stage 1; train the generator and multi-view feature codec for reconstruction without a rate penalty
50k
Stage 3: Joint R-D training
Initialize from Stage 2; jointly train the generator and multi-view feature codec for each λ
60k/rate
Stage 4: MFGS-Base
Initialize from the corresponding Stage 3 checkpoint, freeze only the multi-view feature-extraction backbone, and optimize all remaining modules
60k/rate
Stage 4: MFGS-Adaptive
Initialize independently from the same Stage 3 checkpoint, freeze only the multi-view feature-extraction backbone, and optimize the adaptive model with e(k) , latent gating, Gaussian selection, and Limp
60k/rate
Appendix
Table 5: Training schedule and hyperparameters for the MFGS-Base and MFGS-Adaptive models with MVSplat+ZPressor.
λ
PSNR, three seeds (dB)
PSNR mean ± SD
Size, three seeds (KiB)
Size mean ± SD
0.0256
28.32 / 28.28 / 28.26
28.29±0.03
63.24 / 68.45 / 68.79
66.83±3.11
0.1024
27.99 / 27.88 / 27.91
27.93±0.06
31.69 / 33.69 / 33.87
33.08±1.21
0.4096
27.23 / 26.99 / 27.06
27.09±0.12
14.15 / 15.46 / 15.54
15.05±0.78
1.6384
26.08 / 25.89 / 25.85
25.94±0.12
6.45 / 7.03 / 7.01
6.83±0.33
Appendix
Table 6: Random-seed robustness of the MFGS-Base model on RE10K. Each row reports PSNR and full transmitted bitstream size for three independent training runs, evaluated on all 7,067 scenes. SD denotes the sample standard deviation.
Figure 10: Encoder-side latent-gating ablation on ACID and DL3DV. The no-gating variant omits gating and the importance-alignment loss. Green shading and thin purple boundary curves indicate the ranges for the full and no-gating models, respectively. These ranges span k∈{0.2,0.4,0.6,0.8,1.0} ; lines connect rate settings at k=0.4 .
Figure 11: Importance-ranked allocation of estimated main-latent bits on ACID at k=0.8 . Each panel uses one rate weight λ . Curves show the mean percentage of estimated scene-level y bits in each of 100 bins ordered from low to high encoder- or decoder-side importance. Both rankings average all 1,810 scenes at every λ .
Phase
Component
GMACs
Params. (M)
MVSplat+ FCGS
MFGS-Base
MVSplat+ FCGS
MFGS-Base
Encoding
Multi-view feature extraction
712.42
712.42
12.47
12.47
Gaussian prediction head
170.51
0.00
0.83
0.00
Codec encoding
678.49
120.29
2.54
15.32
Total
1,561.41
832.71
15.83
27.78
Decoding
Entropy-model processing
729.19
5.87
1.78
8.14
Appendix
Table 7: Module-wise encoding and decoding complexity on RE10K at 256×256 . GMACs are estimated with fvcore and exclude G-PCC. Parameters count learned weights only, in millions (M); G-PCC has no learned weights. MVSplat+FCGS includes Gaussian generation. Entropy processing includes context networks and learned grid projections. Reconstruction includes gs and AuxT for MFGS-Base and attribute synthesis for MVSplat+FCGS.
Dataset
Multi-view images I
Pre-Gaussian multi-view features F
Latent y
Bitstream Bλ=0.0256
Bitstream Bλ=0.1024
Bitstream Bλ=0.4096
Bitstream Bλ=1.6384
Decoded Gaussians G^
RE10K
6,144 KiB (6.6%)
204,288 KiB (218.0%)
1,920 KiB (2.0%)
63.2 KiB (0.067%)
31.7 KiB (0.034%)
14.2 KiB (0.015%)
6.5 KiB (0.007%)
93,696 KiB (100%)
ACID
Same as above
Same as above
Same as above
71.9 KiB (0.077%)
37.1 KiB (0.040%)
15.8 KiB (0.017%)
6.6 KiB (0.007%)
Same as above
DL3DV
Same as above
Same as above
Same as above
86.1 KiB (0.092%)
43.9 KiB (0.047%)
20.5 KiB (0.022%)
8.2 KiB (0.009%)
Same as above
Appendix
Table 8: Reference representation sizes and full transmitted bitstream sizes for the MFGS-Base model. Raw tensor sizes assume FP32 storage; percentages are relative to the decoded Gaussian size ∣G^∣ . The raw image size corresponds to eight input views at 256×256 . F and y are internal tensors; only the complete bitstream B is transmitted.
(a) Rate–distortion comparison
Dataset
Common PSNR (dB)
BD-rate (%)
RE10K
26.08–27.15
+440.92
ACID
26.67–28.92
+105.24
DL3DV
20.22–21.55
−8.84
Appendix
Table 9: Comparison with MIV transmission on RE10K, ACID, and DL3DV. BD-rate is computed for MIV relative to MFGS-Base; positive values indicate lower rates for MFGS-Base. Speedup is reported for MFGS-Base relative to MIV.
While feed-forward 3D Gaussian splatting reconstructs renderable Gaussian primitives from sparse context views without per-scene optimization, existing pipelines do not provide a compact scene representation for storage or transmission. A natural solution is to apply existing 3DGS compression methods to the generated Gaussian primitives. However, this approach operates on the final irregular 3D representation and is decoupled from the internal feature-to-Gaussian generation process, which limits compression efficiency. To address this, we introduce CodecSplat, an ultra-compact latent coding framework for feed-forward 3D Gaussian splatting. CodecSplat first encodes an intermediate 2D Gaussian-generation feature into an entropy-coded scene bitstream. At the decoder, the latent feature is reconstructed and used to predict depth and Gaussian parameters, which are then mapped to 3D Gaussian primitives. By integrating compression into the feed-forward Gaussian generation pipeline, CodecSplat avoids inefficient compression over irregular 3D Gaussian primitives and allows the codec to exploit the structured intermediate feature representation. We instantiate CodecSplat on a feed-forward Gaussian splatting backbone with depth-guided multi-view feature refinement and a hierarchical learned feature codec. On DL3DV and RealEstate10K datasets, CodecSplat achieves 23.56-26.36 dB and 24.76-27.05 dB PSNR with only 20.00-107.77 KiB and 3.37-12.51 KiB per scene, respectively. This is roughly one order of magnitude smaller than compressing feed-forward generated Gaussian primitives, while preserving controllable rate-distortion behavior. Code is available at https://github.com/pengpeng-yu/CodecSplat.
Pengpeng Yu, Runqing Jiang, Qi Zhang +3
1Sun Yat-sen University, China · 3Pengcheng Laboratory, China · 2Peking University, China
Feed-forward 3D Gaussian Splatting (3DGS) enables efficient novel-view synthesis from sparse multi-view images, yet its representations remain costly to store and transmit. Existing approaches compress either the input images, incurring heavy receiver-side reconstruction, or the reconstructed Gaussian primitives, which are difficult to compress due to their heterogeneous and irregular attributes. We instead compress compact intermediate features, providing a better balance between compression efficiency and receiver-side complexity. Based on this paradigm, we propose FeCoSplat, a feedback-guided compression framework for feed-forward 3DGS. FeCoSplat first compresses multi-view features to obtain an intermediate 3DGS, whose rendered views are used as feedback to guide a second-stage compression for further refinement. The resulting bitstreams are decoded into a compact implicit state, from which the final Gaussian primitives are reconstructed with a lightweight predictor. Experiments demonstrate that FeCoSplat achieves favorable rate--distortion performance, particularly at low bitrates, while requiring only 3.45M parameters for receiver-side Gaussian reconstruction. Code will be released soon.
Feed-forward 3D Gaussian Splatting (3DGS) enables scalable scene reconstruction without per-scene optimization, yet produces dense Gaussians that are costly to store and transmit. Existing feed-forward Gaussian compression methods formulate decoding as deterministic representation recovery, which becomes inadequate at low bitrates when high-frequency textures and view-dependent appearance are discarded. Although generative models offer a promising alternative, using them as standalone post-processing decouples generation from the transmitted scene structure, thereby compromising cross-view consistency. To address these limitations, we propose GenSplatCodec, a unified feed-forward Gaussian codec that reformulates low-bitrate Gaussian compression as geometry-guided generative decoding. We present a detail-aware feed-forward Gaussian coding scheme within a dual-stream formulation, where the resulting compact Gaussian structural stream is complemented by a lightweight reference appearance stream. We further introduce a geometry-guided one-step generative decoding approach that jointly exploits decoded structural and appearance cues through hierarchical geometry control to reconstruct high-fidelity and view-consistent novel views. Finally, we develop a three-stage optimization strategy that stabilizes the learning of the unified codec and adapts the generative decoder to codec-derived structural and appearance cues. Extensive experiments across multiple datasets demonstrate that GenSplatCodec consistently achieves superior rate-distortion (RD) performance over existing methods.
Qiang Hu, Zhenlong Wu, Lei Huang +3
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai, 200240, China