Feed-Forward Multi-View Feature Coding for 3D Gaussian Splatting

Seungmi Choi; Ji-Hyoun Choi; Suhyun Jo; Tae-Gwon Park; Hui Yong Kim

cs.CVOct 14, 2026

Feed-Forward Multi-View Feature Coding for 3D Gaussian Splatting

Authors: Seungmi Choi, Ji-Hyoun Choi, Suhyun Jo, Tae-Gwon Park, Hui Yong Kim

Organizations: Kyung Hee University

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) methods enable efficient reconstruction of 3D scenes from sparse multi-view images in a single forward pass, without scene-specific optimization. However, existing feed-forward scene-coding pipelines first generate a complete Gaussian representation and then encode its geometry and appearance attributes separately, often involving non-differentiable operations. The resulting 'generate-then-compress' pipeline requires complex attribute-specific codec designs and limits rate-distortion performance by restricting the exploitation of cross-attribute redundancy. We propose multi-view feature coding for Gaussian splatting (MFGS), an end-to-end feed-forward framework for scene coding. Instead of compressing an explicit Gaussian representation, MFGS extracts multi-view features from the input images and encodes them into a compact bitstream. Through end-to-end rate-distortion training, MFGS learns compact multi-view features that jointly represent geometry and appearance, enabling the receiver to predict Gaussian attributes from the decoded features. We further introduce importance-guided Gaussian selection to accommodate different rendering budgets, allowing the receiver to adjust the number of Gaussians used for rendering from the same bitstream. Across RE10K, ACID, and DL3DV, MFGS achieves 99.76-99.86% bitrate savings over the generate-then-compress pipeline, as measured by BD-rate, while reducing total GMACs by 52-55% and providing an approximately 12 times end-to-end speedup.

Figures & tables

Figure 1: Comparison of 3D scene coding pipelines. (a) Generate-then-compress predicts explicit Gaussians and encodes their attributes through specialized paths, limiting joint compression of geometry and appearance. Non-differentiable operations such as G-PCC also hinder end-to-end training. (b) The MFGS framework compresses multi-view features with an established learned image codec and predicts Gaussian attributes from the decoded features at the receiver. This unified path simplifies coding, exploits cross-attribute redundancy, and enables end-to-end rate–distortion training, while compact latent grids reduce computation.

Figure 2: Relative bitrate and complexity of MFGS-Base versus FCGS (generate-then-compress), using the same generator architecture ( Section 4.2 ). FCGS is normalized to $100\%$ ; lower is better. (a) Bitrate at comparable quality; (b) runtime, (c) GMACs, and (d) peak GPU memory. Panels (b)–(d) distinguish encoding and decoding. Bars show means across datasets; whiskers span their minima and maxima.

Figure 3: Overview of the MFGS framework from multi-view feature extraction through Gaussian reconstruction. The sender extracts $F$ from sparse input views and encodes it into a bitstream, while the receiver decodes $\hat{F}$ and predicts Gaussian primitives. The MFGS-Adaptive model additionally supports importance-guided bit allocation and receiver-side Gaussian selection. Green modules are specific to the MFGS-Adaptive model. AE and AD denote arithmetic encoding and decoding, respectively.

Figure 4: Rate–distortion curves for the MFGS-Base model, its frozen-generator variant, FCGS, and TMCV on RE10K, ACID, and DL3DV. The horizontal axis shows scene bitstream size on a logarithmic scale.

Method	RE10K	ACID	DL3DV
FCGS	$0.00$	$0.00$	$0.00$
TMCV	–	–	$+14.59$
MFGS-Base	$-99.86$	$-99.80$	$-99.76$
MFGS-Adaptive ( $k=1.0$ )	$-99.84$	$-99.80$	$-99.77$

Table 1: BD-rate (%) relative to FCGS (lower is better), using identical scene sets across all methods and rate points within each dataset ( Section A.3 ). The MFGS-Adaptive model is evaluated at the full Gaussian budget (

k=1

Figure 5: Target-view reconstructions on ACID, RE10K, and DL3DV. FCGS and TMCV compress the output of the standalone MVSplat+ZPressor generator. Yellow boxes mark regions enlarged in the insets. Each panel reports target-view PSNR and the complete scene bitstream size.

	Encoding				Decoding
Method	GMACs $\downarrow$	Peak (MiB) $\downarrow$	Params. (M)	Runtime (s) $\downarrow$	GMACs $\downarrow$	Peak (MiB) $\downarrow$	Params. (M)	Runtime (s) $\downarrow$
MVSplat+FCGS	1,601.30	8502.66	15.83	3.694	895.62	3971.57	2.19	2.683
MFGS-Base	868.89	3475.75	27.78	0.299	289.17	1279.27	13.75	0.192
Ratio	$0.543\times$	$0.409\times$	$1.75\times$	$0.081\times$	$0.323\times$	$0.322\times$	$6.29\times$	$0.072\times$

Table 2: Encoding and decoding complexity at

256\!\times\!256

on RTX 3090 GPUs. Values are averaged across RE10K, ACID, and DL3DV. Measurements use 100 randomly sampled test scenes from each of RE10K and ACID and the full DL3DV test set. MVSplat+FCGS includes MVSplat+ZPressor generation. For MVSplat+FCGS, only runtime includes G-PCC execution; all other complexity metrics (GMACs, peak GPU memory, and parameters) exclude G-PCC. Parameters count learned weights only, in millions (M); G-PCC has no learned weights. Ratio denotes MFGS-Base divided by MVSplat+FCGS. Encoding and decoding exclude rendering. Operating points are

\lambda=0.0256

for MFGS-Base and

\lambda=0.0016

for MVSplat+FCGS.

Figure 6: Rate–distortion comparison of the MFGS-Base model and matched FCGS baselines on ACID with MonoSplat and TranSplat.

Figure 7: PSNR comparison between the MFGS-Adaptive model and post-hoc pruning baselines across Gaussian keep ratios on ACID and DL3DV at $\lambda{=}0.0256$ . LightGaussian and PUP-3DGS are applied to the Gaussians reconstructed by the MFGS-Base model.

Figure 8: Qualitative pruning comparison on DL3DV across Gaussian keep ratios. Rows show the MFGS-Adaptive model and post-hoc pruning baselines at $\lambda=0.0256$ , and columns show $k$ decreasing from $1.0$ to $0.2$ . Each panel reports PSNR with respect to the ground-truth target image.

Figure 9: Scene-level Gaussian selection on ACID. The top row shows the ground-truth target image, reconstructions at $k{=}0.8$ and $k{=}0.6$ from the same bitstream, and their mean absolute RGB difference. The bottom row shows camera-aligned orthographic projections of excluded Gaussians, colored by decoder-side importance percentile within the full candidate set at each budget. Blue stars mark the reference input-camera center.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Block	Main operation	$k/s$	Channels	Output
Analysis $g_{a}$	$\mathrm{C}$ + GDN	$5/2$	$133{\to}192$	$192{\times}H/2{\times}W/2$
	[ $\mathrm{C}$ + GDN] ( $\times 2$ )	$5/2$	$192{\to}192$	$192{\times}H/8{\times}W/8$
	$\mathrm{C}$	$5/2$	$192{\to}320$	$y_{v}:\;320{\times}H/16{\times}W/16$
Hyper-analysis $h_{a}$	$\mathrm{C}$ + LReLU	$3/1$	$320{\to}192$	$192{\times}H/16{\times}W/16$
	$\mathrm{C}$ + LReLU	$5/2$	$192{\to}192$	$192{\times}H/32{\times}W/32$
	$\mathrm{C}$	$5/2$	$192{\to}192$	$z_{v}:\;192{\times}H/64{\times}W/64$

Appendix

Table 3: Multi-view feature-codec architecture.

\mathrm{C}

and

\mathrm{TC}

denote convolution and transposed convolution, respectively;

k/s

gives the kernel size and stride. Latent tensor shapes are reported per ZPressor plane and omit the batch dimension.

Module	Input	Main operations	Output
Encoder importance $p_{\mathrm{enc}}$	$F$ and a detached copy of the first RGB channel	$\mathrm{C}_{3/1}(134,64)$ + ReLU $\mathrm{C}_{3/1}(64,32)$ + ReLU $\mathrm{C}_{1/1}(32,1)$ + Sigmoid	$S_{\mathrm{enc}}:\;V{\times}H{\times}W$
Latent gating	$y,S_{\mathrm{enc}}$	average-pool $S_{\mathrm{enc}}$ to the latent resolution elementwise modulation before quantization	gated $y$
Keep-ratio embedding	$0<k\leq 1$	sinusoidal encoding with 128 frequencies (256 dimensions) $256{\to}256{\to}256$ MLP with SiLU FiLM conditioning of $g_{s}$ , $\psi$ , and $p_{\mathrm{dec}}$	$e(k):\;256$
Decoder importance $p_{\mathrm{dec}}$	$\hat{F}_{k}$ , detached density and geometry descriptors, $e(k)$	two-layer convolutional trunk FiLM conditioning and Sigmoid head	$S_{\mathrm{dec}}(k):\;V{\times}H{\times}W$
Gaussian selection	$\hat{G}_{\max},S_{\mathrm{dec}}(k),k$	rank the $N$ candidates by $S_{\mathrm{dec}}(k)$ temperature-controlled top- $K$ relaxation during training hard top- $K$ selection at inference	$\hat{G}$ with $K{=}\lfloor kN\rfloor$ Gaussians

Appendix

Table 4: Additional modules used in the MFGS-Adaptive model. FiLM denotes feature-wise linear modulation.

(a) Training schedule
Stage	Configuration	Steps
Stage 1: Generator pretraining	Train the feed-forward Gaussian generator from a pretrained UniMatch initialization	100k
Stage 2: Distortion warm-up	Initialize from Stage 1; train the generator and multi-view feature codec for reconstruction without a rate penalty	50k
Stage 3: Joint R-D training	Initialize from Stage 2; jointly train the generator and multi-view feature codec for each $\lambda$	60k/rate
Stage 4: MFGS-Base	Initialize from the corresponding Stage 3 checkpoint, freeze only the multi-view feature-extraction backbone, and optimize all remaining modules	60k/rate
Stage 4: MFGS-Adaptive	Initialize independently from the same Stage 3 checkpoint, freeze only the multi-view feature-extraction backbone, and optimize the adaptive model with $e(k)$ , latent gating, Gaussian selection, and $\mathcal{L}_{\mathrm{imp}}$	60k/rate

Appendix

Table 5: Training schedule and hyperparameters for the MFGS-Base and MFGS-Adaptive models with MVSplat+ZPressor.

$\lambda$	PSNR, three seeds (dB)	PSNR mean $\pm$ SD	Size, three seeds (KiB)	Size mean $\pm$ SD
0.0256	28.32 / 28.28 / 28.26	$28.29\pm 0.03$	63.24 / 68.45 / 68.79	$66.83\pm 3.11$
0.1024	27.99 / 27.88 / 27.91	$27.93\pm 0.06$	31.69 / 33.69 / 33.87	$33.08\pm 1.21$
0.4096	27.23 / 26.99 / 27.06	$27.09\pm 0.12$	14.15 / 15.46 / 15.54	$15.05\pm 0.78$
1.6384	26.08 / 25.89 / 25.85	$25.94\pm 0.12$	6.45 / 7.03 / 7.01	$6.83\pm 0.33$

Appendix

Table 6: Random-seed robustness of the MFGS-Base model on RE10K. Each row reports PSNR and full transmitted bitstream size for three independent training runs, evaluated on all 7,067 scenes. SD denotes the sample standard deviation.

Figure 10: Encoder-side latent-gating ablation on ACID and DL3DV. The no-gating variant omits gating and the importance-alignment loss. Green shading and thin purple boundary curves indicate the ranges for the full and no-gating models, respectively. These ranges span $k\in\{0.2,0.4,0.6,0.8,1.0\}$ ; lines connect rate settings at $k=0.4$ .

Figure 11: Importance-ranked allocation of estimated main-latent bits on ACID at

k=0.8

. Each panel uses one rate weight

\lambda

. Curves show the mean percentage of estimated scene-level

y

bits in each of 100 bins ordered from low to high encoder- or decoder-side importance. Both rankings average all 1,810 scenes at every

\lambda

Phase	Component	GMACs		Params. (M)
		MVSplat+ FCGS	MFGS-Base	MVSplat+ FCGS	MFGS-Base
Encoding	Multi-view feature extraction	712.42	712.42	12.47	12.47
	Gaussian prediction head	170.51	0.00	0.83	0.00
	Codec encoding	678.49	120.29	2.54	15.32
	Total	1,561.41	832.71	15.83	27.78
Decoding	Entropy-model processing	729.19	5.87	1.78	8.14

Appendix

Table 7: Module-wise encoding and decoding complexity on RE10K at

256\!\times\!256

. GMACs are estimated with fvcore and exclude G-PCC. Parameters count learned weights only, in millions (M); G-PCC has no learned weights. MVSplat+FCGS includes Gaussian generation. Entropy processing includes context networks and learned grid projections. Reconstruction includes

g_{s}

and AuxT for MFGS-Base and attribute synthesis for MVSplat+FCGS.

Dataset	Multi-view images $I$	Pre-Gaussian multi-view features $F$	Latent $y$	Bitstream $B$ $\lambda{=}0.0256$	Bitstream $B$ $\lambda{=}0.1024$	Bitstream $B$ $\lambda{=}0.4096$	Bitstream $B$ $\lambda{=}1.6384$	Decoded Gaussians $\hat{G}$
RE10K	6,144 KiB (6.6%)	204,288 KiB (218.0%)	1,920 KiB (2.0%)	63.2 KiB (0.067%)	31.7 KiB (0.034%)	14.2 KiB (0.015%)	6.5 KiB (0.007%)	93,696 KiB (100%)
ACID	Same as above	Same as above	Same as above	71.9 KiB (0.077%)	37.1 KiB (0.040%)	15.8 KiB (0.017%)	6.6 KiB (0.007%)	Same as above
DL3DV	Same as above	Same as above	Same as above	86.1 KiB (0.092%)	43.9 KiB (0.047%)	20.5 KiB (0.022%)	8.2 KiB (0.009%)	Same as above

Appendix

Table 8: Reference representation sizes and full transmitted bitstream sizes for the MFGS-Base model. Raw tensor sizes assume FP32 storage; percentages are relative to the decoded Gaussian size

|\hat{G}|

. The raw image size corresponds to eight input views at

256\!\times\!256

F

and

y

are internal tensors; only the complete bitstream

B

is transmitted.

(a) Rate–distortion comparison
Dataset	Common PSNR (dB)	BD-rate (%)
RE10K	26.08–27.15	$+440.92$
ACID	26.67–28.92	$+105.24$
DL3DV	20.22–21.55	$-8.84$

Appendix

Table 9: Comparison with MIV transmission on RE10K, ACID, and DL3DV. BD-rate is computed for MIV relative to MFGS-Base; positive values indicate lower rates for MFGS-Base. Speedup is reported for MFGS-Base relative to MIV.

3DGS Compression

3D Gaussian Splatting compression reduces storage and rendering costs while preserving the quality of reconstructed views and scenes.

See all 85 papers

CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting
May 25, 2026Pengpeng Yu, Runqing Jiang, Qi Zhang +33DGS Compression 3D Gaussian Splatting
1Sun Yat-sen University, China · 3Pengcheng Laboratory, China · 2Peking University, China
FeCoSplat: Feedback-Guided Compression for Feed-Forward 3D Gaussian Splatting
Sep 27, 2026Yuxuan Li, Yihang Chen, Yufeng Zhang +23DGS Compression 3D Gaussian Splatting
Shanghai Jiao Tong University · Monash University
GenSplatCodec: Feed-Forward Gaussian Splatting Compression via One-Step Diffusion
Jul 27, 2026Qiang Hu, Zhenlong Wu, Lei Huang +33DGS Compression 3D Gaussian Splatting
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai, 200240, China

May 25, 2026J/K move · Enter open · S save

CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting

Pengpeng Yu, Runqing Jiang, Qi Zhang, Dingquan Li+2

1Sun Yat-sen University, China · 3Pengcheng Laboratory, China · 2Peking University, China

3DGS Compression 3D Gaussian Splatting Representation Learning Feed-Forward 3D Reconstruction Gaussian Splatting Compression

Open paper AskPDF arXiv

While feed-forward 3D Gaussian splatting reconstructs renderable Gaussian primitives from sparse context views without per-scene optimization, existing pipelines do not provide a compact scene representation for storage or transmission. A natural solution is to apply existing 3DGS compression methods to the generated Gaussian primitives. However, this approach operates on the final irregular 3D representation and is decoupled from the internal feature-to-Gaussian generation process, which limits compression efficiency. To address this, we introduce CodecSplat, an ultra-compact latent coding framework for feed-forward 3D Gaussian splatting. CodecSplat first encodes an intermediate 2D Gaussian-generation feature into an entropy-coded scene bitstream. At the decoder, the latent feature is reconstructed and used to predict depth and Gaussian parameters, which are then mapped to 3D Gaussian primitives. By integrating compression into the feed-forward Gaussian generation pipeline, CodecSplat avoids inefficient compression over irregular 3D Gaussian primitives and allows the codec to exploit the structured intermediate feature representation. We instantiate CodecSplat on a feed-forward Gaussian splatting backbone with depth-guided multi-view feature refinement and a hierarchical learned feature codec. On DL3DV and RealEstate10K datasets, CodecSplat achieves 23.56-26.36 dB and 24.76-27.05 dB PSNR with only 20.00-107.77 KiB and 3.37-12.51 KiB per scene, respectively. This is roughly one order of magnitude smaller than compressing feed-forward generated Gaussian primitives, while preserving controllable rate-distortion behavior. Code is available at https://github.com/pengpeng-yu/CodecSplat.

Abstract

3DGS Compression

Explore similar work

CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting

FeCoSplat: Feedback-Guided Compression for Feed-Forward 3D Gaussian Splatting

GenSplatCodec: Feed-Forward Gaussian Splatting Compression via One-Step Diffusion