Only What Was Seen: Observation-Gram Compaction of View-Dependent Appearance in 3D Gaussian Splatting

Krzysztof Pietroszek

cs.CVSep 24, 2026

Only What Was Seen: Observation-Gram Compaction of View-Dependent Appearance in 3D Gaussian Splatting

Authors: Krzysztof Pietroszek

Organizations: Moholo Inc.

Abstract

Most of the memory of a 3D Gaussian Splatting model holds spherical-harmonic colour coefficients, yet each Gaussian is seen only from the narrow cone of directions of the training cameras. We turn this into a distortion metric that other compressors can adopt: a per-Gaussian observation Gram matrix, accumulated from viewing directions and blending weights, is the exact first-order map from coefficient changes to squared image error and needs only the model and the camera poses. Under it, degree reduction becomes a closed-form projection that generalises truncation, degree allocation a Lagrangian rate-distortion problem, and vector quantisation the matrix-weighted Lloyd algorithm, of which Compressed3D's quantiser is the scalar case. Swapped into Compressed3D with everything else unchanged, the metric raises PSNR by +0.49 dB before fine-tuning, with SSIM and LPIPS following, and at matched rate still gains +0.32 dB without a single training image. A training-free stack built on the metric alone is 15% smaller than the image-free GSICO at equal quality on Mip-NeRF 360.

Figures & tables

Figure 1 : A trained Gaussian is only constrained where the cameras saw it. Colour function of one Gaussian of the pretrained garden model on the sphere of viewing directions (top, Mollweide projection; outside the white outline no training view observed it, shaded light) and zoomed on the observed cone (bottom; dots: the training views in which it contributed, size $\propto$ blending-weight statistic). Truncation to degree 1 (middle) discards the bands that carry the colour gradient running through the observed cone. The closed-form observation-Gram projection to the same degree (right, Eq. 8 ) reproduces the trained function where it was observed and is free elsewhere; its predicted image error is $46\times$ lower at identical storage.

Host: Compressed3D [ 31 ] , colour VQ metric	$n$	Size (MB)	PSNR	SSIM	LPIPS	time (s)
published: their sensitivity, their VQ	9	28.8	26.98	0.801	0.238	–
before fine-tuning
reproduced: their sensitivity, their VQ	9	28.7	26.30	0.782	0.258	177
their sensitivity $s_{i}\mathbf{I}$ in generalised Lloyd	9	29.4 (103%)	26.51 (+0.21)	0.788 (+0.006)	0.251 (-0.007)	128
our weight $\tr(\mathbf{A}_{i})\mathbf{I}/16$ in generalised Lloyd	9	29.4 (102%)	26.52 (+0.22)	0.789 (+0.007)	0.251 (-0.007)	128
our matrix $\mathbf{A}_{i}$ in generalised Lloyd	9	30.3 (106%)	26.79 (+0.49)	0.799 (+0.017)	0.238 (-0.019)	127

Table 1: The observation-Gram metric as a drop-in inside a published pipeline (Mip-NeRF 360). Compressed3D is run unmodified (reproduced row) and with only its colour vector-quantisation metric replaced: the same keep mask, pruning, 4096-entry codebook, covariance quantisation, fine-tuning schedule and container. Every variant row is a paired comparison against the reproduction on the

n

scenes that have both; parentheses give the paired difference and the size ratio. Rows isolate the two claims: their gradient-based scalar sensitivity versus our image-free weight (scalar rows), and scalar versus full matrix; the first variant row is the algorithm control (their weights in our Lloyd iterations). Before fine-tuning the difference is what the metric alone determines, after fine-tuning what survives their 5000 image-based iterations. Time: the appearance metric plus the colour VQ on one RTX 4090, i.e. their backward pass over the training images plus their minibatch

k

-means, or our forward pass over the training cameras (poses only, PyTorch rasteriser) plus the generalised Lloyd iterations. The published row is the dataset mean reported in the 3DGS.zip survey.

Reduced3DGS criterion [ 32 ] , post-hoc	AC floats	same degrees		same floats
threshold $\tau$	per Gaussian	truncation (theirs)	projection (ours)	their criterion	our allocation
0.08	23.4	26.87	27.21	26.87	27.28 (23.4)
0.12	14.7	26.32	27.00	26.32	27.28 (14.7)
0.16	8.9	25.82	26.72	25.82	27.25 (8.9)
0.24	3.2	25.11	26.16	25.11	27.05 (3.2)

Table 2: Host B: the SH band-selection criterion of Reduced3DGS (demote a Gaussian when the maximum colour deviation of truncation over the observed views is below

\tau

), applied post-hoc to the official models (test PSNR, mean over 9 Mip-NeRF 360 scenes). Same degrees : the degrees it selects, with the bands truncated (theirs) or projected under the Gram metric (ours). Same floats : its criterion against our Lagrangian allocation at the same average number of AC floats (achieved value in parentheses).

	deg. 3	degree 2 (24)		degree 1 (9)		degree 0 (0)
Dataset	full	trunc.	ours	trunc.	ours	trunc.	ours
Mip-NeRF 360 (9)	27.29	26.16	27.15	25.08	26.71	24.22	25.07
Tanks&Temples (2)	23.39	22.65	23.34	21.80	23.05	21.22	21.77
Deep Blending (2)	29.53	29.17	29.47	28.38	29.42	27.81	28.38

Table 3: Uniform SH degree reduction of the official pretrained 3DGS models: test PSNR averaged over each dataset (number of scenes in parentheses; AC floats per Gaussian in the header). Truncation drops the higher bands; OGC replaces them by the closed-form projection of Eq. 8 . SSIM and LPIPS, which follow the same ordering, and per-scene numbers are in the supplement.

Method	class	Size (MB)	PSNR	SSIM	LPIPS
3DGS-30K (fp32) [ 19 , 2 ]	reference	734.0	27.21	0.815	0.214
HEMGS high-rate [ 24 ]	retrained backbone	21.0	27.93	0.813	0.230
HAC++ high-rate [ 6 ]	retrained backbone	19.4	27.82	0.811	0.231
HAC high-rate [ 5 ]	retrained backbone	22.9	27.77	0.811	0.230
HEMGS low-rate [ 24 ]	retrained backbone	12.5	27.75	0.806	0.248
ContextGS high-rate [ 40 ]	retrained backbone	19.3	27.75	0.811	0.231

Table 4: Mip-NeRF 360, grouped by what a method needs. Published rows are dataset means from the 3DGS.zip survey [ 2 ] (retrieved 2026-09-16) unless noted; our rows are means over 9 scenes, sizes are the encoded bytes of the training-free stack of Sec. 3.7 with the standard geometry stage, and metrics are rendered from the decoded values. Within-class comparisons are against the post-hoc rows; the retrained-backbone and trained rows are cross-class references. a : MesonGS is post-hoc with an image-free importance score; the row without fine-tuning is the survey’s re-evaluation of its c3 configuration, the fine-tuned row is from the ablation table of its paper (its main table uses a different protocol). b : FCGS is a feed-forward compressor with a learned prior, applied post-hoc without per-scene optimisation. c : GSICO is post-hoc, uses no training images and no fine-tuning; its published high-rate operating point takes a retrained Scaffold-GS model as input (its Table III), so only its 3DGS-input row is in our class. d : numbers from the FlexGaussian paper (training-free).

Configuration	Size (MB)	PSNR	SSIM	LPIPS
Mip-NeRF 360 (mean over 9 scenes)
pretrained 3DGS, fp32	793.5	27.29	0.812	0.218
OGC pipeline, 50% pruning ( Tab. 4 )	59.6	26.81	0.802	0.231
+ geometry stage, 50% pruning	23.6	26.71	0.796	0.234
+ appearance refit with images ( Sec. 3.6 )	23.3	26.97	0.795	0.235
+ geometry stage, 60% pruning	19.0	26.31	0.790	0.240

Table 5: A standard geometry stage stacked on our appearance pipeline on all three datasets (published rows: 3DGS.zip survey, retrieved 2026-09-16, dataset means; classes as in Tab. 4 ), and our pipeline applied to models whose primitive count was already reduced during training. The geometry stage is not a contribution of this paper: 16-bit fixed-point positions (Morton-sorted and delta-coded, as in [ 29 ] ), 8-bit uniform quantisation of scales, rotations, opacities and DC colours (as in [ 31 , 32 ] ) and zlib coding of every stream, applied without images or fine-tuning. Sizes are the encoded bytes; metrics are rendered from the decoded values. Bit-depth variants are in the supplement.

Figure 4 : Rate–distortion plane on the three datasets (dataset means). Grey: every point of the sweep over pruning fraction, SH budget and codebook size of our training-free stack; blue: its Pareto front. Published methods are plotted by class (marker), from the 3DGS.zip survey and the GSICO paper; the sizes of the trained and retrained-backbone methods are not achievable without the images and are shown as context.

Dataset	anchor	BD-rate / $\Delta$ size	BD-PSNR / $\Delta$ PSNR
Mip-NeRF 360	GSICO (3DGS input) ∗	$\Delta$ size -15%	+0.09
Mip-NeRF 360	FCGS	–	–
Mip-NeRF 360	HAC	–	-1.43
Tanks&Temples	GSICO (3DGS input) ∗	–	-0.36
Tanks&Temples	FCGS	–	–
Tanks&Temples	HAC ∗	–	-1.44

Table 6: Bjøntegaard metrics of the Pareto front of our training-free stack (sweep over pruning, SH budget and codebook size; dataset means) against published anchors with two operating points (piecewise-cubic on log-rate over the overlapping PSNR range). Negative BD-rate: ours needs fewer bytes at equal PSNR. ∗ Anchor with a single published point, compared at that point: our size at its PSNR and our PSNR at its size. –: the curves do not overlap in rate or quality. GSICO and FCGS are in our class (post-hoc, no images); HAC retrains its backbone.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

garden
layout	AC floats	SH data	SH evaluation (ms)		frame
	per Gaussian	(MB, fp16)	padded	branching	(s)
degree 3	45.0	560	76.0	80.5	8.9
allocation, 9	9.0	140	77.5	17.3	5.9
allocation, 6	6.0	105	75.9	11.8	6.4
allocation, 3	3.0	70	75.5	8.5	8.2

Appendix

Table 7: Rendering cost as a function of the SH layout in our PyTorch rasteriser (Apple M1 Max, one test view, median of repeated runs). SH evaluation: the colour stage with a zero-padded degree-3 kernel (no benefit) and with a kernel that branches on the per-Gaussian degree tag. Frame: complete frame including sorting and compositing, which dominates this implementation; its variation across rows is measurement noise.

trainable SH	params/Gauss.	params+Adam (MB)	GPU working set (GB)	iteration (s)
degree 3 (45 AC)	59	1033	8.8	9.19
degree 1 (9 AC)	23	403	4.5	8.79
degree 0 (none)	14	245	4.5	8.83

Appendix

Table 8: Cost of one training iteration (differentiable render, loss, backward, Adam step) in a pure PyTorch trainer on 1.46M Gaussians at 648

\times

420 (Apple M1 Max, unified memory), as a function of the number of trainable SH coefficients. OGC itself is post-hoc and does not change training; this bounds what a training-time variant could gain: little time, substantial memory.

weight $\omega_{ij}$	deg. 0	deg. 1	$\lambda$	deg. 0	deg. 1
truncation	24.56	25.83	$10^{-6}$	25.60	28.13
$S^{(2)}$ (ours)	25.60	28.09	$10^{-4}$	25.60	28.12
$S^{(1)}$	25.55	28.05	$10^{-3}$	25.60	28.09
pixel count	25.29	27.85	$10^{-2}$	25.60	27.98
hit (0/1)	24.99	27.56	$10^{-1}$	25.58	27.69
			$1$	25.29	27.15

Appendix

Table 9: Ablations (test PSNR, mean of garden and kitchen ). Left: choice of the per-view weight in the Gram matrix (

\lambda=10^{-3}

). Right: regularisation strength with

S^{(2)}

weights;

\lambda\to\infty

is truncation.

pruned	method	PSNR before	PSNR after	SSIM after	time (s)
0%	Adam, 1500 iterations	27.32	27.39	0.867	111
50%	Adam, 1500 iterations	26.96	27.26	0.864	65
50%	Adam, 2500 iterations	26.96	27.24	0.864	108
50%	Adam, 2500 iterations (MSE)	26.97	27.36	0.862	102
50%	Adam, 6000 iterations	26.96	27.26	0.864	256
50%	PCG, 12 $\times$ 3 iterations	26.97	27.48	0.862	184

Appendix

Table 10: Appearance refit on the official pretrained garden model (5.84M Gaussians, 27.32 dB uncompressed by the official rasteriser) after contribution pruning (no compaction), test split. The closed-form refit ( Sec. 3.6 ) against fine-tuning the SH coefficients with Adam at the official learning rates (L1 + D-SSIM loss unless marked MSE); times on one A100. Twelve-by-three PCG iterations add at most a few hundredths of a decibel over eight-by-two at twice the time. Adam with the official L1 + D-SSIM loss reaches a slightly higher SSIM, the metric it optimises.

Configuration	zlib (MB)	rANS (MB)	PSNR	SSIM	LPIPS
garden
appearance pipeline only, 50% pruning (fp16 container)	103.3	–	26.57	0.846	0.129
+ geometry 16/8 bit, no pruning	87.9	79.9	26.86	0.853	0.123
+ geometry 16/8 bit	45.7	41.7	26.51	0.844	0.130
+ geometry 16/6 bit	38.0	33.7	26.43	0.839	0.135
+ geometry 14/8 bit	42.6	39.2	26.10	0.821	0.145

Appendix

Table 11: Bit-depth variants of the geometry stage of Tab. 5 (position bits / attribute bits) on garden and kitchen , with the size under both entropy coders (byte-planed zlib and the rANS coder of the supplement; the appearance-only row is the plain fp16 container). Fourteen-bit positions cost several tenths of a decibel and save little because the delta-coded positions are already cheap; six-bit attributes cost about a tenth of a decibel for a further sixth of the file.

Figure 5 : Spectral analysis of the observation Gram matrices of a pretrained model. Left: distribution of the numerical rank of

\mathbf{A}_{i}

. Right: fraction of the AC coefficient energy

\|\mathbf{K}_{i}\|^{2}

that lies in the null space of

\mathbf{A}_{i}

, i.e. in directions no training view observed.

Figure 6 : Test-view crops: ground truth, uncompressed model, truncation and projection to degree 1, truncation and projection to degree 0.

Scene	their VQ			$s_{i}\mathbf{I}$ , Lloyd			$\tr(\mathbf{A}_{i})\mathbf{I}/16$ , Lloyd			$\mathbf{A}_{i}$ , Lloyd			$\mathbf{A}_{i}$ , 2048			$\mathbf{A}_{i}$ , 1024			$\mathbf{A}_{i}$ , 512
	MB	pre	post	MB	pre	post	MB	pre	post	MB	pre	post	MB	pre	post	MB	pre	post	MB	pre	post
bicycle	47.3	24.44	25.04	48.3	24.53	25.00	48.3	24.55	25.00	49.9	24.78	25.01	49.1	24.73	25.01	48.5	24.68	24.95	47.9	24.62	24.96
bonsai	12.8	30.35	31.38	13.1	30.62	31.41	13.0	30.63	31.43	13.0	31.03	31.60	12.7	30.95	31.45	12.6	30.87	31.55	12.6	30.77	31.32
counter	13.8	27.90	28.66	14.2	28.16	28.74	14.2	28.14	28.73	13.5	28.34	28.79	13.3	28.30	28.66	13.3	28.24	28.68	13.3	28.17	28.68
flowers	31.2	21.00	21.22	32.0	21.09	21.25	31.9	21.11	21.29	32.6	21.23	21.32	–	–	–	–	–	–	31.4	21.13	21.22
garden	46.4	25.88	26.85	47.6	26.15	26.97	47.5	26.15	26.90	48.5	26.67	27.01	47.8	26.60	26.97	47.3	26.52	26.94	46.6	26.42	26.87

Appendix

Table 12: Per-scene results of Tab. 1 (Mip-NeRF 360): size after fine-tuning, test PSNR before and after fine-tuning for every colour-VQ metric inside Compressed3D.

Scene	colour VQ	codes used	index entropy (bit)	index stream (MB)	container (MB)
bicycle	their VQ	85856	9.72	6.49	47.27
bicycle	their sensitivity $s_{i}\mathbf{I}$ in generalised Lloyd	86960	11.53	7.49	48.25
bicycle	our weight $\tr(\mathbf{A}_{i})\mathbf{I}/16$ in generalised Lloyd	86960	11.42	7.45	48.26
bicycle	our matrix $\mathbf{A}_{i}$ in generalised Lloyd	86960	11.84	9.32	49.89
bicycle	our matrix, 2048-entry codebook	84912	10.89	8.67	49.15
bicycle	our matrix, 1024-entry codebook	83888	9.92	8.01	48.54

Appendix

Table 13: Where the size difference between the colour-VQ metrics of Tab. 1 comes from. Every array of the container is identical in kind and shape across variants; only the colour-index stream differs. Their minibatch EMA

k

-means uses the 4096-entry codebook unevenly (about 10 bit of zero-order entropy per index), whereas the generalised Lloyd assignment uses it almost uniformly (about 12 bit), which is more rate for the same codebook; the smaller codebooks give the rate-matched points of the main table. Codes used counts codebook entries plus the Gaussians kept uncompressed.

Configuration	Size (MB)	PSNR	SSIM	LPIPS
3DGS-30K reproduced (fp32)	793.5	27.29	0.812	0.218
fp16 attributes	416.9	27.29	0.812	0.218
+ allocation (9 AC floats)	175.7	27.26	0.812	0.219
+ Gram VQ	118.1	27.02	0.806	0.227
+ importance VQ (instead)	118.1	26.74	0.797	0.236
prune 50% (fp16)	208.5	27.12	0.810	0.221

Appendix

Table 14: Stage-by-stage ablation of the appearance pipeline on Mip-NeRF 360 (mean over 9 scenes), in a plain container without entropy coding (fp32 positions; fp16 scales, rotations, opacities, DC colours and codebooks; 12-bit codes; 2-bit degree tags); all rows are rendered from the rounded values. The geometry stage of Tab. 4 is applied on top of the last Gram-VQ row.

garden
Scene	geometry quantisation	Size (MB)	PSNR	SSIM	LPIPS
	uniform 16/8 bit	41.7	26.51	0.844	0.130
	uniform 14/8 bit	39.2	26.10	0.821	0.145
	uniform 16/6 bit	33.7	26.43	0.839	0.135
	uniform 14/6 bit	31.1	26.03	0.817	0.149
	uniform 12/5 bit	25.4	22.90	0.595	0.285

Appendix

Table 15: Geometry quantisation under the image-free Gauss–Newton sensitivities ( Appendix M ) against uniform bit depths, on the 50%-pruned models with the same appearance stage and the same rANS coder. Adaptive: four sensitivity classes per Gaussian (2-bit tag), bit depth per attribute group and class from a Lagrangian sweep over

\mu

. Bjøntegaard metrics of the adaptive curve against the uniform curve over the overlapping PSNR range.

Configuration	garden : PSNR near / mid / far			kitchen : PSNR near / mid / far
uncompressed	22.65	22.66	21.02	28.41	22.89	19.27
truncation, degree 1	21.92	22.40	21.58	25.46	22.18	19.47
projection, degree 1	22.40	22.19	20.74	27.09	22.99	19.70
truncation, degree 0	21.22	21.90	21.31	24.43	21.54	19.14
projection, degree 0	21.46	21.74	20.90	25.24	22.07	19.38
allocation (truncated), 9 floats	22.71	22.84	21.38	27.88	22.86	19.40

Appendix

Table 16: Extrapolation protocol. Models trained by the official 3DGS trainer with a contiguous

90\text{\,}\mathrm{\SIUnitSymbolDegree}

arc of the capture held out; statistics accumulated over the training cameras only; test PSNR on the held-out arc grouped by angular distance to the nearest training camera (near

<

10\text{,}\mathrm{\SIUnitSymbolDegree}

, mid $10\text{\,}$ – $25\text{\,}\mathrm{\SIUnitSymbolDegree}$ , far $\geq$25\text{\,}\mathrm{\SIUnitSymbolDegree}

; garden: 4/17/28 views; kitchen: 12/25/44 views). The last two rows perturb the uncompressed model along the null space of

\mathbf{A}_{i}

only.

		degree 3	degree 2 (24 AC floats)		degree 1 (9 AC floats)		degree 0 (0 AC floats)
Dataset	Metric	full	truncation	OGC (ours)	truncation	OGC (ours)	truncation	OGC (ours)
Mip-NeRF 360 (9/9)	PSNR	27.29	26.16	27.15	25.08	26.71	24.22	25.07
	SSIM	0.812	0.797	0.811	0.777	0.804	0.756	0.777
	LPIPS	0.218	0.232	0.219	0.249	0.224	0.264	0.245
Tanks&Temples (2/2)	PSNR	23.39	22.65	23.34	21.80	23.05	21.22	21.77
	SSIM	0.842	0.830	0.842	0.814	0.837	0.799	0.813

Appendix

Table 17: Uniform SH degree reduction of the official pretrained 3DGS models, averaged over each dataset (number of scenes evaluated in parentheses). Truncation drops the higher bands; OGC replaces them by the closed-form observation-Gram projection ( Eq. 8 ). Same storage in each column pair; no training images, gradients or fine-tuning are used.

coefficients	VQ metric	PSNR	SSIM	LPIPS	pred. $D$ ( $\times 10^{3}$ )
degree 3 (45)	none (uncompressed)	28.96	0.893	0.119	–
degree 3 (45)	Euclidean	27.96	0.875	0.143	156.5
degree 3 (45)	importance-weighted	28.20	0.877	0.141	129.3
degree 3 (45)	Gram (ours)	28.55	0.886	0.130	52.1
allocated (9)	none	28.89	0.892	0.120	–
allocated (9)	importance-weighted	28.02	0.874	0.143	145.7

Appendix

Table 18: Vector quantisation of AC coefficients with a 4096-entry codebook (mean of garden and kitchen ). The last column is the predicted distortion ( Eq. 7 , summed over training views) of the quantised model.

Scene	#Gaussians	full	trunc. 2	ours 2	trunc. 1	ours 1	trunc. 0	ours 0
bicycle	6.13M	25.03	24.36	24.95	23.67	24.60	23.06	23.69
bonsai	1.24M	32.06	29.34	31.78	27.64	30.84	26.57	28.06
counter	1.22M	28.91	27.13	28.70	25.69	27.87	24.64	25.52
flowers	3.64M	21.51	21.24	21.48	20.77	21.31	20.22	20.57
garden	5.83M	27.20	26.06	27.04	24.85	26.60	23.74	24.62
kitchen	1.85M	30.72	28.69	30.52	26.81	29.59	25.37	26.59

Appendix

Table 19: Per-scene test PSNR of uniform degree reduction (truncation vs. our projection).

Explore similar work

Sep 27, 2026cs.CV

FeCoSplat: Feedback-Guided Compression for Feed-Forward 3D Gaussian Splatting

Feed-forward 3D Gaussian Splatting (3DGS) enables efficient novel-view synthesis from sparse multi-view images, yet its representations remain costly to store and transmit. Existing approaches compress either the input images, incurring heavy receiver-side reconstruction, or the reconstructed Gaussian primitives, which are difficult to compress due to their heterogeneous and irregular attributes. We instead compress compact intermediate features, providing a better balance between compression efficiency and receiver-side complexity. Based on this paradigm, we propose FeCoSplat, a feedback-guided compression framework for feed-forward 3DGS. FeCoSplat first compresses multi-view features to obtain an intermediate 3DGS, whose rendered views are used as feedback to guide a second-stage compression for further refinement. The resulting bitstreams are decoded into a compact implicit state, from which the final Gaussian primitives are reconstructed with a lightweight predictor. Experiments demonstrate that FeCoSplat achieves favorable rate--distortion performance, particularly at low bitrates, while requiring only 3.45M parameters for receiver-side Gaussian reconstruction. Code will be released soon.

Yuxuan Li, Yihang Chen, Yufeng Zhang +2

Shanghai Jiao Tong University · Monash University

Feed-Forward 3D Gaussian Splatting Novel View Synthesis Data Compression Methods

Ask PDF

May 25, 2026cs.CV

CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting

While feed-forward 3D Gaussian splatting reconstructs renderable Gaussian primitives from sparse context views without per-scene optimization, existing pipelines do not provide a compact scene representation for storage or transmission. A natural solution is to apply existing 3DGS compression methods to the generated Gaussian primitives. However, this approach operates on the final irregular 3D representation and is decoupled from the internal feature-to-Gaussian generation process, which limits compression efficiency. To address this, we introduce CodecSplat, an ultra-compact latent coding framework for feed-forward 3D Gaussian splatting. CodecSplat first encodes an intermediate 2D Gaussian-generation feature into an entropy-coded scene bitstream. At the decoder, the latent feature is reconstructed and used to predict depth and Gaussian parameters, which are then mapped to 3D Gaussian primitives. Note that, by integrating compression into the feed-forward Gaussian generation pipeline, CodecSplat avoids inefficient compression over irregular 3D Gaussian primitives and allows the codec to exploit the structured intermediate feature representation. We instantiate CodecSplat on a feed-forward Gaussian splatting backbone with depth-guided multi-view feature refinement and a hierarchical learned feature codec. On DL3DV and RealEstate10K datasets, CodecSplat achieves 23.56-26.36 dB and 24.76-27.05 dB PSNR with only 20.00-107.77 KiB and 3.37-12.51 KiB per scene, respectively. This is roughly one order of magnitude smaller than compressing feed-forward generated Gaussian primitives, while preserving controllable rate-distortion behavior.

Pengpeng Yu, Runqing Jiang, Qi Zhang +3

Sun Yat-sen University, China · Pengcheng Laboratory, China · Peking University, China

Feed-Forward 3D Gaussian Splatting Feed-Forward 3D Gaussian 3D Gaussian Primitives 3D Generation

Ask PDF

Mar 23, 2026cs.CV

Drop-In Perceptual Optimization for 3D Gaussian Splatting

Despite their output being ultimately consumed by human viewers, 3D Gaussian Splatting (3DGS) methods often rely on ad-hoc combinations of pixel-level losses, resulting in blurry renderings. To address this, we systematically explore perceptual optimization strategies for 3DGS by searching over a diverse set of distortion losses. We conduct the first-of-its-kind large-scale human subjective study on 3DGS, involving 39,320 pairwise ratings across several datasets and 3DGS frameworks. A regularized version of Wasserstein Distortion, which we call WD-R, emerges as the clear winner, excelling at recovering fine textures without incurring a higher splat count. WD-R is preferred by raters more than

2.3\times

over the original 3DGS loss, and

1.5\times

over the current best method Perceptual-GS. WD-R also consistently achieves state-of-the-art LPIPS, DISTS, and FID scores across various datasets, and generalizes across recent frameworks, such as Mip-Splatting and Scaffold-GS, where replacing the original loss with WD-R consistently enhances perceptual quality within a similar resource budget (number of splats for Mip-Splatting, model size for Scaffold-GS), and leads to reconstructions being preferred by human raters

1.8\times

and

3.6\times

, respectively. We also find that this carries over to the task of 3DGS scene compression, with

\approx 50\%

bitrate savings for comparable perceptual metric performance.

Ezgi Ozyilkan, Zhiqi Chen, Oren Rippel +2

Apple · New York University Tandon School of Engineering

Perceptual Quality 3D Generation Pixel-Level Prediction

Ask PDF