cs.CVJan 21, 2025

GSVC: Efficient Video Representation and Compression Through 2D Gaussian Splatting

Authors: Longan Wang, Yuang Shi, Wei Tsang Ooi

Organizations: Department of Computer Science National University of Singapore Singapore

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In this work, we explore 2D Gaussian splat as a new primitive for representing and compressing videos, showing its promise as an alternative to traditional signal-based primitives. We introduce GSVC, a new Gaussian splatting-based video codec that incorporates common techniques used in classical video codecs to exploit temporal redundancy between nearby frames. In GSVC, each frame is represented by a set of overlapping Gaussian splats, which, when rendered together, approximate the original frame. Gaussian splats for the first frame and subsequent frames following scene changes (called anchor frames) are learned from scratch, while the remaining frames (called propagation frames) are initialized from the Gaussian splats of their previous frames and then fine-tuned. Scene changes are detected through loss differences during the training process. To capture large motion or newly appeared objects, we randomly add Gaussian splats to propagation frames before fine-tuning. GSVC supports a knob for rate-distortion trade-offs through controlling the number of Gaussian splats for each frame. Gaussian splats with low contributions to the frame quality can be removed during training. GSVC supports two-pass encoding by propagating the Gaussian splats from a future propagation frame back to an anchor frame, further improving the video quality for challenging video scenes. To achieve efficient compression, GSVC uses a residual-based Gaussian splats quantization framework that encodes only changes to the Gaussian parameters between frames. Experimental results show that GSVC achieves rate-distortion performance comparable to state-of-the-art video codec implementations for AV1 and VVC. GSVC inherits the advantages of Gaussian splatting in rendering, allowing it to achieve the rendering speed of 1500 fps for a 1920×10801920\times 1080 video. Project page: https://longanwang-cs.github.io/gsvc/.

Figures & tables

Figure 1. Using Gaussian splats to model an image. The image is taken from the UVG Jockey video and has the resolution of 1920×10801920\times 1080 . The sub-figures show that as the number of Gaussian splats NN increases, the learned splats increasingly approximate the image’s content.
Figure 2. Intermediate reconstruction results after tt training iterations using 10,000 Gaussian splats on a 1920×10801920\times 1080 frame from the UVG Jockey video.
Figure 3. PSNR across frames 1–15 on Jockey with 10k Gaussian splats for (a) anchor-only reconstruction versus frame-prediction reconstruction (first frame as anchor frame and remaining frames as propagation frames), and (b) impact of two-pass training, comparing the original anchor frame (single pass) with the refined anchor frame obtained through two-pass training (two passes).
Figure 4. Gaussian Splat Center Distributions ( 45k45k ) and Corresponding Renderings on f1f_{1} of Beauty , HoneyBee , and Jockey .
Figure 5. Distribution of Gaussian Splat Centers on Video Dynamics.
Figure 6. Δℓ\Delta\boldsymbol{\ell} and PSNR over Frames on SceneMix .
Figure 7. Quantization Pipeline. (a) shows the encoding pipeline, and (b) shows the decoding pipeline. We store the quantized I-frame Gaussian splats, QI\boldsymbol{Q}^{I} , and the quantized differences of the P-frame Gaussian splats, ΔQP\Delta\boldsymbol{Q}^{P} .
Frame TypeI-frameP-frame
Covariance scaling/offset24 bytes48 bytes
Color codebooks192 bytes360 bytes
Total Per-frame Info216 bytes408 bytes
Splat TypeIntra-coded Splats (Anchor/Propagation)Inter-coded Splats (Propagation)
Position32 bits24 bits
Quantized Cholesky vector18 bits15 bits
Table 1. Stored parameters and side information for I-frames and P-frames.
Figure 8. Rate-distortion performance of GSVC and the baselines in terms of PSNR, MS-SSIM, and VMAF.
Figure 9. Effect of Gaussian propagation on PSNR, MS-SSIM, and VMAF across Gaussian budgets before quantization on UVG.
Figure 10. Ablation study on the look-ahead length of PSNR gain, encoding latency overhead, and peak GPU memory overhead before quantization on UVG.
Figure 11. Effect of two-pass encoding on PSNR, MS-SSIM, and VMAF across Gaussian budgets before quantization.
Figure 12. Effects of GSP, GSA, and DKS on PSNR, MS-SSIM, and VMAF across Gaussian budgets before quantization on UVG.
Figure 13. Ablation study of Gaussian pruning strategies on UVG in terms of average PSNR, MS-SSIM, and per-frame training time before quantization. ContributionFT\mathrm{Contribution}^{\mathrm{FT}} denotes GSP (contribution-based pruning with fine-tuning), while RandomFT\mathrm{Random}^{\mathrm{FT}} and Random\mathrm{Random} denote random pruning with and without fine-tuning, respectively.
Figure 14. Threshold sensitivity of DKS, HSV, and ORB. We report the precision and recall under varying detection thresholds. The shaded regions indicate stable threshold ranges over which both precision and recall remain 1.
Figure 15. Effect of residual-based quantization on PSNR, MS-SSIM, and VMAF across bitrates on UVG.
More in this topic

Video Compression

Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.

See all 77 papers

Explore similar work

CardsList
Oct 6, 2026

Efficient Gaussian Splatting Sequence Compression with Standard Video Codecs

This paper presents a novel effective Gaussian Splatting (GS) sequence Compression method that utilizes the Video codec (GSCV). Existing video-based GS sequence compression relies on the Parallel Linear Assignment Sorting (PLAS) and tracked primitive information to convert GS into smooth 2D videos. However, tracked information is not available for most practical applications, and without it, using the vanilla PLAS can generate images exhibiting weak inter-frame correlation, due to its stochastic nature. GSCV incorporates a simple yet efficient Inter-PLAS method to produce close images between the I- and P-frames of GS, enhancing the inter-frame performance of video codec greatly. GSCV also realizes a new pipeline based on the state-of-the-art video codecs with high bit-depth GS images, achieving higher compressibility while simultaneously providing a higher quality upper bound. Experimental results show that the proposed GSCV exhibits obviously improved performance over MPEG video and point cloud-based anchors in GS sequence compression. The code is available at https://github.com/Qi-Yangsjtu/GSCV.

Qi Yang, Shuting Xia, Le Yang +2

School of Science and Engineering, University of Missouri - Kansas City · Cooperative Medianet Innovation Center, Shanghai Jiaotong University · Electrical and Computer Engineering, University of Canterbury +1

3DGS CompressionVideo CompressionGaussian Splatting Compression
AskPDF
Mar 11, 2026

P-GSVC: Layered Progressive 2D Gaussian Splatting for Scalable Image and Video

Gaussian splatting has emerged as a competitive explicit representation for image and video reconstruction. In this work, we present P-GSVC, the first layered progressive 2D Gaussian splatting framework that provides a unified solution for scalable Gaussian representation in both images and videos. P-GSVC organizes 2D Gaussian splats into a base layer and successive enhancement layers, enabling coarse-to-fine reconstructions. To effectively optimize this layered representation, we propose a joint training strategy that simultaneously updates Gaussians across layers, aligning their optimization trajectories to ensure inter-layer compatibility and a stable progressive reconstruction. P-GSVC supports scalability in terms of both quality and resolution. Our experiments show that the joint training strategy can gain up to 1.9 dB improvement in PSNR for video and 2.6 dB improvement in PSNR for image when compared to methods that perform sequential layer-wise training. Project page: https://longanwang-cs.github.io/PGSVC-webpage/

Longan Wang, Yuang Shi, Wei Tsang Ooi

National University of Singapore, Singapore

Gaussian Splatting CompressionGaussian Splatting
AskPDF
Nov 3, 2026

GSCV++: Lightweight 3D Gaussian Splatting Sequence Compression via Standard Video Codecs

This paper presents GSCV++, a lightweight pipeline for 3D {G}aussian {S}platting (GS) sequences {C}ompression with standard {V}ideo codecs. To meet the requirements of low-complexity and hardware compatibility from the industry, GSCV++ jointly improves preprocessing, 3D-to-2D mapping, and practical video compression configuration of GS compression. First, for preprocessing, energy-adaptive inter-frame spherical harmonic (SH) alternative current (AC) dimensionality reduction (EA-SHDR) is proposed to produces a compact and stable temporal SH AC feature space. It can reduce the data volume of following mapping and practical compression, as well as less complexity while eliminating redundant information. Second, for 3D-to-2D mapping, prior-based Inter-MiniPLAS (P-MiniPLAS) is designed to generate smooth 2D feature maps through prior-based weighting factor inference and small scale remapping. Compared with vanilla Parallel Linear Assignment Sorting (PLAS) and Morton scan, P-MiniPLAS ensures the generated 2D maps are more suitable for the video codec while maintaining a low level of computational complexity. Third, for practical compression, content-aware generalized compression (CA-GC) is developed to use a non-uniform bit-depth split for coordinate map generation, and a coding unit size selection is introduced for efficient compression complexity control, which fully exploits the performance of video codecs without involving normative changes. Experiments demonstrate that GSCV++ reports over 20% in intra- and 60% in inter-prediction rate-distortion gains compared with state-of-the-art (SOTA) methods. GSCV++ also achieves faster 2D map generation, encoding, and decoding than other SOTA methods using the same type of video codecs. Ablation studies verify the effectiveness and robustness of the proposed modules and provide insights for subsequent research.

Qi Yang, Yingzhan Xu, Kai Zhang +1

Bytedance Inc., San Diego · Bytedance Inc., Beijing

3DGS CompressionVideo Compression
AskPDF