cs.CVSep 28, 2026
SaveRate-Distortion Adaptive Primitive Selection for Omnidirectional Gaussian Splatting
Organizations: Harbin Institute of Technology, Shenzhen, China · Shenzhen University, Shenzhen, China
Abstract
Learned image codecs (LICs) achieve high reconstruction quality, but their decoding speed is often insufficient for immersive virtual reality (VR). Gaussian splatting (GS) codecs render much faster, yet still lag in reconstruction quality and typically decide primitive allocation without considering the coding cost of each primitive. We introduce OIC-GS, an omnidirectional GS codec with a new hierarchical HEALPix primitive grid representation. Gaussian primitives are anchored at predefined spherical locations, eliminating explicit coordinate coding. Finer levels refine their coarser ancestors, naturally supporting coarse-to-fine reconstruction and layered transmission. The predefined grid also enables efficient viewport decoding by selecting only view-relevant primitives. We further introduce a lightweight entropy model for quantized primitives and optimize the codec under a spherical rate-distortion objective. Primitives with insufficient rate-distortion benefit are automatically removed when their quantized opacity becomes zero, allowing OIC-GS to adapt both primitive density and level of detail without a fixed primitive budget. A single bitstream supports full-sphere, viewport-dependent, and progressive decoding. The first viewport reaches final quality after decoding only 52% of the bitstream, and is then rendered at 1,270 FPS. On a 100-image omnidirectional benchmark, OIC-GS outperforms all evaluated GS codecs, reducing WS-PSNR BD-rate by 49.6% over GaussianImage++ and 68.6% over SGI, which uses a learned entropy model.
Figures & tables
Figure 1: Overview of OIC-GS. Primitives are anchored on hierarchical HEALPix levels and carry a quantized feature and opacity, coded level by level. The levels are rendered from coarse to fine; a primitive whose quantized opacity is zero becomes inactive, and its cell falls back to the coarser levels. The rendering state is reconstructed identically at the encoder and the decoder.
Figure 2: The rendering-conditioned entropy model (RCC). Even cells (pass 1) are coded with the coarse and parent contexts; the decoded even cells are then added to the context of the odd cells (pass 2). The ContextMLP predicts a 3-component Gaussian mixture for the arithmetic coder, and the decoder reconstructs the identical context by construction. Cell indices are illustrative.
Figure 3: Rate-distortion performance on the 100-image main benchmark. OIC-GS outperforms every GS codec at every tested bitrate on all four metrics.
Figure 4: Rate-distortion performance on the 100 SUN360 images. OIC-GS again outperforms every GS codec at every tested bitrate on all four metrics.
Figure 5: Reconstructed viewports. Every baseline uses at least as many bits as OIC-GS. The labels give the viewport PSNR and, for each zoomed-in patch, the PSNR of the textured window. App. G shows five scenes in all six directions.
| Main benchmark | SUN360 | |||||||
| Method | WS-PSNR | V-PSNR | V-SSIM | V-LPIPS | WS-PSNR | V-PSNR | V-SSIM | V-LPIPS |
| Gaussian splatting image codecs | ||||||||
| SGI | ||||||||
| GaussianImage | ||||||||
| GaussianImage++ | ||||||||
| LIG | ||||||||
Table 1: BD-rate ( Bjontegaard, 2001 ) against JPEG on the main benchmark and on the 100 SUN360 images; a negative value means fewer bits at equal quality.
Figure 6: Schematic comparison of first-viewport delivery over a 10 Mbps link for full-ERP codecs, tiled JPEG2000 and OIC-GS. Size: bytes transmitted before the first viewport is shown; Transmit: transfer time of these bytes.
| Method | Category | Entropy decoding (ms) | Reconstruction (FPS) |
|---|---|---|---|
| JPEG ( Wallace, 1992 ) | Traditional | ||
| JPEG2000 ( Skodras et al., 2001 ) | Traditional | ||
| ELIC ( He et al., 2022 ) | LIC, checkerboard context | ||
| MLIC++ ( Jiang et al., 2025 ) | LIC, multi-reference context | ||
| mbt2018 ( Minnen et al., 2018 ) | LIC, autoregressive | ||
| cheng2020-attn ( Cheng et al., 2020 ) | LIC, autoregressive | ||
Table 2: Decoding time and rendering speed on one ERP image (one idle RTX 4090). Entropy decoding includes the context networks of the LIC rows and the rendering-conditioned contexts of OIC-GS; reconstruction is the synthesis network of the LIC rows and the rendering of the GS rows. JPEG and JPEG2000 report the full CPU decoding speed, and – marks codecs without a learned entropy model. The viewport row decodes only the streams of the first viewport. Details are given in App. D .
| Variant (default in parentheses) | rate at equal quality (%), is worse | |
|---|---|---|
| WS-PSNR | V-PSNR | |
| Full model | ||
| Rate-distortion adaptive primitive selection (Sec. 3.1 ) | ||
| w/o rate-aware opacity weighting ( ) | ||
| selection threshold ( ) | ||
| selection threshold ( ) | ||
Table 3: Ablation studies of the main components and of the design parameters of each module on the 100 images of the main benchmark: rate difference against the full model at equal quality at (positive is worse). The protocol, the variant specifications and the SUN360 results are given in App. E.2 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Constant | Value |
|---|---|
| Evaluation grid of Eq. ( 4 ) | target grid, , directions |
| Candidate set | cells (the cell containing and its 8 neighbors); |
| all primitives at | |
| Denominator floor of Eq. ( 4 ) | |
| Fill opacity in Eq. ( 3 ) | |
| Selection threshold in Eq. ( 2 ) |
Table 4: Rendering and selection constants.
| Item | Setting |
|---|---|
| Optimizer | Adam ( Kingma and Ba, 2014 ) |
| Learning rate, features | at the finest level, decreasing linearly with the level to at |
| Learning rate, opacities | |
| Learning rate, entropy model | |
| Learning rate, ColorMLP | |
| Learning rate, kernel widths |
Table 5: Training hyperparameters.
| bpp | WS-PSNR (dB) | V-PSNR (dB) | V-SSIM | V-LPIPS | |
|---|---|---|---|---|---|
| 0.1448 | 23.83 | 27.61 | 0.7099 | 0.3750 | |
| 0.2885 | 25.36 | 29.79 | 0.7888 | 0.2522 | |
| 0.3843 | 26.05 | 30.72 | 0.8202 | 0.2083 | |
| 0.4931 | 26.67 | 31.58 | 0.8469 | 0.1733 | |
| 0.6576 | 27.45 | 32.63 | 0.8764 | 0.1372 | |
| 0.9494 | 28.50 | 34.06 | 0.9103 | 0.0981 |
Table 6: Operating points of OIC-GS: mean values over the 100 panoramas, measured end-to-end against the original ERP images (Sec. 4.1 ). These are the points of OIC-GS plotted in Fig. 3 . Bitrates are the written bitstream sizes in bits (bytes 8) divided by the number of ERP pixels. The decoding measurements of App. D use the trained models of the first five points.
| Baseline | OIC-GS BD-rate (%) vs. baseline | |||
|---|---|---|---|---|
| WS-PSNR | V-PSNR | V-SSIM | V-LPIPS | |
| SGI | ||||
| GaussianImage++ | ||||
| GaussianImage | ||||
| LIG | ||||
Table 7: Direct pairwise BD-rates of OIC-GS against the GS baselines, each computed on the common quality range of that pair.
Figure 7: Bit allocation over the sphere on the benchmark panoramas at matched bitrates. Bits are assigned to latitude bands, divided by the solid angle of each band, and normalized by the global mean of each method; for OIC-GS, the feature bits are shown. (a) The ERP-based codecs follow the sampling density of the projection, while OIC-GS is distributed substantially more evenly; GaussianImage and LIG are omitted from (a) for readability. (b) The polar bands (the outermost bands on each side, ) cover of the sphere and receive of the feature bits of OIC-GS, close to their area share, compared with - for the ERP-based codecs, close to the ERP pixel share.
| Fixed viewport | ||
| Stage (per frame) | ms | share |
| Level-0 splatting ( primitives) | ||
| Levels 1-6 splatting ( ) | ||
| Coarse-level downsampling | ||
| Blending (Eq. ( 5 )) | ||
| ColorMLP | ||
Table 8: Per-frame time breakdown of the repeated rendering of a fixed viewport (one idle RTX 4090). The viewport rendering evaluates the target-grid directions inside the padded frustum and maps them to a frame that stays on the GPU; the host copy is not included. Stage times are GPU kernel times recorded with the PyTorch profiler over frames, as medians over models (scenes 1, 7 and 46 bitrates; two viewports, yaw and , per model).
| Format | File size | size | Saved (yaw / ) |
|---|---|---|---|
| 32 equal streams per pass (default) | B ( bpp) | – | – |
| Tile-aligned ( cells/tile) | B ( bpp) | / |
Table 9: Two storage formats of the same symbols on the representative scene ( viewports). The tile-aligned format increases the file size by ( benchmark mean) and allows - of the file to be skipped for one viewport; the default format has no tile boundaries to skip.
| First view (bpp) | Deferred | (KB) | Unchanged | (FPS) | |
|---|---|---|---|---|---|
| / |
Table 10: Progressive streaming: the models of scenes 1, 7 and 46 at the point of Table 6 played end-to-end in fresh processes ( sessions; first viewport at yaw , followed by a -frame pan to ). “ (KB)” is the data size of phase , and “ (FPS)” is the rendering speed of phase . “Deferred” is the fraction of the file not yet received when the first viewport is displayed in each session. “Unchanged” counts the sessions whose first-viewport pixels are bit-identical after completion; gives the re-splatting/cached speeds of Table 11 .
| Strategy | ms/frame | FPS | GT-PSNR | Notes |
|---|---|---|---|---|
| Re-splatting, visible set with margin | updates of the visible set | |||
| Cached sphere, lookup only | bit-identical to re-splatting |
Table 11: Head-rotation speed on the representative scene ( frames from yaw to , at , idle RTX 4090). FPS is computed from the unrounded mean frame time. Both strategies are measured in a single run that decodes once and then pans; over the sessions of Table 10 , they run at / FPS on average. GT-PSNR, the mean viewport PSNR along the trajectory against the ground-truth projection, is identical for all strategies, so the acceleration does not affect quality.
Figure 8: Progressive decoding by level, from the written bitstream of scene 1 at the operating point ( KB; Lanczos-3 resampling to ERP). Each panel shows the full sphere reconstructed only from the levels , labeled with the bytes of this prefix (share of the file) and the end-to-end WS-PSNR. Because of the coarse-to-fine order of Sec. 3.2 , every prefix gives a complete sphere with less detail: a recognizable preview requires the first of the file (levels ), and the final prefix is bit-exact with full decoding.
| Reference round-trip | Native sphere PSNR (dB) | End-to-end ERP WS-PSNR (dB) | |||||
|---|---|---|---|---|---|---|---|
| Scene | fidelity | L6 | L7 | L6 | L7 | ||
| 87 | 30.921 | 32.952 | 45.466 | 28.702 | 30.737 | ||
| 88 | 32.016 | 33.349 | 43.963 | 29.225 | 31.660 | ||
| 84 | 32.659 | 32.515 | 41.484 | 28.959 | 31.959 | ||
| 86 | 33.541 | 32.290 | 40.666 | 28.892 | 32.574 | ||
| 73 | 34.341 | 35.009 | 44.354 | 31.496 | 33.848 | ||
Table 12: Paired L6/L7 resolution analysis. “Reference round-trip fidelity” is the WS-PSNR of the original ERP image after the reference ERP HEALPix ERP round trip; it is used to stratify the image selection and as the reference score in Fig. 9 , and it is not a mathematical upper bound. Both the native and the end-to-end scores improve on all scenes. These fits use only the distortion loss, continuous features and all primitives active, and are therefore not codec operating points or results at matched bitrates.
Figure 9: Resolution analysis. Left: the finest level at improves the end-to-end ERP WS-PSNR for every selected image ( dB on average; range - dB). Right: images with lower round-trip fidelity of the reference ERP HEALPix ERP path, which indicates stronger resampling loss and more high-frequency content, obtain a larger native L7 gain (Pearson correlation ). The native gain is also correlated with the global and nadir high-frequency energy ( and ), and is dB on average at the nadir against dB at the equator.
| Metric | BD-rate (%) | at matched bitrate | Negative on |
|---|---|---|---|
| WS-PSNR | dB | ||
| V-PSNR | dB | ||
| native sphere PSNR | dB |
Table 13: The L7 codec against the L6 configuration on the 20-image subset of App. F.2 , at the three operating points , computed with the estimator of Table 1 on the common quality range of each pair. A negative value means fewer bits at equal quality; “Negative on” counts the images with a defined BD-rate. Native sphere PSNR is measured on the HEALPix representation and excludes the effect of resampling.
| Level 7 | Selection ratio, L6 config L7 config | Active primitives | |||
|---|---|---|---|---|---|
| median (range) | Level 4 | Level 5 | Level 6 | L6 L7 | |
| ( - ) | K K | ||||
| ( - ) | K K | ||||
| ( - ) | K K | ||||
Table 14: Selection ratios after training, once the inactive primitives are removed, on the 20-image subset of App. F.2 at the three operating points . Level 7 contains cells; the median column gives the per-image median selection ratio (range in parentheses), and the remaining columns compare the same level between the two configurations at the same .
Figure 10: Locations of active primitives of the additional level on scene 87: active level-7 cells are shown in red over the ground truth. They concentrate on foliage, rocky ground and the shoreline, and are absent from the sky and open water; decreasing (bottom) selects more of them without changing where they are located.
Figure 11: Additional qualitative results (part 1 of 3): scene 55 in all six viewing directions, under the protocol described in the text. Across the five scenes, OIC-GS achieves the highest viewport PSNR among the GS codecs and COIN in of the panels, and the highest patch PSNR in . On this scene, the only exception is the zenith, where GaussianImage++ and GaussianImage exceed OIC-GS by and dB at and bpp, compared with bpp for OIC-GS; the other four exceptions are shown in Fig. 12 . OIC-GS matches or exceeds JPEG2000 in of the equatorial panels of scenes 7, 32 and 55, while JPEG2000 remains ahead at the nadir of every scene.
Figure 12: Additional qualitative results (part 2 of 3): scenes 74 (top) and 46 (bottom) in all six viewing directions, under the protocol of Fig. 11 . Every baseline uses more bits than OIC-GS on both scenes ( against - bpp on scene 74, and against - bpp on scene 46). Four of the five panels in Figs. 11 – 13 where a GS codec exceeds OIC-GS in viewport PSNR are shown here, all for GaussianImage++ at a - higher bitrate and by - dB: the yaw- and nadir views of scene 74 and the zenith and nadir of scene 46. The nadir patch of scene 74 is the only panel where no textured window favors OIC-GS over JPEG; at the zenith of scene 55, the best window ties.
Figure 13: Additional qualitative results (part 3 of 3): scenes 7 (top) and 32 (bottom) in all six viewing directions, under the protocol of Fig. 11 . Every baseline uses more bits than OIC-GS on both scenes ( against - bpp on scene 7, and against - bpp on scene 32), and OIC-GS achieves the highest viewport PSNR among the GS codecs and COIN in all twelve panels.