Real-time decoding of omnidirectional images is essential for immersive virtual reality (VR) experiences. Compared with conventional transform-based learned image compression (LIC), overfitted codecs offer lower decoding complexity by representing each image with a lightweight implicit neural representation (INR). However, existing approaches typically operate on the equirectangular projection (ERP), which introduces nonuniform spherical sampling and spatial distortion, leading to redundant sampling near the poles and latitude-dependent receptive fields. To address these limitations, we propose a spherical overfitted codec based on a multi-scale Hierarchical Equal Area isoLatitude Pixelization (HEALPix) representation, which enables equal-area sampling and efficient hierarchical modeling of spherical signals. Building on this representation, we develop a Spherical Inter-Scale Conditional entropy model that exploits spatial dependencies and cross-scale correlations among HEALPix latents to improve coding efficiency. We further introduce a Multi-Ring Spherical Convolution that aggregates neighboring pixels over a nearly uniform spherical receptive field across latitudes. Experiments on a public omnidirectional image dataset demonstrate BD-rate reductions of 16.3%, 26.5%, and 19.5% over VVC in terms of V-PSNR, V-MS-SSIM, and V-LPIPS, respectively. Our method also achieves a 13.1% BD-rate reduction over ERP-based Cool-Chic 5.0 in terms of V-PSNR, while providing 2.6-3.6x faster decoding than the evaluated learned autoencoder codecs.
Figures & tables
Fig. 1: Representations of an omnidirectional image, including the ERP, pseudocylindrical representation [ 1 ] and ours.
Fig. 2: Overall architecture. Each panorama is represented by hierarchical latent grids {Yi}i=0N on the HEALPix sphere ( Nside∈{2,4,…,256} ), which are quantized and fitted to the image. Top: the SISC entropy model fφ predicts the Laplace parameters of each latent from same-level and ancestor contexts (Fig. 3 ) for the arithmetic coder. Bottom: the spherical upsampler fγ merges the levels from coarse to fine, and the multi-ring synthesis network fθ (Fig. 4 ) reconstructs the sphere. The latents and the decoder weights are trained with SOAP on D+λRlatent/(HW) ; the weight rate R(φ,γ,θ) of Eq. ( 12 ) is not differentiated and is handled by a rate–distortion search over the weight quantization at the end of fitting (Sec. III-E ).
Projection
SW
W
NW
N
NE
E
SE
S
Mean
ERP
11.99
48.26
11.99
0.00
11.99
48.26
11.99
0.00
18.06
HEALPix
11.15
10.28
11.15
13.83
11.15
10.28
11.15
13.83
11.60
TABLE I: Relative standard deviation (%) of neighbor distances in each direction over the sphere, for ERP at 768×1536 and HEALPix at Nside=256 . Lower is more uniform; the best mean is in bold.
Fig. 3: Spherical Inter-Scale Conditional (SISC) Entropy Model. Left: for a latent Y^i,j (red dot), the intra-scale context collects its K nearest decoded neighbors (pink), and the inter-scale context is obtained by Lineari from its decoded ancestors on the coarser levels (colored dots). Right: the shared predictor fφ maps the concatenated contexts to the Laplace parameters (Eq. ( 5 )).
Fig. 4: Multi-Ring Spherical Convolution. Receptive fields of radius R∈{1,2,3} on the HEALPix 8 -neighbor graph ( Nside=4 ). Each neighborhood BR(v) contains every sampling point within R hops of the center (highlighted).
Fig. 5: Spherical nearest-parent upsampling from level i to level i+1 . The fine sampling point v=v0 and its eight neighbors v1,…,v8 are mapped to their parents at level i , and the region of each fine sampling point is drawn in the color of its parent.
Fig. 6: Rate–distortion curves on the 100 -image benchmark under V-PSNR (left), V-MS-SSIM (center), and V-LPIPS (right, lower is better), computed on the six viewports of Sec. IV-A . C5 denotes Cool-Chic 5.0.
Fig. 7: Rate–distortion curves at 1536×3072 on the 99 images , computed on six 1024×512 viewports. LPIPS uses AlexNet, and C5 denotes Cool-Chic 5.0.
768×1536 ( 100 images)
1536×3072 ( 99 images)
Ours vs.
V-PSNR
V-MS-SSIM
V-LPIPS
V-PSNR
V-MS-SSIM
V-LPIPS
VVC [ 20 ]
−16.3
−26.5
−19.5
−15.8
−24.2
−19.0
Cheng [ 25 ]
−24.4
−25.5
−25.8
−19.0
−20.8
−28.4
ELIC [ 26 ]
−19.3
−25.3
−23.3
−10.5
−16.3
−18.8
C3 [ 16 ]
−20.7
−31.4
−23.3
−27.5
−35.3
−29.4
Cool-Chic 5.0 [ 18 ]
−13.1
−24.9
−13.9
−18.7
−27.6
−18.8
TABLE II: BD-rate(%) of our codec against each baseline under the three viewport metrics; negative values mean fewer bits at the same quality. The left half uses the common range of 0.1 – 0.5 bpp (Fig. 6 ), and the right half uses the range shared by each pair of codecs (Fig. 7 ).
Rate change at equal quality (%)
Variant
MAC/px
V-PSNR
V-MS-SSIM
V-LPIPS
Ours
4619
—
—
—
(a) Representation
HEALPix, 1 level
2897
+44.5
+52.5
+28.6
ERP, 8 levels
6928
+4.2
+6.6
+5.6
ERP, 1 level
4346
+57.6
+72.5
+38.1
TABLE III: Rate change (%) of each ablation variant against our codec at equal quality under the three viewport metrics; positive values mean more bits at the same quality. Each row of (a)–(c) changes a single component of our codec. MAC/px is the number of decoder multiplications per image pixel.
Fig. 8: Visual comparison at similar rates on one panorama: six viewports and six 5.3× crops. The columns show the reference and seven codecs, sorted by V-PSNR. The crops are taken from regions where our reconstruction is closer to the reference than all baselines. Each baseline is shown at its operating point closest to ours in rate (five of the six use 3 – 55% more bits, and OSLO-IC uses 1.7% fewer), and all six have a V-PSNR 0.45 – 1.09 dB lower than ours.
Fig. 9: Measured rate per latent for the three finest levels of one panorama, for the full model (top) and for the same codec fitted without the inter-scale context CInter (bottom); the number under each panel is the rate of that level in bpp, and the right column shows the reconstruction with its V-PSNR and total rate. In both rows, the rate follows the image structure rather than the latitude. Without the inter-scale context, the finest level costs 46% more ( 0.026→0.038 bpp), and the total rate rises by 29% ( 0.127→0.164 bpp) at a similar V-PSNR.
Fig. 10: Areal bit density versus latitude , i.e., bits per unit area on the sphere normalized by the equatorial value, for Cool-Chic 5.0 on ERP and for our codec on the test set. Toward the poles, the ERP codec rises steeply, though below the secδ law of Eq. ( 1 ) (thin line), whereas ours stays near the uniform ideal (dotted).
Method
Paradigm
Enc. (s) ↓
Dec. (s) ↓
VVC [ 20 ]
Traditional
160.080
0.324
Cheng [ 25 ]
Autoencoder
14.253
27.910
ELIC [ 26 ]
Autoencoder
15.662
25.331
MLIC++ [ 21 ]
Autoencoder
16.893
21.131
LIC360 [ 4 ]
Autoencoder
0.466†
0.601†
OSLO-IC [ 6 ]
Autoencoder
22.004
20.105
TABLE IV: Encoding and decoding time per image on the test set ( 768×1536 ). Decoding times are measured on a single CPU core with batch size 1 . The encoding time of our codec is the per-image fitting time (Algorithm 1 ) on one GPU. VVC is CPU-only reference software. † LIC360 runs only on a GPU.
Learned image codecs (LICs) achieve high reconstruction quality, but their decoding speed is often insufficient for immersive virtual reality (VR). Gaussian splatting (GS) codecs render much faster, yet still lag in reconstruction quality and typically decide primitive allocation without considering the coding cost of each primitive. We introduce OIC-GS, an omnidirectional GS codec with a new hierarchical HEALPix primitive grid representation. Gaussian primitives are anchored at predefined spherical locations, eliminating explicit coordinate coding. Finer levels refine their coarser ancestors, naturally supporting coarse-to-fine reconstruction and layered transmission. The predefined grid also enables efficient viewport decoding by selecting only view-relevant primitives. We further introduce a lightweight entropy model for quantized primitives and optimize the codec under a spherical rate-distortion objective. Primitives with insufficient rate-distortion benefit are automatically removed when their quantized opacity becomes zero, allowing OIC-GS to adapt both primitive density and level of detail without a fixed primitive budget. A single bitstream supports full-sphere, viewport-dependent, and progressive decoding. The first viewport reaches final quality after decoding only 52% of the bitstream, and is then rendered at 1,270 FPS. On a 100-image omnidirectional benchmark, OIC-GS outperforms all evaluated GS codecs, reducing WS-PSNR BD-rate by 49.6% over GaussianImage++ and 68.6% over SGI, which uses a learned entropy model.
Yulong Cheng, Youneng Bao, Junfeng Zhou +2
Harbin Institute of Technology, Shenzhen, China · Shenzhen University, Shenzhen, China
Implicit neural representation (INR) has achieved remarkable progress in novel view synthesis and image/video coding in recent years.Compared to conventional end-to-end image codecs, INR-based compressors demonstrate significant advantages in decoding complexity. However, their practical application has been hindered by the inferior encoding speed and underutilized decoding efficiency.In this work, we propose a feedforward INR image coding architecture, Practical INR Image Codec (PIC), that computes all the necessary information for INR network in a single forward pass, achieving an encoding speed of 20 FPS. Additionally, we implement a highly optimized decoder that reaches 2000 FPS decoding speed, significantly surpassing JPEG's performance at comparable rate-distortion (RD) performance. To the best of our knowledge, this work presents the first learning-based image codec that simultaneously outperforms or is comparable with JPEG in both RD performance and decoding speed while maintaining practical encoding speed. Code is available at https://github.com/actcwlf/PIC.
Xiang Liu, Jinxiang Wang, Bin Chen +5
Tsinghua University, China · Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, China +1
Given the popularity of 360° images on social media platforms, 360° image compression becomes a critical technology for media storage and transmission. Conventional 360° image compression pipeline projects the spherical image into a single 2D plane, leading to issues of oversampling and distortion. In this paper, we propose a novel viewport-based neural compression pipeline for 360° images. By replacing the image projection in conventional 360° image compression pipelines with viewport extraction and efficiently compressing multiple viewports, the proposed pipeline minimizes the inherent oversampling and distortion issues. However, viewport extraction impedes information sharing between multiple viewports during compression, causing the loss of global information about the spherical image. To tackle this global information loss, we design a neural viewport codec to capture global prior information across multiple viewports and maximally compress the viewport data. The viewport codec is empowered by a transformer-based ViewPort ConText (VPCT) module that can be integrated with canonical learning-based 2D image compression structures. We compare the proposed pipeline with existing 360° image compression models and conventional 360° image compression pipelines building on learning-based 2D image codecs and standard hand-crafted codecs. Results show that our pipeline saves an average of 14.01% bit consumption compared to the best-performing 360° image compression methods without compromising quality. The proposed VPCT-based codec also outperforms existing 2D image codecs in the viewport-based neural compression pipeline. Our code can be found at: https://github.com/Jingwei-Liao/VPCT.
Jingwei Liao, Bo Chen, Klara Nahrstedt +1
George Mason University, Fairfax, USA · University of Illinois Urbana-Champaign, Urbana, USA