cs.CVOct 23, 2026

HEALPix-Based Implicit Neural Compression of Omnidirectional Images

Authors: Yulong Cheng, Youneng Bao, Junfeng Zhou, Mu Li, Jie Wen

Organizations: Harbin Institute of Technology, Shenzhen, China · Shenzhen University, Shenzhen, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Real-time decoding of omnidirectional images is essential for immersive virtual reality (VR) experiences. Compared with conventional transform-based learned image compression (LIC), overfitted codecs offer lower decoding complexity by representing each image with a lightweight implicit neural representation (INR). However, existing approaches typically operate on the equirectangular projection (ERP), which introduces nonuniform spherical sampling and spatial distortion, leading to redundant sampling near the poles and latitude-dependent receptive fields. To address these limitations, we propose a spherical overfitted codec based on a multi-scale Hierarchical Equal Area isoLatitude Pixelization (HEALPix) representation, which enables equal-area sampling and efficient hierarchical modeling of spherical signals. Building on this representation, we develop a Spherical Inter-Scale Conditional entropy model that exploits spatial dependencies and cross-scale correlations among HEALPix latents to improve coding efficiency. We further introduce a Multi-Ring Spherical Convolution that aggregates neighboring pixels over a nearly uniform spherical receptive field across latitudes. Experiments on a public omnidirectional image dataset demonstrate BD-rate reductions of 16.3%, 26.5%, and 19.5% over VVC in terms of V-PSNR, V-MS-SSIM, and V-LPIPS, respectively. Our method also achieves a 13.1% BD-rate reduction over ERP-based Cool-Chic 5.0 in terms of V-PSNR, while providing 2.6-3.6x faster decoding than the evaluated learned autoencoder codecs.

Figures & tables

Fig. 1: Representations of an omnidirectional image, including the ERP, pseudocylindrical representation [ 1 ] and ours.
Fig. 2: Overall architecture. Each panorama is represented by hierarchical latent grids {Yi}i=0N\{Y_{i}\}_{i=0}^{N} on the HEALPix sphere ( Nside∈{2,4,…,256}N_{\mathrm{side}}\in\{2,4,\ldots,256\} ), which are quantized and fitted to the image. Top: the SISC entropy model fφf_{\varphi} predicts the Laplace parameters of each latent from same-level and ancestor contexts (Fig. 3 ) for the arithmetic coder. Bottom: the spherical upsampler fγf_{\gamma} merges the levels from coarse to fine, and the multi-ring synthesis network fθf_{\theta} (Fig. 4 ) reconstructs the sphere. The latents and the decoder weights are trained with SOAP on D+λ Rlatent/(HW)D+\lambda\,\mathcal{R}_{\mathrm{latent}}/(HW) ; the weight rate R(φ,γ,θ)\mathcal{R}(\varphi,\gamma,\theta) of Eq. ( 12 ) is not differentiated and is handled by a rate–distortion search over the weight quantization at the end of fitting (Sec. III-E ).
ProjectionSWWNWNNEESESMean
ERP11.9948.2611.990.0011.9948.2611.990.0018.06
HEALPix11.1510.2811.1513.8311.1510.2811.1513.8311.60
TABLE I: Relative standard deviation (%) of neighbor distances in each direction over the sphere, for ERP at 768×1536768\times 1536 and HEALPix at Nside=256N_{\mathrm{side}}=256 . Lower is more uniform; the best mean is in bold.
Fig. 3: Spherical Inter-Scale Conditional (SISC) Entropy Model. Left: for a latent Y^i,j\hat{Y}_{i,j} (red dot), the intra-scale context collects its KK nearest decoded neighbors (pink), and the inter-scale context is obtained by Lineari\mathrm{Linear}_{i} from its decoded ancestors on the coarser levels (colored dots). Right: the shared predictor fφf_{\varphi} maps the concatenated contexts to the Laplace parameters (Eq. ( 5 )).
Fig. 4: Multi-Ring Spherical Convolution. Receptive fields of radius R∈{1,2,3}R\in\{1,2,3\} on the HEALPix 88 -neighbor graph ( Nside=4N_{\mathrm{side}}{=}4 ). Each neighborhood BR(v)B_{R}(v) contains every sampling point within RR hops of the center (highlighted).
Fig. 5: Spherical nearest-parent upsampling from level ii to level i+1i{+}1 . The fine sampling point v=v0v=v_{0} and its eight neighbors v1,…,v8v_{1},\dots,v_{8} are mapped to their parents at level ii , and the region of each fine sampling point is drawn in the color of its parent.
Fig. 6: Rate–distortion curves on the 100100 -image benchmark under V-PSNR (left), V-MS-SSIM (center), and V-LPIPS (right, lower is better), computed on the six viewports of Sec. IV-A . C5 denotes Cool-Chic 5.0.
Fig. 7: Rate–distortion curves at 1536×30721536\times 3072 on the 9999 images , computed on six 1024×5121024\times 512 viewports. LPIPS uses AlexNet, and C5 denotes Cool-Chic 5.0.
 768×1536768\times 1536 ( 100100 images)1536×30721536\times 3072 ( 9999 images)
Ours vs.V-PSNRV-MS-SSIMV-LPIPSV-PSNRV-MS-SSIMV-LPIPS
VVC [ 20 ]−16.3-16.3−26.5-26.5−19.5-19.5−15.8-15.8−24.2-24.2−19.0-19.0
Cheng [ 25 ]−24.4-24.4−25.5-25.5−25.8-25.8−19.0-19.0−20.8-20.8−28.4-28.4
ELIC [ 26 ]−19.3-19.3−25.3-25.3−23.3-23.3−10.5-10.5−16.3-16.3−18.8-18.8
C3 [ 16 ]−20.7-20.7−31.4-31.4−23.3-23.3−27.5-27.5−35.3-35.3−29.4-29.4
Cool-Chic 5.0 [ 18 ]−13.1-13.1−24.9-24.9−13.9-13.9−18.7-18.7−27.6-27.6−18.8-18.8
TABLE II: BD-rate(%) of our codec against each baseline under the three viewport metrics; negative values mean fewer bits at the same quality. The left half uses the common range of 0.10.1 – 0.50.5 bpp (Fig. 6 ), and the right half uses the range shared by each pair of codecs (Fig. 7 ).
  Rate change at equal quality (%)
VariantMAC/pxV-PSNRV-MS-SSIMV-LPIPS
Ours4619———
(a) Representation
HEALPix, 1 level2897+44.5+44.5+52.5+52.5+28.6+28.6
ERP, 8 levels6928+4.2+4.2+6.6+6.6+5.6+5.6
ERP, 1 level4346+57.6+57.6+72.5+72.5+38.1+38.1
TABLE III: Rate change (%) of each ablation variant against our codec at equal quality under the three viewport metrics; positive values mean more bits at the same quality. Each row of (a)–(c) changes a single component of our codec. MAC/px is the number of decoder multiplications per image pixel.
Fig. 8: Visual comparison at similar rates on one panorama: six viewports and six 5.3×5.3\times crops. The columns show the reference and seven codecs, sorted by V-PSNR. The crops are taken from regions where our reconstruction is closer to the reference than all baselines. Each baseline is shown at its operating point closest to ours in rate (five of the six use 33 – 55%55\% more bits, and OSLO-IC uses 1.7%1.7\% fewer), and all six have a V-PSNR 0.450.45 – 1.091.09 dB lower than ours.
Fig. 9: Measured rate per latent for the three finest levels of one panorama, for the full model (top) and for the same codec fitted without the inter-scale context CInter\mathbf{C}^{\mathrm{Inter}} (bottom); the number under each panel is the rate of that level in bpp, and the right column shows the reconstruction with its V-PSNR and total rate. In both rows, the rate follows the image structure rather than the latitude. Without the inter-scale context, the finest level costs 46%46\% more ( 0.026→0.0380.026\rightarrow 0.038 bpp), and the total rate rises by 29%29\% ( 0.127→0.1640.127\rightarrow 0.164 bpp) at a similar V-PSNR.
Fig. 10: Areal bit density versus latitude , i.e., bits per unit area on the sphere normalized by the equatorial value, for Cool-Chic 5.0 on ERP and for our codec on the test set. Toward the poles, the ERP codec rises steeply, though below the sec⁡δ\sec\delta law of Eq. ( 1 ) (thin line), whereas ours stays near the uniform ideal (dotted).
MethodParadigmEnc. (s) ↓\downarrowDec. (s) ↓\downarrow
VVC [ 20 ]Traditional160.080160.0800.3240.324
Cheng [ 25 ]Autoencoder14.25314.25327.91027.910
ELIC [ 26 ]Autoencoder15.66215.66225.33125.331
MLIC++ [ 21 ]Autoencoder16.89316.89321.13121.131
LIC360 [ 4 ]Autoencoder0.466†0.466^{\dagger}0.601†0.601^{\dagger}
OSLO-IC [ 6 ]Autoencoder22.00422.00420.10520.105
TABLE IV: Encoding and decoding time per image on the test set ( 768×1536768\times 1536 ). Decoding times are measured on a single CPU core with batch size 11 . The encoding time of our codec is the per-image fitting time (Algorithm 1 ) on one GPU. VVC is CPU-only reference software. † LIC360 runs only on a GPU.
More in this topic

Learned Image Compression

Learned image compression uses neural encoders, quantization, and entropy models to reduce image bitrates while balancing fidelity, perceptual quality, and task utility.

See all 120 papers

Explore similar work

CardsList

  1. Rate-Distortion Adaptive Primitive Selection for Omnidirectional Gaussian Splatting

    Sep 28, 2026Yulong Cheng, Youneng Bao, Junfeng Zhou +23DGS Compression3D Gaussian Pruning

    Harbin Institute of Technology, Shenzhen, China · Shenzhen University, Shenzhen, China

  2. PIC: Revisiting INR for Image Coding with Fast Encoding and Sub-Millisecond Decoding

    Sep 8, 2026Xiang Liu, Jinxiang Wang, Bin Chen +5Efficient Neural Network InferenceLearned Image Compression

    Tsinghua University, China · Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, China +1

  3. Viewport-based Neural 360° Image Compression

    Mar 24, 2026Jingwei Liao, Bo Chen, Klara Nahrstedt +1Learned Image Compression

    George Mason University, Fairfax, USA · University of Illinois Urbana-Champaign, Urbana, USA