Implicit neural representations (INRs) have emerged as a promising paradigm for video compression, providing compact neural representations with flexible spatial and temporal reconstruction. Hierarchical grid-based architectures such as HiNeRV achieve strong rate--distortion performance, but require extensive per-video optimization, resulting in high encoding costs. To address this limitation, we propose ScoutNeRV, a content-adaptive initialization framework for accelerating the optimization of hierarchical video INRs. ScoutNeRV employs a lightweight, offline-trained scout network that analyzes a small number of sampled frames and selects a suitable pre-trained expert from a memory bank through hard routing. The hierarchical grid and decoder parameters of the selected expert are then transferred to initialize the target HiNeRV model before video-specific fine-tuning. On the unseen ReadySetGo sequence, ScoutNeRV achieves an initial PSNR of 34.95~dB, compared with 13.70~dB for standard initialization, corresponding to a 21.25~dB improvement before fine-tuning. After only 37 epochs, ScoutNeRV reaches 36.92~dB and remains within 0.42--0.80~dB of the 300-epoch HiNeRV baseline across the evaluated rate--distortion configurations. Furthermore, the proposed initialization achieves a 9.25× wall-clock speedup in the reported runtime experiment. These results demonstrate that content-aware expert initialization can substantially reduce the optimization cost of hierarchical video INRs while retaining competitive reconstruction and compression performance. The code is available at https://github.com/nasserdeveloper/ScoutNeRV.
Figures & tables
Figure 1: Convergence comparison on the Jockey sport sequence, showing the initialization advantage of ScoutNeRV.
Figure 2: Qualitative comparison of standard HiNeRV initialization and ScoutNeRV on the Jockey sequence at epoch 0.
Figure 3: Overview of the ScoutNeRV framework.
Figure 4: Rate–distortion performance on the unseen ReadySetGo sequence.
Domain
Expert source
Evaluation target
Sport
Jockey
ReadySetGo
News/face
Beauty
Unseen news clips
Animation
Big Buck Bunny
Unseen animation clips
Table 1: Source and target sequences used for expert transfer.
Method
Backbone
PSNR ↑
MS-SSIM ↑
NeRV * 2
NeRV
25.50
0.85
HiNeRV † 12
HiNeRV
32.46
0.9582
MetaNeRV 13
NeRV
26.82
0.88
RNeRV 8
NeRV
26.38
0.87
ScoutNeRV †
HiNeRV
39.94
0.9828
Table 2: Fast-encoding comparison on the Jockey source sequence under a 30-epoch optimization budget.
Initialization
Source expert
PSNR at epoch 0 ↑
Standard HiNeRV
None
13.70 dB
Wrong-domain expert
News
15.50 dB
Generic pre-training
Mixed
19.20 dB
ScoutNeRV
Sport
34.95 dB
Table 3: Effect of expert selection on the unseen ReadySetGo sequence.
Sequence
Initialization
Loss ↓
PSNR ↑
Beauty S
Standard
0.2290
15.10
ScoutNeRV
0.0358
34.51
Jockey S
Standard
0.2323
17.50
ScoutNeRV
0.0152
36.50
ReadySetGo U
Standard
0.3131
13.70
ScoutNeRV
0.0125
34.95
Table 4: Initialization quality across different video sequences.
Figure 5: Qualitative results. ScoutNeRV ( 37 epochs, right) achieves visual quality on par with the 300 -epoch HiNeRV (left), validating the efficiency of our domain-expert initialization.
Scalable video coding (SVC) encodes a video into a layered bitstream consisting of a base layer and one or multiple enhancement layers, enabling decoding at different bitrate/quality/resolution operating points to accommodate diverse device capabilities and network conditions. Due to its practical flexibility, SVC has been incorporated into major video coding standards and has recently attracted growing interest for both scene-agnostic and scene-adaptive neural video codecs. Among the latter, Implicit neural representation (INR) based codecs achieve compression by overfitting a compact neural network to an individual video, offering fast decoding and competitive coding efficiency compared to scene-agnostic neural codecs. However, research on scalable INR-based compression remains in its infancy: these methods support scalable coding by introducing additional network layers, which couple the bitrate with the decoding complexity and also cannot achieve comparable performance with strong scalable/non-scalable codecs. In this context, this paper proposes S-NVRC, a scalable INR-based video codec that jointly supports fine-grained bitrate and decoding complexity scalability from a single embedded bitstream. It adopts a coarse-to-fine prefix for feature grids and a nested prefix for network layers, which scale bitrate and decoding complexity, respectively. The proposed S-NVRC spans a wide range of bitrate and decoding-complexity using a single encoding (training) and outperforms SHM 12.4 and the multi-layer VTM-20.0, by 43.7% and 5.6% in BD-rate on the UVG dataset, while also providing flexible complexity scalability. Implemented code will be provided.
Tianhao Peng, Ho Man Kwan, Fan Zhang +2
Visual Information Lab, University of Bristol, UK · Tencent Media Lab, Palo Alto, USA
Implicit neural representations (INRs) have recently emerged as a promising approach to video compression, delivering competitive rate-distortion performance alongside rapid decoding. However, existing neural video codecs struggle to balance complexity and scalability. Lightweight models often suffer from degraded compression performance when scaled to different bitrate/quality levels, whereas high-performance models exhibit limited scalability, as their model complexity typically increases with quality. This lack of a unified architecture capable of maintaining consistent complexity across a wide range of bitrates severely limits their diverse real-world deployment. To address these challenges, we introduce NVRC++, a novel INR-based video codec that utilizes a lightweight INR with multiple high-resolution feature grids, providing high scalability at any given complexity level. This is paired with an optimization framework that enables efficient overfitting on high-resolution grids for long video sequences, thereby exploiting spatio-temporal redundancies without prohibitive computational or memory overhead. Additionally, an advanced entropy model is designed for efficiently compressing the high-dimensional grid parameters. As a result, NVRC++ provides four complexity levels (from 7kMACs/pixel to 360kMACs/pixel), each spanning wide bitrate and quality ranges while supporting real-time decoding. The experimental results show that NVRC++ offers a much faster decoding speed (up to 7.6x) compared to the SOTA INR-based video codec, NVRC, while delivering comparable performance.
Ho Man Kwan, Tianhao Peng, Fan Zhang +3
Visual Information Lab, University of Bristol, UK · Network Connectivity Services Research, BT, UK
Implicit Neural Representations (INR) provide a compact and content-adaptive paradigm for video compression, typically representing a video through shared network parameters and frame-indexed embeddings. Compared to conventional or autoencoder-based codecs, these approaches exploit temporal redundancy within videos in an implicit manner, which potentially results in sub-optimal compression performance. In this paper, we propose G-NeRV, a graph-inspired INR that explicitly improves temporal redundancy exploitation in the implicit latent space. Motivated by the total correlation principles in information theory, we construct a temporal neighborhood over frame embeddings and perform message passing to aggregate reusable information from neighboring frames through an adaptive gate controlling the injection of neighboring information. Inspired by the reference frame buffer in conventional video coding, a memory bank mechanism has been further designed to enable efficient temporal-neighbor retrieval under random frame-index sampling in INR training. This new representation model has been integrated into an advanced representation compression framework and compared with existing conventional and neural video codecs. The results show that the G-NeRV codec outperforms the state-of-the-art INR-based codec, NVRC, and the latest standard video codec, VVC VTM, by 8.86% and 14.68% (in BD-rate), respectively, measured by PSNR on the UVG dataset.
Changqi Wang, Ge Gao, Fan Zhang +4
University of Bristol · Bytedance Inc., San Diego, USA