TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference

Muhammad Hannan Akhtar; Ihab Amer; Tamer Shanableh

cs.CVApr 10, 2026

TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference

Authors: Muhammad Hannan Akhtar, Ihab Amer, Tamer Shanableh

Organizations: Department of Computer Science & Engineering, American University of Sharjah, Sharjah, United Arab Emirates

Abstract

Implicit neural video representations encode entire video sequences within the parameters of a neural network and enable constant time frame reconstruction. Recent work on Neural Representations for Videos (NeRV) has demonstrated competitive reconstruction performance while avoiding the sequential decoding process of conventional video codecs. However, most existing studies focus on moderate or high capacity models, leaving the behavior of extremely compact configurations required for constrained environments insufficiently explored. This paper presents a systematic study of tiny NeRV architectures designed for efficient deployment. Two lightweight configurations, NeRV-T and NeRV-T+, are introduced and evaluated across multiple video datasets in order to analyze how aggressive capacity reduction affects reconstruction quality, computational complexity, and decoding throughput. Beyond architectural scaling, the work investigates strategies for improving the performance of compact models without increasing inference cost. Knowledge distillation with frequency-aware focal supervision is explored to enhance reconstruction fidelity in low-capacity networks. In addition, the impact of lowprecision inference is examined through both post training quantization and quantization aware training to study the robustness of tiny models under reduced numerical precision. Experimental results demonstrate that carefully designed tiny NeRV variants can achieve favorable quality efficiency trade offs while substantially reducing parameter count, computational cost, and memory requirements. These findings provide insight into the practical limits of compact neural video representations and offer guidance for deploying NeRV style models in resource constrained and real-time environments. The official implementation is available at https: //github.com/HannanAkhtar/TinyNeRV-Implementation.

Figures & tables

Figure 1: Width-based scaling of NeRV variants. All models share identical depth and decoding structure; capacity differences arise solely from channel dimensionality.

Stage	Conv grid	NeRV-T	NeRV-T+	NeRV-S
	$(H\times W)$	Arithmetic $\rightarrow$ GFLOPs	Arithmetic $\rightarrow$ GFLOPs	Arithmetic $\rightarrow$ GFLOPs
Block1 ( $s=5$ )	$9\times 16$	$144\cdot 400\cdot(9\cdot 16)\rightarrow 0.0166$	$144\cdot 375\cdot(9\cdot 15)\rightarrow 0.0146$	$144\cdot 650\cdot(9\cdot 26)\rightarrow 0.0438$
Block2 ( $s=2$ )	$45\times 80$	$3600\cdot 128\cdot(9\cdot 16)\rightarrow 0.1327$	$3600\cdot 256\cdot(9\cdot 15)\rightarrow 0.2488$	$3600\cdot 384\cdot(9\cdot 26)\rightarrow 0.6470$
Block3 ( $s=2$ )	$90\times 160$	$14400\cdot 128\cdot(9\cdot 32)\rightarrow 1.0617$	$14400\cdot 256\cdot(9\cdot 64)\rightarrow 4.2467$	$14400\cdot 384\cdot(9\cdot 96)\rightarrow 9.5551$
Block4 ( $s=2$ )	$180\times 320$	$57600\cdot 128\cdot(9\cdot 32)\rightarrow 4.2467$	$57600\cdot 256\cdot(9\cdot 64)\rightarrow 16.9869$	$57600\cdot 384\cdot(9\cdot 96)\rightarrow 38.2206$
Block5 ( $s=2$ )	$360\times 640$	$230400\cdot 128\cdot(9\cdot 32)\rightarrow 16.9869$	$230400\cdot 256\cdot(9\cdot 64)\rightarrow 67.9477$	$230400\cdot 384\cdot(9\cdot 96)\rightarrow 152.8824$

Table 1: Layer-wise arithmetic and GFLOPs for NeRV-T, NeRV-T+, and NeRV-S on 720

\times

1280 video (BBB). For the

3\times 3

convolution blocks, multiplications are computed as

\text{Mult}=(HW)\,C_{out}\,(9C_{in})

, and GFLOPs

=2\cdot\text{Mult}/10^{9}

. PixelShuffle operations involve only tensor rearrangement and incur negligible arithmetic cost.

Figure 2: Proposed frequency–focal knowledge distillation. Teacher and student predictions are decomposed into low-frequency (smooth) and high-frequency (edge-dominant residual) components. A focal weighting term $w(k)$ emphasizes spatial locations with large teacher–student discrepancies in the high-frequency domain, guiding the tiny student toward improved detail reconstruction while preserving global structure.

Figure 3: Representative frames from the four evaluated sequences: Big Buck Bunny, honeybee, readysetgo, and yachtride. The sequences exhibit distinct structural patterns, motion characteristics, and texture density, providing a diverse evaluation setting for tiny neural video representations.

Model	Params (M)	FLOPs (G)	FPS	PSNR (dB)	MS-SSIM	PSNR/GFLOP
NeRV-T	0.80	22.62	197.9	27.84	0.882	1.231
NeRV-T+	1.68	89.80	82.7	29.35	0.913	0.327
NeRV-S [ 9 ]	3.20	201.9	47.5	32.11	0.959	0.159
NeRV-M [ 9 ]	6.28	202.9	46.6	36.02	0.984	0.178
NeRV-L [ 9 ]	12.53	204.2	45.3	39.70	0.993	0.194

Table 2: Capacity scaling of NeRV models on the BBB dataset under full-precision inference. FPS is measured using single-GPU inference. PSNR/GFLOP measures reconstruction quality normalized by computational cost.

Figure 4: Quality–compute trade-off on BBB for NeRV model variants under full precision.

Method	GFLOPs	Parameters
NeRV (Baseline) [ 9 ]	201.9	3.20M
E-NeRV [ 27 ]	208.0	3.31M
D-NeRV [ 87 ]	181.0	3.00M
HiNeRV [ 17 ]	190.0	3.25M
FFNeRV [ 35 ]	204.0	3.19M
SNeRV-B [ 29 ]	363.6	3.00M

Table 3: Comparison of computational complexity and parameter count with representative neural video representation methods reported in the literature.

Dataset	Model	T-PSNR (dB)	tSSIM
honeybee	NeRV-S	38.390	0.9674
	NeRV-M	38.613	0.9678
	NeRV-L	38.994	0.9687
	NeRV-T	38.334	0.9672
	NeRV-T+	38.356	0.9673
readysetgo	NeRV-S	26.915	0.8838

Table 4: Temporal stability results (FP32) across model scales and datasets.

Method	PSNR (dB)	MS-SSIM	$\Delta$ PSNR
Baseline (300 epochs)	27.84	0.8824	–
Extended Training (600 epochs)	28.19	0.8912	+0.35
KD Feature-Level Distillation	27.98	0.8835	+0.14
KD Temporal-Difference Distillation	28.51	0.8958	+0.67
KD Final-Output Distillation	28.60	0.8970	+0.76
KD Frequency–Focal Distillation	28.68	0.8979	+0.84

Table 5: Training-time improvements for NeRV-T on BBB under full-precision inference.

\Delta

PSNR is measured relative to the 300-epoch baseline. KD variants are ordered by increasing PSNR.

Method	PSNR (dB)	MS-SSIM	$\Delta$ PSNR
Baseline (300 epochs)	29.35	0.9134	–
Extended Training (600 epochs)	29.64	0.9160	+0.29
KD Feature-Level Distillation	29.45	0.9092	+0.10
KD Temporal-Difference Distillation	29.82	0.9182	+0.47
KD Final-Output Distillation	30.05	0.9215	+0.70
KD Frequency–Focal Distillation	30.23	0.9231	+0.88

Table 6: Training-time improvements for NeRV-T+ on BBB under full-precision inference.

\Delta

PSNR is measured relative to the 300-epoch baseline. KD variants are ordered by increasing PSNR.

	NeRV-T (PSNR dB)		NeRV-T+ (PSNR dB)
Precision	PTQ	QAT	PTQ	QAT
FP32 (Baseline)	27.84		29.35
INT8	27.84	27.88	29.31	29.36
INT6	26.92	27.63	28.78	29.10

Table 7: Moderate-precision deployment results on BBB using weights-only quantization. FP32 denotes the full-precision baseline without quantization.

Model	PTQ (PSNR dB)	QAT (PSNR dB)	KD+QAT (PSNR dB)
NeRV-T	24.66	27.12	27.66
NeRV-T+	26.31	28.72	29.25

Table 8: INT4 results on BBB using weights-only quantization. PTQ denotes post-training quantization, QAT denotes quantization-aware training, and KD+QAT combines knowledge distillation with quantization-aware training.

Figure 5: Qualitative comparison across the honeybee, readysetgo, and yachtride sequences. Each row corresponds to one sequence, and each column shows reconstructions from different model configurations. NeRV-L serves as a high-capacity reference, while NeRV-S provides a baseline. NeRV-T+ and NeRV-T illustrate the effect of aggressive capacity reduction in the tiny regime, and the distilled NeRV-T model shows the impact of knowledge distillation on reconstruction fidelity.

		honeybee		readysetgo		yachtride
Model	Method	PSNR	MS-SSIM	PSNR	MS-SSIM	PSNR	MS-SSIM
NeRV-S	FP32 Baseline	39.15	0.9914	28.45	0.9490	29.03	0.9249
NeRV-M	FP32 Baseline	40.75	0.9933	31.78	0.9754	31.65	0.9595
NeRV-L	FP32 Baseline	41.70	0.9944	34.42	0.9864	34.23	0.9792
NeRV-T	FP32 Baseline	34.26	0.9794	23.63	0.8608	26.23	0.8785
NeRV-T	FT-only	34.45	0.9789	23.83	0.8693	26.34	0.8807

Table 9: Full-precision results across additional sequences. “FT-only” denotes extended fine-tuning without distillation, while “KD Frequency–Focal” denotes frequency–focal distillation fine-tuning.

		honeybee		readysetgo		yachtride
Model	Precision	PTQ	QAT	PTQ	QAT	PTQ	QAT
NeRV-T	INT8	34.17	34.24	23.62	23.66	26.23	26.35
NeRV-T	INT6	33.05	33.90	23.51	23.63	26.11	26.23
NeRV-T+	INT8	36.92	36.97	25.07	25.10	27.22	27.24
NeRV-T+	INT6	35.89	36.54	24.91	25.06	27.09	27.20

Table 10: Moderate-precision quantization across additional sequences (PSNR dB).

	NeRV-T			NeRV-T+
Dataset	PTQ	QAT	KD+QAT	PTQ	QAT	KD+QAT
honeybee	26.54	31.19	31.70	29.22	33.67	33.91
readysetgo	22.13	23.29	23.52	23.26	24.61	24.88
yachtride	24.84	25.95	26.32	25.69	26.77	26.97

Table 11: INT4 deployment results across additional sequences (PSNR dB).

Explore similar work

LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression
Mar 18, 2026Tamer ShanablehVideo Representation Learning Neural Video Compression
American University of Sharjah, UAE, Department of Computer Science and Engineering
TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos
Feb 18, 2026Namitha Padmanabhan, Matthew Gwilliam, Abhinav ShrivastavaNeural Video Compression Implicit Neural Representations
Department of Computer Science, University of Maryland, College Park, USA.
Video Compression with Graph-inspired Neural Representation
Sep 2, 2026Changqi Wang, Ge Gao, Fan Zhang +4Neural Video Compression Implicit Neural Representations
University of Bristol · Bytedance Inc., San Diego, USA

Mar 18, 2026J/K move · Enter open · S save

LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression

Tamer Shanableh

American University of Sharjah, UAE, Department of Computer Science and Engineering

Video Representation Learning Neural Video Compression Low-Rank Approximation Convolutional Neural Networks Video Compression

Open paper AskPDF arXiv

Neural Representations for Videos (NeRV) encode entire video sequences within neural network parameters, offering an alternative paradigm to conventional video codecs. However, the convolutional decoder of NeRV remains computationally expensive and memory intensive, limiting its deployment in resource-constrained environments. This paper proposes LRConv-NeRV, an efficient NeRV variant that replaces selected dense 3x3 convolutional layers with structured low-rank separable convolutions, trained end-to-end within the decoder architecture. By progressively applying low-rank factorization from the largest to earlier decoder stages, LRConv-NeRV enables controllable trade-offs between reconstruction quality and efficiency. Extensive experiments demonstrate that applying LRConv only to the final decoder stage reduces decoder complexity by 68%, from 201.9 to 64.9 GFLOPs, and model size by 9.3%, while incurring negligible quality loss and achieving approximately 9.2% bitrate reduction. Under INT8 post-training quantization, LRConv-NeRV preserves reconstruction quality close to the dense NeRV baseline, whereas more aggressive factorization of early decoder stages leads to disproportionate quality degradation. Compared to existing work under layer-aligned settings, LRConv-NeRV achieves a more favorable efficiency versus quality trade-off, offering substantial GFLOPs and parameter reductions while maintaining higher PSNR/MS-SSIM and improved temporal stability. Temporal flicker analysis using LPIPS further shows that the proposed solution preserves temporal coherence close to the NeRV baseline, results establish LRConv-NeRV as a potential architectural alternative for efficient neural video decoding under low-precision and resource-constrained settings.

Abstract

Explore similar work

LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression

TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos

Video Compression with Graph-inspired Neural Representation