ANVIL: Accelerator-Native Video Interpolation via Codec Motion Vector Priors

Shibo Liu

eess.IVMar 27, 2026

ANVIL: Accelerator-Native Video Interpolation via Codec Motion Vector Priors

Authors: Shibo Liu

Organizations: College of Science, North China University of Science and Technology, Tangshan 063210, China

Abstract

Real-time 30-to-60 fps video frame interpolation on mobile neural processing units (NPUs) requires each synthesized frame within 33.3 ms. We show that mainstream flow-based video frame interpolation faces three structural deployment barriers on mobile NPUs: spatial sampling operators exceed the frame budget or lack hardware support, iterative flow refinement collapses under 8-bit integer post-training quantization, and memory-bound operators dominate the inference graph. ANVIL addresses these barriers by reusing motion vectors from the H.264/AVC decoder to prealign input frames, removing learned optical flow, spatial sampling, and iterative accumulation from the accelerator graph. The remaining residual is refined by a convolution-dominated network composed almost entirely of compute-bound operators. On a Snapdragon 8 Gen 3 device, ANVIL achieves 12.8 ms 1080p inference at 8-bit integer precision; an open-source Android player sustains 28.4 ms median end-to-end latency over 30-minute continuous playback. Per-operator causal analysis identifies quantized accumulation on recurrent flow states as a key mechanism behind integer quantization failure in iterative methods. The current design targets H.264/AVC playback with decoder-exposed motion vectors.

Figures & tables

Fig. 1: ANVIL three-processor pipeline. CPU densifies and downsamples MVs ( ${\sim}$ 2.9 ms); GPU (Vulkan) performs median filtering, Gaussian blur, and sub-pixel remap ( ${\sim}$ 3.7 ms); HTP runs the INT8 residual network ( ${\sim}$ 13–17 ms). CPU/GPU preparation for frame $N{+}1$ is pipelined with HTP inference for frame $N$ .

Fig. 2: UNet-v3b architecture. Input: 6-channel prealigned pair. Encoder: 4 levels with channel widths [16, 32, 64, 64] (ANVIL-S) or [16, 32, 96, 96] (ANVIL-M); encoder blocks per level

(1,1,1,2)

(ANVIL-S) or

(1,1,2,2)

(ANVIL-M). Bottleneck: 4 ResBlocks (ANVIL-S) or 8 (ANVIL-M) at

\frac{H}{16}\times\frac{W}{16}

. Decoder uses Add skip connections with 1 ResBlock per level. Output: 3-channel residual added to the prealigned blend. BN is folded into Conv at deploy time.

SoC	NPU	ANVIL-S	ANVIL-M	RIFE ‡	$\leq$ 33.3ms
SD 7+ Gen2	HTP V69	46.0 ms	56.6 ms	45.6 ms	$\times^{\dagger}$
SD 8 Gen2	HTP V73	15.5 ms	20.8 ms	20.7 ms	✓
SD 8 Gen3	HTP V75	12.8 ms	16.7 ms	14.7 ms	✓

TABLE I: HTP INT8 network inference latency at 1080p (NPU forward pass only; end-to-end in Table IX ). † 720p fallback: ANVIL-S 10.5 ms , ANVIL-M 14.1 ms . ‡ RIFE at 360p.

Group	Operator	Freq	HTP V75	APU 790	Impact
A	Conv2d 3 $\times$ 3	9/9	1.0 $\times$	1.0 $\times$	Baseline
A	Conv2d 1 $\times$ 1	8/9	0.9 $\times$	1.0 $\times$
A	Conv + ReLU	5/9	1.0 $\times$	1.0 $\times$
A	ConvTranspose2d	7/9	3.9 $\times$	2.7 $\times$	Moderate cost both platforms
A	Residual Add	9/9	2.3 $\times$	1.8 $\times$
B	GridSample	7/9	3.2 $\times$	N/A	MTKEXT custom op, public SDK unavailable

TABLE II: Cross-vendor operator compatibility for 17 VFI operators at 1080p. Latency normalized to Conv 3

{\times}

3 = 1.0

{\times}

. “Freq”: methods using the operator (out of 9 surveyed). Groups: (A) universal, (B) partial, (C) limited, (D) iterative patterns.

Method	Stg	Mode	PSNR: FP32 $\to$ INT8 ( $\Delta$ )	95% CI
ANVIL-S	0	direct	29.28 $\,\to\,$ 29.09 ( $\boldsymbol{-}$ 0.19 ) †	— †
ANVIL-M	0	direct	29.37 $\,\to\,$ 29.28 ( $\boldsymbol{-}$ 0.09 ) †	— †
RIFE	3	flow $\uparrow$	26.79 $\,\to\,$ 25.90 ( $-$ 0.89)	[ $-$ 0.96, $-$ 0.83]
IFRNet	4	flow $\uparrow$	25.43 $\,\to\,$ 25.03 ( $-$ 0.40)	[ $-$ 0.44, $-$ 0.37]
IFRNet	4	frame	25.84 $\,\to\,$ 21.46 ( $-$ 4.38)	[ $-$ 4.75, $-$ 4.01]

TABLE III: INT8 quantization quality on Xiph 1080p. “Mode”: flow

\uparrow

outputs flow for CPU warp; frame outputs RGB directly. † Full 2,662-triplet QNN on-device evaluation (aggregate PSNR; per-sample bootstrap CI not available for device-side inference). Calibration and eval protocol details in Sec. IV .

Quantized Ops	RIFE (3-stage)		IFRNet (4-stage)
Quantized Ops	CosSim	$\Delta$	CosSim	$\Delta$
None (FP32)	1.000	—	1.000	—
Conv, ConvTr	0.952	$-$ 0.048	0.945	$-$ 0.055
$+$ PReLU	0.952	0.000	0.945	0.000
$+$ Add	0.815	$-$ 0.137	0.878	$-$ 0.068
Full W8A8	0.790	$\approx$ Add	0.859	$\approx$ Add

TABLE IV: Per-operator instrumented INT8 quantization. CosSim measured against FP32 output. The same causal pattern holds for both methods: Conv introduces initial error, PReLU has zero effect, and Add triggers the collapse . Full W8A8

\approx

Conv

+

Add.

Method	Params	Vimeo90K			Xiph 1080p			Deployment Status
Method	Params	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS	Deployment Status
Zero-parameter baselines
Naive Blend	0	25.61	0.753	0.099	25.32	0.658	0.165	—
MV Blend	0	31.20	0.926	0.053	28.98	0.813	0.115	—
ANVIL (ours) — 1080p INT8 deployable
ANVIL-S	855K	33.45	0.949	0.037	29.65	0.835	0.154	Deployable

TABLE V: Quality and deployment status on Vimeo90K and Xiph 1080p. Bold : best among deployable methods. For context, non-deployable methods reach 35–36 dB on Vimeo90K (IFRNet [ 2 ] 35.80, AMT-S [ 3 ] 35.72, EMA-VFI [ 6 ] 36.11).

Fig. 3: Visual comparison on Xiph 1080p ( $3\times$ magnified insets). (a) old_town_cross : ANVIL smoothing suppresses noise, RIFE preserves detail. (b) tractor : ANVIL over-smooths edges, RIFE produces ghosting. (c) riverbed : both fail on stochastic texture.

∗ Exceeds 33.3 ms deadline even at INT8.
Res.	Mode	FP32	ORT $\Delta$	QNN $\Delta$	INT8 lat.
360p	flow $\uparrow$	28.54	$-$ 2.03	$-$ 1.43	14.7 ms
360p	frame $\uparrow$	27.00	$-$ 3.80	$-$ 3.44	17.8 ms
480p	flow $\uparrow$	29.13	$-$ 2.90	$-$ 2.24	28.2 ms
480p	frame $\uparrow$	28.23	$-$ 5.12	$-$ 4.84	36.3 ms ∗

TABLE VI: RIFE resolution reduction strategies on Xiph 1080p. ORT

\Delta

: W8A8 via ORT CPU. QNN

\Delta

: W8A8 vs FP16 on HTP V75. Both backends confirm INT8 degradation; all FP16 latencies exceed 33.3 ms .

Flow construction	Post-processing	PSNR
ZOH	none (sub-pixel remap only)	29.63
OBMC (cosine)	none	29.58
Daala	Gaussian $\sigma{=}6$	30.89
ZOH	median 5 $+$ Gaussian $\boldsymbol{\sigma{\approx}8}$	31.20

TABLE VII: Prealignment method ablation on Vimeo90K test set (3,782 triplets, MV Blend PSNR). All methods use the same block-level H.264 MVs; they differ in flow construction and post-processing.

Model	Params	Vimeo PSNR	INT8 1080p
Route A (no prealignment)
A-small	33K	29.00	—
Route D (MV prealignment)
D-tiny-nomv	1.8K	30.67	3.52 ms
D-mini-nomv	10.6K	31.08	4.26 ms
D-mid-nomv	33K	31.38	8.16 ms

TABLE VIII: Capacity scaling under the 33.3 ms latency budget. Prealigned variants (Route D) are fully trained on Vimeo90K with basic prealignment. A-small (Route A) receives raw frames without MV prealignment.

Fig. 4: Quality scaling with model capacity. Basic prealignment (dashed) and smoothed prealignment (solid) series both scale without saturation. Dashed horizontal lines show RIFE HDv3 (3.04M) and NAFNet ceiling (17.1M, smoothed prealignment retrained) as reference points.

Stage	HW	Latency	Operation
P1a	CPU	2.9 ms	ZOH densify + $4{\times}$ downsample + YUV pack
P1b+P2	GPU	3.7 ms	median 5 + Gauss $\sigma\!=\!2$ + warp + quant
Copy	CPU	0.9 ms	12 MB uint8 NHWC to QNN buffer
P3	HTP	17.0 ms	INT8 graphExecute (async, pipelined)
P4	GPU	3.3 ms	dequant + residual + RGB $\to$ YUV420
Total per pair		28.4 ms	median; 94.9% $\leq$ 33.3 ms

TABLE IX: End-to-end per-stage latency on SM8650, ANVIL-S 1080p INT8. Medians over 54,623 consecutive frame pairs (30-minute playback).

Method	tOF	tOF ${}_{\text{dev}}$ $\downarrow$	WE $\downarrow$
GT	8.93	—	0.034
RIFE HDv3	7.42	1.62	0.027
ANVIL-S	6.91	2.09	0.027
ANVIL-M	6.87	2.09	0.027
MV Blend	6.13	2.83	0.026

TABLE X: Temporal quality on Xiph 1080p (12 sequences, 15 consecutive pairs each). tOF: mean inter-frame optical flow magnitude (RAFT-Small at 540p). WE: warping error.

Video Frame Interpolation

Video frame interpolation estimates intermediate frames between observed video frames to create smoother playback and reconstruct temporal detail.

See all 26 papers

FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation
Mar 5, 2026Ganggui Ding, Hao Chen, Xiaogang XuVideo Frame Interpolation Video Diffusion Models
⋆Zhejiang University · † Chinese University of Hong Kong
SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation
Aug 13, 2026Jisoo Jeong, Hong Cai, Jamie Menjay Lin +5Video Diffusion Models Diffusion Guidance
Qualcomm AI Research · Qualcomm Technologies, Inc.
SVI360: Spherical Video Interpolation
Jul 13, 2026Le-Kim Nguyen, Renato Martins, Pascal Vasseur +1Video Frame Interpolation
Université Bourgogne Europe, ICB UMR 6303 CNRS, France · Université de Picardie Jules Verne, MIS UR 4290, France

Mar 5, 2026J/K move · Enter open · S save

FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation

Ganggui Ding, Hao Chen, Xiaogang Xu

⋆Zhejiang University · † Chinese University of Hong Kong

Video Frame Interpolation Video Diffusion Models Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Large pre-trained video diffusion models excel in video frame interpolation but struggle to generate high fidelity frames due to reliance on intrinsic generative priors, limiting detail preservation from start and end frames. Existing methods often depend on motion control for temporal consistency, yet dense optical flow is error-prone, and sparse points lack structural context. In this paper, we propose FC-VFI for faithful and consistent video frame interpolation, supporting

4\times

x and

8\times

interpolation, boosting frame rates from 30 FPS to 120 and 240 FPS at

2560\times 1440

resolution while preserving visual fidelity and motion consistency. We introduce a temporal modeling strategy on the latent sequences to inherit fidelity cues from start and end frames and leverage semantic matching lines for structure-aware motion guidance, improving motion consistency. Furthermore, we propose a temporal difference loss to mitigate temporal inconsistencies. Extensive experiments show FC-VFI achieves high performance and structural integrity across diverse scenarios.

Abstract

Video Frame Interpolation

Explore similar work

FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation

SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation

SVI360: Spherical Video Interpolation