InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior

Weimin Bai; Suzhe Xu; Yiwei Ren; Jinhua Hao; Ming Sun; Wenzheng Chen; He Sun

cs.CVNov 18, 2025

InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior

Authors: Weimin Bai, Suzhe Xu, Yiwei Ren, Jinhua Hao, Ming Sun, Wenzheng Chen, He Sun

Organizations: Peking University · Huaqiao University · Kuaishou Technology

Abstract

Video inverse problems are fundamental to streaming, telepresence, and AR/VR, where high perceptual quality must coexist with tight latency constraints. Diffusion-based priors currently deliver state-of-the-art reconstructions, but existing approaches either adapt image diffusion models with ad hoc temporal regularizers - leading to temporal artifacts - or rely on native video diffusion models whose iterative posterior sampling is far too slow for real-time use. We introduce InstantViR, an amortized inference framework for ultra-fast video reconstruction powered by a pre-trained video diffusion prior. We distill a powerful bidirectional video diffusion model (teacher) into a causal autoregressive student that maps a degraded video directly to its restored version in a single forward pass, inheriting the teacher's strong temporal modeling while completely removing iterative test-time optimization. The distillation is prior-driven: it only requires the teacher diffusion model and known degradation operators, and does not rely on externally paired clean/noisy video data. To further boost throughput, we replace the video-diffusion backbone VAE with a high-efficiency LeanVAE via an innovative teacher-space regularized distillation scheme, enabling low-latency latent-space processing. Across streaming random inpainting, Gaussian deblurring and super-resolution, InstantViR matches or surpasses the reconstruction quality of diffusion-based baselines while running at over 35 FPS on NVIDIA A100 GPUs, achieving up to 100 times speedups over iterative video diffusion solvers. These results show that diffusion-based video reconstruction is compatible with real-time, interactive, editable, streaming scenarios, turning high-quality video restoration into a practical component of modern vision systems.

Figures & tables

Figure 1: We introduce InstantViR, a real-time video inverse problem solver that drastically outperforms slow sampling-based methods in both speed and quality. Bottom-right : At 832 $\times$ 480 resolution, our amortized framework is over 100 $\times$ faster than sampling-based baselines like SVI [ 10 ] , achieving over 35 FPS and the excellent quality. Left and Bottom-left : Qualitative examples demonstrate versatile, high-fidelity reconstruction for inpainting and deblurring, along with optional text-guided control (e.g., "pink lips", "light-blue collar").

Figure 2: Overview of the InstantViR framework. (Top) Training: We train a single-step solver $q_{\phi}$ using only degraded measurements $\boldsymbol{y}$ . The solver is optimized with two objectives: a data fidelity loss (ensuring the reconstruction matches the measurement $\boldsymbol{y}$ ) and a prior distillation loss (using a frozen video diffusion prior [ 24 ] , $s_{\theta}$ , to enforce temporal consistency and realism). (Bottom) Inference: The trained solver $q_{\phi}$ operates as a fast, feed-forward network, processing the video in a causal, block-wise, and autoregressive manner. This enables real-time, single-step streaming reconstruction with optional text guidance.

Figure 3: Qualitative comparison for video random inpainting. (Top) On the Open-Sora dataset [ 12 ] , our model reconstructs a high-fidelity and temporally consistent video from a 50% masked measurement. The zoomed-in face sequence demonstrates the stability and fine detail of our single-step result. (Bottom) We demonstrate strong zero-shot generalization on the REDS dataset [ 33 ] , our method generates a sharp, coherent video that is perceptually close to the ground truth.

Figure 4: Results for text-guided video reconstruction. InstantViR not only restores high-fidelity details from severely masked measurements (top row) but also uses text prompts to control the reconstruction. (Left) Given a masked input, our method can add specific details like a "black headband" or "wear glasses" as guided by the text. (Right) The framework can generate multi-modal, semantically distinct, and plausible outputs from the same masked input, such as reconstructing the subject with either "close eyes" or "open eyes" , demonstrating fine-grained controllability.

Method	FVD $\downarrow$			Avg. FPS $\uparrow$
Method	Inpainting	Super-Res.	Deblur	Avg. FPS $\uparrow$
DPS	375.81	711.61	783.10	<0.02
DiffIR2VR	-	311.61	-	0.12
SVI	219.90	176.60	154.38	0.29
VISION-XL	224.74	172.79	138.79	<0.17
InstantViR (Ours)	136.06	153.13	110.51	13.91

Table 1: Quantitative results of temporal quality and inference speed. The former is evaluated with FVD

\downarrow

and the latter is with FPS

\uparrow

. Best results are in bold , suboptimal are underlined .

Method	50% Random Inpainting			4 $\times$ Super-Resolution			Gaussian Deblur
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
DPS	27.68	0.92	0.32	22.78	0.91	0.46	23.54	0.88	0.46
DiffIR2VR	-	-	-	33.44	0.92	0.33	-	-	-
SVI	29.42	0.90	0.17	33.85	0.96	0.17	26.93	0.89	0.31
VISION-XL	30.83	0.95	0.25	35.69	0.98	0.24	30.03	0.93	0.28
InstantViR (Ours)	30.54	0.97	0.12	34.91	0.96	0.23	31.85	0.97	0.17

Table 2: Quantitative results of spatial quality. Metrics include PSNR, SSIM, LPIPS. Best results are in bold , suboptimal are underlined .

Panel of Figure 5 — Figure 5: Qualitative comparison for video Gaussian deblurring.

Panel of Figure 6 — Figure 6: Qualitative comparison for video super-resolution (4x).

Panel of Figure 1 — Figure 1: Video Inpainting qualitative comparison. Each row shows a complete sequence reconstructed by a specific method. InstantViR (both WanVAE Ours and LeanVAE Ours † variants) produces coherent content for every frame while requiring only a single feed-forward pass.

Panel of Figure 2 — Figure 2: Video Deblurring qualitative comparison. Rows correspond to different methods; columns show consecutive frames covering the entire clip. InstantViR (both WanVAE Ours and LeanVAE Ours † variants) restores fine structures consistently across time, outperforming slower diffusion-based baselines.

Panel of Figure 3 — Figure 3: Video Super-Resolution (4 $\times$ ) qualitative comparison. InstantViR restores temporally consistent structures, outperforming slower diffusion-based baselines in both sharpness and coherence.

Panel of Figure 4 — Figure 4: Text-Guided Video Deblurring. InstantViR can generate diverse outcomes from the same blurred input based on text prompts, adding specific features like glasses or changing eye color while maintaining temporal coherence.

Diffusion-Based Inverse Problems

Diffusion-based inverse problems use generative priors to recover images and physical fields from incomplete, noisy, or indirect measurements.

See all 145 papers

Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models
May 20, 2026Taesung Kwon, Jonghyun Park, Hyungjin Chung +1Video Diffusion Models Video Restoration
KAIST · EverEx
FastVR: Efficient Streaming Video Restoration with One-Step Diffusion
Sep 29, 2026Xiaoxu Chen, Qin Yang, Haoran Bai +2Diffusion Transformer Video Restoration
Alibaba Group · Xidian University
InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution
Mar 27, 2026Jintong Hu, Bin Chen, Zhenyu Hu +3Video Super-Resolution Diffusion Models
Insta360 Research · Peking University · Wuhan University

May 20, 2026J/K move · Enter open · S save

Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models

Taesung Kwon, Jonghyun Park, Hyungjin Chung, Jong Chul Ye

KAIST · EverEx

Video Diffusion Models Video Restoration Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Diffusion models provide powerful priors for zero-shot video inverse problems, but their real-time deployment is hindered by two inefficiencies: high initial latency caused by holistic video restoration, and low throughput resulting from multiple VAE passes to enforce measurement consistency in pixel space. To overcome these limitations, we propose Autoregressive Video Inverse problem Solver (AVIS). The AVIS framework leverages autoregressive video diffusion models to restore videos in a streaming manner, naturally eliminating latency bottlenecks. Specifically, AVIS initializes reverse diffusion with a measurement-consistent estimate, reducing the required sampling steps. Compared to leading non-autoregressive solvers, AVIS drastically reduces initial latency from 114s to 4s and increases throughput from 0.71 to 1.18 FPS while achieving superior restoration quality. We further introduce a highly accelerated variant, dubbed AVIS Flash, that enforces measurement consistency solely on the first chunk. AVIS Flash substantially boosts throughput to 5.91 FPS on a single RTX 4090 GPU while maintaining competitive performance and achieving a favorable efficiency-performance trade-off, paving the way toward real-time deployment.

Abstract

Diffusion-Based Inverse Problems

Explore similar work

Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models

FastVR: Efficient Streaming Video Restoration with One-Step Diffusion

InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution