Uni-Classifier: Leveraging Video Diffusion Priors for Universal Guidance Classifier

Yujie Zhou; Pengyang Ling; Jiazi Bu; Bingjie Gao; Li Niu

cs.CVMar 20, 2026

Uni-Classifier: Leveraging Video Diffusion Priors for Universal Guidance Classifier

Authors: Yujie Zhou, Pengyang Ling, Jiazi Bu, Bingjie Gao, Li Niu

Organizations: Shanghai Jiao Tong University · University of Science and Technology of China

Abstract

In practical AI workflows, complex tasks often involve chaining multiple generative models, such as using a video or 3D generation model after a 2D image generator. However, distributional mismatches between the output of upstream models and the expected input of downstream models frequently degrade overall generation quality. To address this issue, we propose Uni-Classifier (Uni-C), a simple yet effective plug-and-play module that leverages video diffusion priors to guide the denoising process of preceding models, thereby aligning their outputs with downstream requirements. Uni-C can also be applied independently to enhance the output quality of individual generative models. Extensive experiments across video and 3D generation tasks demonstrate that Uni-C consistently improves generation quality in both workflow-based and standalone settings, highlighting its versatility and strong generalization capability.

Figures & tables

Fig. 1: Some generated images lead to substandard results for subsequent models. (a) Explicit patterns. Positions or postures with a low likelihood of movement result in static scenes or only camera motion. Incomplete objects lead to distorted 3D objects. (b) Implicit patterns. Models generate nearly static or corrupted videos from visually plausible images.

Fig. 2: The pipeline of Uni-C. The upper part illustrates the data collection process and the use of LVLM knowledge to create positive and negative samples. The lower part shows the training and inference process of the Uni-C.

Fig. 3: Statistical results regarding motion in the generated videos (I2V) with 100 random initial noises, in which the spatial mean of the motion has been subtracted.

Fig. 4: Visual comparison in workflow-based video generation.

Fig. 5: Visual comparison in workflow-based 3D generation.

Fig. 6: Visual comparison in individual text-to-video generation.

Fig. 7: Visual comparison in text-to-3D generation.

Method	Dynamic Degree	Aesthetic Quality	Imaging Quality
SD $\rightarrow$ SVD	0.0250	0.5978	0.6859
+ Image Priors	0.0320	0.5998	0.6861
+ Uni-C	0.0500	0.6113	0.6921
SD $\rightarrow$ PIA	0.0500	0.6344	0.7110
+ Image Priors	0.0675	0.6355	0.7177
+ Uni-C	0.1250	0.6408	0.7191

TABLE I: VBench metrics for video generation.

Method	IQA $(\uparrow)$	IAA $(\uparrow)$	Clip Score $(\uparrow)$
SD $\rightarrow$ SV3D	3.2246	2.9141	28.15
+ Uni-C	3.3272	3.0156	28.93
SD $\rightarrow$ ImageDream	2.8164	2.2656	28.10
+ Uni-C	2.9102	2.3555	28.22
MVDream	2.4668	1.6250	26.57
+ Uni-C	2.5883	1.7260	27.82

TABLE II: Quantitative metrics in 3D generation quality.

Task	$\lambda$
SD $\rightarrow$ SVD [ 3 ]	8
SD $\rightarrow$ PIA [ 25 ]	8
Animatediff [ 8 ]	2
LaVie [ 20 ]	1
SD $\rightarrow$ SV3D [ 17 ]	6
SD $\rightarrow$ ImageDream [ 18 ]	6

TABLE III: Guidance weight

\lambda

selection in quantitative experiments.

Fig. 8: Influence of different $\lambda$ on text-to-3D generation.

Fig. 9: Visual comparison with Image Priors model. With the assistance of Image Priors, although the object’s position is improved, the overall posture remains unsuitable for subsequent motion generation.

Fig. 10: Visual comparison with prompt engineering. Adding terms like ”full body” and ”dynamic pose” to the text prompt does not yield satisfactory results.

Fig. 11: Visual comparison with different video diffusion models for constructing image-video pair data. Uni-C and Uni-C-PIA refer to classifiers trained with labels generated using SVD and PIA, respectively. The similar results exhibited by the two models demonstrate the commonality of video diffusion priors across different video generation models.

Fig. 12: Additional qualitative results on video and 3D generation tasks.

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 493 papers

UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors
May 1, 2026Houyuan Chen, Hong Li, Xianghao Kong +8Video Diffusion Models Unified Multimodal Models
Beihang University, Beijing, China · Nanjing University, Nanjing, China · BAAI, Beijing, China +3
Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
Aug 11, 2026Yueting Zhu, Yuehao Song, Kaicheng Zhang +5Video Diffusion Models Long-Horizon Video Generation
Huazhong University of Science & Technology · Anyverse Dynamics · Horizon Robotics
UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation
Jun 17, 2026Lin Zhang, Sicheng Mo, Zefan Cai +6Autoregressive Video Generation Controllable Video Generation
University of Wisconsin Madison · University of California Los Angeles · University of California Davis +1

May 1, 2026J/K move · Enter open · S save

UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

Houyuan Chen, Hong Li, Xianghao Kong, Tianrui Zhu+7

Beihang University, Beijing, China · Nanjing University, Nanjing, China · BAAI, Beijing, China+3

Video Diffusion Models Unified Multimodal Models Multimodal Diffusion Models

Open paper AskPDF arXiv

Recent progress has shown that video diffusion models (VDMs) can be repurposed for diverse multimodal graphics tasks. However, existing methods often train separate models for each problem setting, which fixes the input-output mapping and limits the modeling of correlations across modalities. We present UniVidX, a unified multimodal framework that leverages VDM priors for versatile video generation. UniVidX formulates pixel-aligned tasks as conditional generation in a shared multimodal space, adapts to modality-specific distributions while preserving the backbone's native priors, and promotes cross-modal consistency during synthesis. It is built on three key designs. Stochastic Condition Masking (SCM) randomly partitions modalities into clean conditions and noisy targets during training, enabling omni-directional conditional generation instead of fixed mappings. Decoupled Gated LoRA (DGL) introduces per-modality LoRAs that are activated when a modality serves as the generation target, preserving the strong priors of the VDM. Cross-Modal Self-Attention (CMSA) shares keys and values across modalities while keeping modality-specific queries, facilitating information exchange and inter-modal alignment. We instantiate UniVidX in two domains: UniVid-Intrinsic, for RGB videos and intrinsic maps including albedo, irradiance, and normal; and UniVid-Alpha, for blended RGB videos and their constituent RGBA layers. Experiments show that both models achieve performance competitive with state-of-the-art methods across distinct tasks and generalize robustly to in-the-wild scenarios, even when trained on fewer than 1,000 videos. Project page: https://houyuanchen111.github.io/UniVidX.github.io/

Abstract

Generative Modeling

Explore similar work

UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation

UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation