V2A-DPO: Omni-Preference Optimization for Video-to-Audio Generation

Nolan Chan; Timmy Gang; Yongqian Wang; Yuzhe Liang; Dingdong Wang

cs.SDMar 11, 2026

V2A-DPO: Omni-Preference Optimization for Video-to-Audio Generation

Authors: Nolan Chan, Timmy Gang, Yongqian Wang, Yuzhe Liang, Dingdong Wang

Organizations: The Chinese University of Hong Kong, Hong Kong SAR, China · National Research Council Canada, Canada · The University of Warwick, UK · Shanghai Jiao Tong University, China

Abstract

This paper introduces V2A-DPO, a novel Direct Preference Optimization (DPO) framework tailored for flow-based video-to-audio generation (V2A) models, incorporating key adaptations to effectively align generated audio with human preferences. Our approach incorporates three core innovations: (1) AudioScore-a comprehensive human preference-aligned scoring system for assessing semantic consistency, temporal alignment, and perceptual quality of synthesized audio; (2) an automated AudioScore-driven pipeline for generating large-scale preference pair data for DPO optimization; (3) a curriculum learning-empowered DPO optimization strategy specifically tailored for flow-based generative models. Experiments on benchmark VGGSound dataset demonstrate that human-preference aligned Frieren and MMAudio using V2A-DPO outperform their counterparts optimized using Denoising Diffusion Policy Optimization (DDPO) as well as pre-trained baselines. Furthermore, our DPO-optimized MMAudio achieves state-of-the-art performance across multiple metrics, surpassing published V2A models.

Figures & tables

Figure 1: Illustration of the proposed V2A-DPO framework , including (a) our proposed AudioScore to rate the generated audio with multi-dimensional scores; (b) omni-preference pair data generation combining the automatically generated preference pairs based on AudioScore with a small amount of human-annotated preference pairs; (c) curriculum learning-empowered DPO to optimize V2A models on the complex and simple pairs gradually, which are split according to complexity score $score_{c}$ .

Sys.	Method	Params	Distribution matching			Audio quality	Semantic align.	Temporal align.
Sys.	Method	Params	$\bm{\mathrm{FD_{PaSST}}}\downarrow$	$\bm{\mathrm{KL_{PANNs}}}\downarrow$	$\bm{\mathrm{KL_{PaSST}}}\downarrow$	IS $\uparrow$	IB-score $\uparrow$	DeSync $\downarrow$
1	Seeing&Hearing $\diamond$	415M	219.01	2.26	2.30	8.58	33.99	1.20
2	V-AURA $\diamond$ $\star$	695M	218.50	2.42	2.07	10.08	27.64	0.65
3	FoleyCrafter $\diamond$	1.22B	140.09	2.30	2.23	15.68	25.68	1.23
4	V2A-Mapper $\diamond$ $\star$	229M	84.57	2.69	2.56	12.47	22.58	1.23
5	ThinkSound $\dagger$	1.30B	54.92	1.32	1.52	16.03	34.13	0.46

Table 1: Video-to-audio results on the VGGSound test set.

\diamond

: the evaluation results in [ 8 ] as using the same test set.

\dagger

: reproduced using official evaluation code.

\star

: does not use text prompt during testing.

Sys.	$\bm{\beta}$	$\bm{score_{\Delta}}$	$\bm{\mathrm{FD_{PaSST}}}\downarrow$	IS $\uparrow$	IB-score $\uparrow$	DeSync $\downarrow$
0	600	0.7(81%)	51.38	19.21	34.08	0.35
1	1000	0.7(81%)	82.36	11.15	24.88	0.43
2	800		64.85	17.33	29.91	0.38
3	400		53.46	19.46	33.92	0.35
4	200		55.81	18.79	33.93	0.36
5	600	1.0(0%)	52.66	18.83	33.79	0.36

Table 2: Ablation study on the effect of the KL divergence constraint parameter

\beta

and preset threshold

score_{\Delta}

on the performance of optimized MMAudio using V2A-DPO.

Figure 2: Illustration of generation performance of V2A models.

Audio-Video Generation

Audio-video generation creates or edits synchronized sound and moving images from text, media, or other multimodal inputs.

See all 157 papers

HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion
Aug 12, 2026Wenshuo Peng, Kaipeng ZhangAudio-Video Generation Audio Diffusion Models
Electornic Engineering Department, Tsinghua University, Haidian District, Beijing, 100084 China. · OpenGVlab, Shanghai AI Laboratory, Xuhui District, Shanghai, 200030, China.
Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space
Jul 7, 2026Thanh V. T. Tran, Ngoc-Son Nguyen, Luong Tran +4Audio-Video Generation Audio-Visual Synchronization
FPT Software AI Center, Vietnam · NVIDIA Corporation, USA
SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning
May 12, 2026Xin Cheng, Xihua Wang, Ying Ba +5Audio-Video Generation Direct Preference Optimization
Renmin University of China · Westlake University

Aug 12, 2026J/K move · Enter open · S save

HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion

Wenshuo Peng, Kaipeng Zhang

Electornic Engineering Department, Tsinghua University, Haidian District, Beijing, 100084 China. · OpenGVlab, Shanghai AI Laboratory, Xuhui District, Shanghai, 200030, China.

Audio-Video Generation Audio Diffusion Models Audio-Visual Synchronization

Open paper AskPDF arXiv

Video-to-audio (V2A) generation faces significant challenges in achieving precise temporal synchronization and high perceptual quality due to the complex, ambiguous relationship between visual and auditory cues. Existing methods typically compress video inputs into single feature representations, leading to significant loss of temporal dynamics and fine-grained visual information. These approaches also rely on reconstruction-based training objectives that poorly correlate with human perceptual judgments of audio quality and appropriateness. We propose HarmoniDPO, a novel framework that integrates preference-based optimization into diffusion-based V2A generation to address these limitations. (1) Our approach leverages a dual video representation: combining global context with frame-wise features to preserve temporal dynamics and semantic detail. (2) Inspired by reinforcement learning from human feedback (RLHF), HarmoniDPO employs online Direct Preference Optimization (online-DPO) to fine-tune a diffusion-based V2A model from preference judgments, enhancing perceptual quality and alignment. (3) Additionally, we introduce Dual-scale Diffusion Search (DDS), a test time scaling algorithm that adaptively optimizes output fidelity during inference. Experiments demonstrate that HarmoniDPO outperforms state-of-the-art methods in audio-video synchronization and subjective audio quality, offering a robust solution for generating realistic, human-preferred audio from video.

Abstract

Audio-Video Generation

Explore similar work

HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion

Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space

SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning