Organizations: The Chinese University of Hong Kong, Hong Kong SAR, China · National Research Council Canada, Canada · The University of Warwick, UK · Shanghai Jiao Tong University, China
This paper introduces V2A-DPO, a novel Direct Preference Optimization (DPO) framework tailored for flow-based video-to-audio generation (V2A) models, incorporating key adaptations to effectively align generated audio with human preferences. Our approach incorporates three core innovations: (1) AudioScore-a comprehensive human preference-aligned scoring system for assessing semantic consistency, temporal alignment, and perceptual quality of synthesized audio; (2) an automated AudioScore-driven pipeline for generating large-scale preference pair data for DPO optimization; (3) a curriculum learning-empowered DPO optimization strategy specifically tailored for flow-based generative models. Experiments on benchmark VGGSound dataset demonstrate that human-preference aligned Frieren and MMAudio using V2A-DPO outperform their counterparts optimized using Denoising Diffusion Policy Optimization (DDPO) as well as pre-trained baselines. Furthermore, our DPO-optimized MMAudio achieves state-of-the-art performance across multiple metrics, surpassing published V2A models.
Figures & tables
Figure 1: Illustration of the proposed V2A-DPO framework , including (a) our proposed AudioScore to rate the generated audio with multi-dimensional scores; (b) omni-preference pair data generation combining the automatically generated preference pairs based on AudioScore with a small amount of human-annotated preference pairs; (c) curriculum learning-empowered DPO to optimize V2A models on the complex and simple pairs gradually, which are split according to complexity score scorec .
Sys.
Method
Params
Distribution matching
Audio quality
Semantic align.
Temporal align.
FDPaSST↓
KLPANNs↓
KLPaSST↓
IS ↑
IB-score ↑
DeSync ↓
1
Seeing&Hearing ⋄
415M
219.01
2.26
2.30
8.58
33.99
1.20
2
V-AURA ⋄⋆
695M
218.50
2.42
2.07
10.08
27.64
0.65
3
FoleyCrafter ⋄
1.22B
140.09
2.30
2.23
15.68
25.68
1.23
4
V2A-Mapper ⋄⋆
229M
84.57
2.69
2.56
12.47
22.58
1.23
5
ThinkSound †
1.30B
54.92
1.32
1.52
16.03
34.13
0.46
Table 1: Video-to-audio results on the VGGSound test set. ⋄ : the evaluation results in [ 8 ] as using the same test set. † : reproduced using official evaluation code. ⋆ : does not use text prompt during testing.
Sys.
β
scoreΔ
FDPaSST↓
IS ↑
IB-score ↑
DeSync ↓
0
600
0.7(81%)
51.38
19.21
34.08
0.35
1
1000
0.7(81%)
82.36
11.15
24.88
0.43
2
800
64.85
17.33
29.91
0.38
3
400
53.46
19.46
33.92
0.35
4
200
55.81
18.79
33.93
0.36
5
600
1.0(0%)
52.66
18.83
33.79
0.36
Table 2: Ablation study on the effect of the KL divergence constraint parameter β and preset threshold scoreΔ on the performance of optimized MMAudio using V2A-DPO.
Figure 2: Illustration of generation performance of V2A models.
Video-to-audio (V2A) generation faces significant challenges in achieving precise temporal synchronization and high perceptual quality due to the complex, ambiguous relationship between visual and auditory cues. Existing methods typically compress video inputs into single feature representations, leading to significant loss of temporal dynamics and fine-grained visual information. These approaches also rely on reconstruction-based training objectives that poorly correlate with human perceptual judgments of audio quality and appropriateness. We propose HarmoniDPO, a novel framework that integrates preference-based optimization into diffusion-based V2A generation to address these limitations. (1) Our approach leverages a dual video representation: combining global context with frame-wise features to preserve temporal dynamics and semantic detail. (2) Inspired by reinforcement learning from human feedback (RLHF), HarmoniDPO employs online Direct Preference Optimization (online-DPO) to fine-tune a diffusion-based V2A model from preference judgments, enhancing perceptual quality and alignment. (3) Additionally, we introduce Dual-scale Diffusion Search (DDS), a test time scaling algorithm that adaptively optimizes output fidelity during inference. Experiments demonstrate that HarmoniDPO outperforms state-of-the-art methods in audio-video synchronization and subjective audio quality, offering a robust solution for generating realistic, human-preferred audio from video.
Video-to-audio (V2A) generation aims to synthesize realistic audio that is both semantically consistent with and temporally synchronized to a silent video. Despite recent progress, many methods still rely on multi-stage training, resulting in high computational costs and long runtimes, or transform visual input into text to leverage pretrained text-to-audio models, sacrificing fine-grained temporal cues. To overcome these limitations, we propose Flowley, an end-to-end, single-stage training architecture that produces soundtracks by combining visual features with textual prompts. Crucially, we introduce Progressive Soft-masked Cross-Attention, which embeds audio-visual synchronization directly within its attention mechanism, adding zero additional computational cost compared to standard attention layers. We further observe that existing V2A benchmarks lack sound-oriented descriptive captions, which can potentially degrade the quality of the synthesized audio. To remedy this, we propose SoundCap, a plug-and-play pipeline for creating detailed, sound-aware captions that guide the model. Remarkably, without integrating any pretrained audio-visual alignment modules, Flowley achieves state-of-the-art performance on VGGSound across multiple metrics. Moreover, by incorporating SoundCap, we further exceed the performance of the strongest existing close-sourced methods in terms of audio quality in the zero-shot setting.
Thanh V. T. Tran, Ngoc-Son Nguyen, Luong Tran +4
FPT Software AI Center, Vietnam · NVIDIA Corporation, USA
Recent advancements in video-audio joint generation have achieved remarkable success in semantic correspondence. However, achieving precise temporal synchronization, which requires fine-grained alignment between audio events and their visual triggers, remains a challenging problem. The post-training method for joint generation is largely dominated by Supervised Fine-Tuning, but the commonly used Mean Squared Error loss provides insufficient penalties for subtle temporal misalignments. Direct Preference Optimization offers an alternative by introducing explicit misaligned counterparts to better improve temporal sensitivity. In this paper we propose a post-training framework SyncDPO, leveraging DPO to improve the temporal sensitivity of V-A joint generation. Conventional DPO pipelines typically depend on costly sampling-and-ranking procedures to construct preference pairs, resulting in substantial computational cost. To improve efficiency, we introduce a suite of on-the-fly rule-based negative construction strategies that distort temporal structures without incurring additional annotation or sampling. We demonstrate that the temporal alignment capability can be effectively reinforced by providing explicit negative supervision through temporally distorted V-A pairs. Accordingly, we implement a curriculum learning strategy that progressively increases the difficulty of negative samples, transitioning from coarse misalignment to subtle inconsistencies. Extensive objective and subjective experiments across four diverse benchmarks, ranging from ambient sound videos to human speech videos, demonstrate that SyncDPO significantly outperforms other methods in improving model's temporal alignment capability. It also demonstrates superior generalization on out-of-distribution benchmark by capturing intrinsic motion-sound dynamics. Demo and code is available in https://syncdpo.github.io/syncdpo/.