Text-to-audio (T2A) generation has advanced considerably in recent years, yet existing methods continue to face challenges in accurately rendering complex text prompts, particularly those involving intricate audio effects, and achieving precise text-audio alignment. While prior approaches have explored data augmentation, explicit timing conditioning, and reinforcement learning, overall synthesis quality remains constrained. In this work, we experiment with reinforcement learning to further enhance T2A generation quality, building on diffusion transformer (DiT)-based architectures. Our method first employs a large language model (LLM) to generate high-fidelity, richly detailed audio captions, substantially improving text-audio semantic alignment, especially for ambiguous or underspecified prompts. We then apply Group Relative Policy Optimization (GRPO), a recently introduced reinforcement learning algorithm, to fine-tune the T2A model. Through systematic experimentation with diverse reward functions (including CLAP, KL, FAD, and their combinations), we identify the key drivers of effective RL in audio synthesis and analyze how reward design impacts final audio quality. Experimental results demonstrate that GRPO-based fine-tuning yield substantial gains in synthesis fidelity and prompt adherence.
Figures & tables
Model
FDOpenl3 ↓
KLPaSST ↓
CLAP score ↑
TangoFlux
181.49 ± 5.66
2.1420 ± 0.0202
0.3259 ± 0.0254
+dataAug
161.93 ± 5.52
1.0279 ± 0.0110
0.5456 ± 0.0342
+wtRewd
158.76 ± 5.74
1.0117 ± 0.0192
0.5673 ± 0.0299
Table 1: Comparing result of baseline TangoFlux model, TangoFlux with data augmentation model, TangoFlux with weighted GRPO (wtRewd) and data augmentation model.
Model
FDOpenl3 ↓
KLPaSST ↓
CLAP score ↑
TangoFlux
161.93 ± 5.52
1.0279 ± 0.0110
0.5456 ± 0.0342
+CLAP
171.78 ± 5.45
1.0600 ± 0.0144
0.5652 ± 0.0329
+FAD
157.72 ± 5.31
1.0900 ± 0.0176
0.5531 ± 0.0271
+KL
167.45 ± 6.42
1.0210 ± 0.0183
0.5588 ± 0.0297
+wtRewd
158.76 ± 5.74
1.0117 ± 0.0192
0.5673 ± 0.0299
Table 2: Comparing result of baseline TangoFlux and its RL version with CLAP, FAD, KL, and wtRewd (weighted rewards) as rewards.
Reinforcement Learning (RL) has become an effective paradigm for enhancing Large Language Models (LLMs) and visual generative models. However, its application in text-to-audio (TTA) generation remains largely under-explored. Prior work typically employs offline methods like Direct Preference Optimization (DPO) and leverages Contrastive Language-Audio Pretraining (CLAP) models as reward functions. In this study, we investigate the integration of online Group Relative Policy Optimization (GRPO) into TTA generation. We adapt the algorithm for Flow Matching-based audio models and demonstrate that online RL significantly outperforms its offline counterparts. Furthermore, we incorporate rewards derived from Large Audio Language Models (LALMs), which can provide fine-grained scoring signals that are better aligned with human perception. With only 470M parameters, our final model, Resonate, establishes a new SOTA on TTA-Bench in terms of both audio quality and semantic alignment.
Xiquan Li, Junxi Liu, Wenxi Chen +3
X-LANCE Lab, Shanghai Jiao Tong University, China · SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China
While recent few-step sampling text-to-audio generation models like MeanAudio substantially accelerate generation by modeling average velocities, their strict one-step generation quality still lags significantly behind multi-step counterparts. We propose FdAudio to bridge this gap. Unlike MeanAudio, which relies solely on regression against target velocity fields, our post-training approach optimizes the final one-step distribution directly across pre-trained embedding spaces via a multi-representation Fréchet-distance (FD) loss. Crucially, to prevent the multi-step degradation that naive post-training with FD-loss causes, we introduce a MeanFlow consistency objective as a structural anchor. Results demonstrate that FdAudio establishes state-of-the-art one-step T2A generation quality among few-step systems, yielding an 11.4% reduction in FD score and a 28.8% improvement in FAD score relative to the baseline MeanAudio framework. Notably, we solve FD post-training's naive multi-step degradation issue by proposing the MeanFlow anchor, enabling a 25-step sampling path to maintain high-fidelity audio synthesis that matches or surpasses strong multi-step models at a fraction of their computational latency.
Text-to-audio (TTA) generation aims to synthesize realistic audio that faithfully reflects natural-language descriptions. Most TTA systems adopt a two-stage latent paradigm: an audio tokenizer is optimized for reconstruction and then frozen, after which a generative model is trained in the resulting latent space. However, reconstruction-oriented representations may be suboptimal for generation, motivating joint representation and generative learning. To this end, we introduce Unite-Audio, to our knowledge, is the first to jointly learn continuous audio representations and latent flow matching for TTA. By coupling reconstruction with self-supervised generative prediction, Unite-Audio allows the generative objective to directly shape the latent space rather than treating it as a fixed intermediate representation. We further employ Flow-GRPO post-training to improve text-conditioned generation. Experiments show competitive TTA performance with a compact latent flow model, while ablation studies confirm the benefit of jointly learning the audio representation and generative model. Audio samples are available at https://runwushi.github.io/Unite-Audio.
Runwu Shi, Kai Li, Yujin Wang +8
Institute of Science Tokyo · Tsinghua University · Wuhan University +2