cs.SDMay 28, 2025

AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion

Authors: Junqi Zhao, Jinheng Zhao, Haohe Liu, Yun Chen, Lu Han, Haoyang Zhang, Xubo Liu, Mark Plumbley, 

Organizations: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK · Laboratory of Noise and Audio Research, Institute of Acoustics, Chinese Academy of Sciences, China

Abstract

Diffusion models have significantly improved the quality and diversity of text-to-audio (TTA) generation but are hindered by slow inference speed. Recent flow-based approaches have been explored for accelerating TTA generation. However, these approaches typically rely on dedicated flow-based generative frameworks and do not directly exploit existing pre-trained diffusion-based TTA models. To address this limitation while leveraging the strong generative capabilities of pre-trained diffusion models, we integrate rectified diffusion into these models to improve TTA generation efficiency. Specifically, we propose AudioTurbo, which learns first-order approximate ordinary differential equation (ODE) paths from deterministic noise-sample pairs generated by a pre-trained TTA model. Experiments show that our approach outperforms a flow-matching-based acceleration model in both generation quality and inference speed.

Figures & tables

Explore similar work

Mar 9, 2026

SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving

Text-to-audio (T2A) diffusion models generate high-quality audio but require tens of neural function evaluations (NFEs), resulting in substantial inference latency. Existing acceleration methods primarily optimize each generation trajectory independently, even when reusable acoustic structure exists in prior examples. We introduce SoundWeaver, the first training-free framework for compositional cross example acoustic reuse. Rather than generating from pure noise, SoundWeaver constructs a prompt-conditioned acoustic prior from segments distributed across cached audios and warm-starts diffusion from an intermediate noise level. We introduce an Acoustic Composition Graph (ACG), which uses neural codec representations and segment-level identification to identify candidate cross-cache transitions and structured path search to jointly optimize prompt relevance and acoustic continuity. We further establish a warm-start error bound showing how prior quality and diffusion SNR jointly govern the admissible skip depth, motivating an adaptive contextual-bandit Skip Gater that selects the warm-start level for each request. Across T2A diffusion backbones, SoundWeaver achieves 1.48-3.74x generation speedup with only a 1K-clip cache while improving generation quality.
May 1, 2026

Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation

Autoregressive (AR) models with diffusion heads have recently achieved strong text-to-audio performance, yet their iterative decoding and multi-step sampling process introduce high-latency issues. To address this bottleneck, we propose a one-step sampling framework that combines an energy-distance training objective with representation-level distillation. An energy-scoring head maps Gaussian noise directly to audio latents in one step, eliminating the need for a costly recursive diffusion sampling process, while distillation from a masked autoregressive (MAR) text-to-audio model preserves the strong conditioning learned during diffusion training. On the AudioCaps benchmark, our method consistently outperforms prior one-step baselines such as ConsistencyTTA, SoundCTM, AudioLCM and AudioTurbo, on both objective and subjective metrics, while substantially narrowing the quality gap to AR diffusion systems with multi-step sampling. Compared to the state-of-the-art AR diffusion system, IMPACT, our approach achieves up to 8.58.5x faster batch inference with highly competitive audio quality. These results demonstrate that combining energy-distance training with representation-level distillation provides an effective recipe for fast, high-quality text-to-audio synthesis.
Jul 11, 2026

FdAudio: MeanFlow-Anchored Fréchet-Distance Post-Training for One-Step Text-to-Audio Generation

While recent few-step sampling text-to-audio generation models like MeanAudio substantially accelerate generation by modeling average velocities, their strict one-step generation quality still lags significantly behind multi-step counterparts. We propose FdAudio to bridge this gap. Unlike MeanAudio, which relies solely on regression against target velocity fields, our post-training approach optimizes the final one-step distribution directly across pre-trained embedding spaces via a multi-representation Fréchet-distance (FD) loss. Crucially, to prevent the multi-step degradation that naive post-training with FD-loss causes, we introduce a MeanFlow consistency objective as a structural anchor. Results demonstrate that FdAudio establishes state-of-the-art one-step T2A generation quality among few-step systems, yielding an 11.4% reduction in FD score and a 28.8% improvement in FAD score relative to the baseline MeanAudio framework. Notably, we solve FD post-training's naive multi-step degradation issue by proposing the MeanFlow anchor, enabling a 25-step sampling path to maintain high-fidelity audio synthesis that matches or surpasses strong multi-step models at a fraction of their computational latency.