Audio Diffusion Models

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 63

All topics

Beyond Token Revision: Investigating Mask-and-Replace Diffusion for Zero-Shot Text-to-Speech
Oct 7, 2026Hounsu Kim, Joonyong Park, Yuki Saito +2Zero-Shot TTS Audio Diffusion Models
KAIST, Republic of Korea · The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan
VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
Oct 7, 2026Jingqi Sun, Haozhan Tang, Shulin He +1Speech Separation Diffusion-Based Inverse Problems
Department of Computer Science and Engineering Southern University of Science and Technology, Shenzhen, China
WorldSonus: Bringing Sound to Worlds
Oct 6, 2026Pengjun Fang, Jingyi Fa, Kam Man Wu +9Audio-Video Generation Audio Diffusion Models
The Hong Kong University of Science and Technology · Noiz AI · MetaX +1
Adapting a Latent Audio Diffusion Model to Historical Guqin Recordings: A Listening-Driven Case Study
Oct 5, 2026Huanchen CaiText-to-Music Generation Music Generation Evaluation
Division of Speech, Music and Hearing (TMH) KTH Royal Institute of Technology, Stockholm, Sweden
AuraSE: Low-Hallucination Generative Speech Enhancement via Multimodal Flow Matching and Inference Policy Optimization
Oct 5, 2026Yingda Shen, Yao Qian, Yuxuan Hu +7Inference-Time Optimization Audio Diffusion Models
The Chinese University of Hong Kong, Shenzhen · Microsoft Research
Smorph: Playable Sound Morphing with Diffusion Models
Oct 5, 2026Annie Chu, Hugo Flores García, Johannes Imort +5Diffusion Model Guidance Audio Editing
Northwestern University, Evanston, IL, USA · Adobe Research, San Francisco, CA, USA · Wizdom Music, New York, NY, USA
AudioGAR: Bridging Reconstruction and Generation in Latent Audio Generative Models
Oct 5, 2026Xianghong Fang, Geeyang Tay, Wentao Ma +2Neural Audio Codecs Audio Diffusion Models
University of Toronto · Vijil
TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models
Sep 30, 2026Jiawei Yu, Jian LiuText-to-Music Generation Audio Diffusion Models
School of Computing University of Georgia
Ghost in the Encoder: Decodable Artist Identity Representations in Lyrics-to-Song Generation
Sep 30, 2026Arhan Vohra, Choenden Kyirong, Laura Ibáñez-Martínez +1Text-to-Music Generation Audio Diffusion Models
Music Technology Group, Universitat Pompeu Fabra
Game Sound-Effect Completion with Event-Level Transformation Hints
Sep 30, 2026Xinrui Jiang, Heng YuAudio Editing Audio Diffusion Models
Department of Electrical Engineering Stanford University · Department of Computer Science Stanford University
RAWD-TTS: Ratio-Free Reward Alignment for Discrete-Diffusion Voice Cloning
Sep 29, 2026Maxim Maslov, Kirill Borodin, Vasilii Kudryavtsev +2Audio Diffusion Models Zero-Shot TTS
lab260, Yerevan, Armenia · BitmanagerAI, Dubai, UAE · MTUCI, Moscow, Russia
DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation
Sep 14, 2026Zhongjie Duan, Shengchuan Gao, Hong Zhang +1Controllable Music Generation Audio Diffusion Models
Alibaba Group · Shanghai Jiao Tong University
Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis
Sep 3, 2026Sanyuan Chen, Min-Jae Hwang, Sho Inoue +12TTS Synthesis Audio Diffusion Models
FAIR at Meta
HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion
Aug 12, 2026Wenshuo Peng, Kaipeng ZhangAudio-Video Generation Audio Diffusion Models
Electornic Engineering Department, Tsinghua University, Haidian District, Beijing, 100084 China. · OpenGVlab, Shanghai AI Laboratory, Xuhui District, Shanghai, 200030, China.
Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects
Aug 5, 2026Yisu Zong, Jinjie Shi, Joshua ReissControllable Music Generation Audio Diffusion Models
Centre for Digital Music, Queen Mary University of London, UK
Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing
Aug 5, 2026Iftach Shoham, Tali Dror, Oren Gal +3Speech Editing Audio Diffusion Models
Ben-Gurion University of the Negev · University of Haifa
Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation
Aug 5, 2026Zehua Chen, Junyou Wang, Yuxuan Jiang +5Audio-Video Generation Audio Diffusion Models
Tsinghua University, Beijing, China · Hefei University of Technology, Hefei, China · Monash University, Australia +1
On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs
Aug 4, 2026Hendrik Vincent Koops, Hao Hao Tan, Elio QuintonNeural Audio Codecs Neural Representation Geometry
Music & Audio Machine Learning Lab · Universal Music Group, London, U.K.
GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model
Aug 4, 2026Guanrou Yang, Tian Tan, Qian Chen +8Flow Matching TTS Synthesis
Shanghai Jiao Tong University · Shanghai Innovation Institute · Independent Researcher +2
AnyBand: Unified Multi-Bandwidth Speech Extension via Frequency-Aware In-Context Spectral Infilling
Aug 1, 2026Junchuan Zhao, Minh Duc Vu, Bowen Zhang +1Audio Diffusion Models Speech Enhancement
School of Computing, National University of Singapore · Department of Statistics & Data Science, National University of Singapore · College of Computing and Data Science, Nanyang Technological University
VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition
Jul 30, 2026Yukun Chen, Tianrui Wang, Zhaoxi Mu +2Singing Voice Synthesis Audio Diffusion Models
Xi’an Jiaotong University · Nanyang Technological University · Ant Group +1
WanSong v1.0 Technical Report
Jul 16, 2026Binghui Chen, Pandeng Li, Yu Liu +1Controllable Music Generation Audio Diffusion Models
Wan Team, Alibaba Group
Efficient Text-to-Audio Generation via Pruning
Jul 14, 2026Arshdeep Singh, Yi Yuan, Yun Chen +2Text-to-Audio Generation Latent Diffusion Models
1King’s College London (KCL) · University of Surrey, UK
Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion
Jul 14, 2026Ben Maman, Frank Zalkow, Hans-Ulrich Berendes +3Audio Diffusion Models Audio Generation
International Audio Laboratories Erlangen, Germany · Fraunhofer IIS, Erlangen, Germany
UD-ASD: A Unified Diffusion Model for Anomalous Sound Detection
Jul 14, 2026Pengxiang Gao, Yu Qiu, Yanzhi SongAudio Diffusion Models Fault Detection
University of Science and Technology of China, China
ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models
Jul 10, 2026Sang-Hoon Lee, Ha-Yeong ChoiAudio Representation Learning Latent Diffusion Models
Department of Artificial Intelligence, Ajou University, Suwon, Korea · KT Corp., Seoul, Korea
A Quantized Native Runtime for On-Device Semantic Audio Generation
Jul 9, 2026Matteo Spanio, Antonio RodàText-to-Audio Generation Controllable Music Generation
Centro di Sonologia Computazionale (CSC) · Dept. of Information Engineering, University of Padova
EscFOA: Enhancing Spatial Learning for Visually Impaired Learners via Generative Spatial Audio in 360-Degree Educational Environments
Jul 8, 2026Ziyu Luo, Xiaowei Dai, Siying Zhu +1Audio Diffusion Models Spatial Audio
School of Computer and Artificial Intelligence, Beijing Technology and Business University, Beijing, China · College of Fundamental Studies, Beijing Vocational College of Labour and Social Security, Beijing, China
Adaptive Oscillatory Inductive Bias for Modeling Sharp Prosodic Dynamics in Diffusion-Based TTS
Jun 24, 2026Sandipan Dhar, Nirmesh J. Shah, Ashishkumar P. Gudmalwar +1TTS Synthesis Audio Diffusion Models
Media Analysis, Sony Research India
Bridging Self-Supervised Learning and Speech Enhancement: A Wav2Vec2-Conditioned Framework
Jun 21, 2026Shuubham Ojha, Carol Espy-WilsonAudio Diffusion Models Speech Enhancement
Institute for Systems Research, University of Maryland, College Park

Oct 7, 2026, eess.ASJ/K move · Enter open · S save

Beyond Token Revision: Investigating Mask-and-Replace Diffusion for Zero-Shot Text-to-Speech

Hounsu Kim, Joonyong Park, Yuki Saito, Satoru Fukayama+1

KAIST, Republic of Korea · The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan

Zero-Shot TTS Audio Diffusion Models Speech Language Models

Open paper AskPDF arXiv

Unlike autoregressive models, discrete diffusion-based models for zero-shot text-to-speech generate speech tokens in parallel and can revisit earlier predictions. Mask-and-replace training extends mask-only training by randomly replacing some tokens, and its gains are commonly attributed to self-correction, the ability to revise previously generated tokens. However, exposure to randomly perturbed context during training may itself improve generation, raising the question of whether these gains require inference-time token revision. To investigate this question, we use DeMaR, which combines mask-and-replace training with confidence-ranked mask-only sampling while preserving the total training corruption probability. Trained from scratch on LibriTTS, DeMaR achieves lower word error rates (WER) than autoregressive and mask-only diffusion baselines using the same speech tokenizer. This advantage persists when each token remains unchanged after first being unmasked. Matched training conditions on two heterogeneous speech tokenizers show that both noisy-context augmentation and replacement supervision improve WER under this restriction. These findings demonstrate training-side benefits of replacement beyond enabling inference-time token revision.

Momentum

Related topics

Latest papers 63