Overview

Audio generation studies how to create speech, music, sound effects, and spatial sound from inputs such as text, video, reference audio, or musical scores. Common approaches include diffusion and flow-matching models, autoregressive models that predict audio tokens or continuous latents, and neural or physical synthesis systems with explicit controls. Models increasingly combine tasks such as speech, music, and effects, while benchmarks assess sound quality, instruction following, timing, and alignment with visual scenes. A central challenge is balancing fidelity and controllability with diversity, reliable long-form timing, and fast streaming; objective metrics also do not always match listener judgments.

Latest papers

CardsList

140 papers8 papers in the last four weeks, down 58% on the four weeks before. 0.1% of all new papers.

  1. Direct Conditioning of Audio Diffusion Transformers on fMRI Reveals Cortical Contributions to Sound Reconstruction

    Nov 9, 2026Matteo Ciferri, Tonio Weidler, Matteo Ferrante +2Audio GenerationAudio Diffusion Models

  2. AudioCodex: Streaming Autoencoders for Live Audio

    Nov 9, 2026Marco Pasini, Javier Nistal, Cyran Aouameur +2Neural Audio CodecsAutoencoders

  3. Small Data Generative AI for Culturally Specific Artistic Creation: A Case Study on the Sonic Space of sarir

    Nov 8, 2026Arshia Sobhan, Paul Triana, Philippe PasquierGenerative ModelingAudio Generation

  4. Latent Terrain: Adapting Neural Audio Autoencoders as Design Materials in NIME

    Nov 4, 2026Shuoyang Jasper Zheng, Keigo Yoshida, Nico García-Peguinho +4AutoencodersAudio Generation

  5. MFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow Matching

    Oct 30, 2026Hao Dai, Panyu Chen, Jagmohan ChauhanText-to-Audio GenerationAudio Generation

  6. Nedimo: An Evidence-Grounded Multilingual Pipeline for AI-Generated Sleep Stories

    Oct 29, 2026Georgios Makridis, Stefanos KampourisAudio GenerationNarrative Generation

  7. Automated Parameter Sensitivity Ranking in Audio Synthesis

    Oct 28, 2026Ogulcan Kibar, Baris OzcanAudio Generation

  8. Speech LLMs That Breathe: Modeling Non-Verbal Vocalizations in Dialogue

    Oct 19, 2026Heyang Liu, Lixin Zhang, Jiayi Huang +6Nonverbal Vocalization GenerationSpeech Language Models

  9. Semantic-Critical Regions for Video-to-Audio Generation

    Oct 16, 2026Yun Fang, Wei Guo, Zhuonan Liang +2Audio GenerationAudio-Video Generation

  10. Point-and-Play: Instrument Timbre Cloning from Music Mixtures

    Oct 14, 2026Baotong Tian, Zhiyao DuanAudio GenerationMusic Generation

  11. PhysicsSliders: Learning Physical Controls for Audio Foley Generation

    Oct 13, 2026Peiwei Chang, Yongqiang Chen, Yuelan Cheng +6Sound Effects GenerationAudio Generation

  12. Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation

    Oct 12, 2026Xinlong Deng, Yong Tian, Jian Guan +3Audio GenerationConditional Generative Modeling

  13. The NVVSpeech Challenge 2026: Overview and Lessons for Speech Understanding and Generation with Nonverbal Vocalizations

    Oct 12, 2026Liumeng Xue, Yuang Cao, Jingbin Hu +12Nonverbal Vocalization GenerationAudio Understanding

  14. AlignAudio: A Unified Dual-Alignment Framework for Noise-Robust Speech-Conditioned Generation

    Oct 10, 2026Yixuan Li, Xuenan Xu, Zeyu Xie +1Audio GenerationVoice Conversion

  15. Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

    Oct 8, 2026Joseph Cameron, Alan BlackwellAudio GenerationLanguage Model Self-Assessment

  16. SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model

    Oct 8, 2026Aviad Dahan, Rajaei Khatib, Yonatan Bitton +3Audio-Video GenerationAudio Source Separation

  17. WorldSonus: Bringing Sound to Worlds

    Oct 6, 2026Pengjun Fang, Jingyi Fa, Kam Man Wu +9Audio-Video GenerationAudio Diffusion Models

  18. Smorph: Playable Sound Morphing with Diffusion Models

    Oct 5, 2026Annie Chu, Hugo Flores García, Johannes Imort +5Diffusion Model GuidanceAudio Editing

  19. Relational Synthesis: Structure-Mediated Concatenative Synthesis for Foley and Retrieval-Augmented Audio Generation

    Oct 5, 2026Keren Shao, Ayaka Kawano, Shlomo DubnovSound Effects GenerationAudio Generation

  20. AudioGAR: Bridging Reconstruction and Generation in Latent Audio Generative Models

    Oct 5, 2026Xianghong Fang, Geeyang Tay, Wentao Ma +2Neural Audio CodecsAudio Diffusion Models

  21. Game Sound-Effect Completion with Event-Level Transformation Hints

    Sep 30, 2026Xinrui Jiang, Heng YuAudio EditingAudio Diffusion Models

  22. Audible World Models: Spatially Aware Sound Generation for 3D Worlds

    Sep 29, 2026Duowen Chen, Jinjin He, Gouthaman KV +2Sound Effects Generation3D Scene Generation

  23. YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality

    Sep 27, 2026Ruibin Yuan, Jiahao Pan, Junyan Jiang +32Audio Representation LearningControllable Music Generation

  24. Generating the Unheard: Phylogeny-Guided Latent Generation for Ancestral Sound Reconstruction

    Sep 14, 2026Tianyi Xu, Shrinaath Narasimhan, Evan Gorstein +3BioacousticsAudio Generation

  25. StepAudio 3 Gen Technical Report

    Sep 11, 2026Bin Lin, Bo Zhao, Boyang Wang +68TTS SynthesisZero-Shot TTS