Overview

Speech foundation models learn from large speech or audio-text collections, often using self-supervised methods such as masked prediction, then support tasks including automatic speech recognition, speaker analysis, emotion recognition, and text-to-speech. Common approaches include fine-tuning models such as wav2vec 2.0, HuBERT, WavLM, and Whisper; parameter-efficient adapters; and generative models built around discrete audio tokens or continuous speech latents. Researchers evaluate them on suites such as SUPERB, multilingual recognition tests, and benchmarks for robustness and speech generation. Models now span many tasks and languages, but performance can falter under noise, codec changes, domain shifts, and limited training data, leaving an open tension between broad generality, reliable adaptation, and efficient deployment.

Latest papers

CardsList

71 papers5 papers in the last four weeks, down 64% on the four weeks before. 0.0% of all new papers.

  1. Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioning

    Aug 12, 2026Xulin Fan, Jialu Li, Mohammad Nur Hossain Khan +4Speech Foundation ModelsDomain Generalization

  2. MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages

    Aug 5, 2026Qiongqiong Wang, Ai Ti Aw, Nancy F. Chen +18Speech Foundation ModelsSpeech Processing

  3. dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model

    Aug 2, 2026Hankun Wang, Bohan Li, Shi Lian +7Speech Foundation ModelsSpeech Editing

  4. Normal-Anchored First-Order Model-Agnostic Meta-Learning based Whisper Fine-Tuning for Enhancing Fairness of Cleft Lip and Palate Speech Recognition

    Jul 31, 2026Susmita Bhattacharjee, Jagabandhu Mishra, H. S. Shekhawat +2Speech Foundation ModelsMeta-Learning

  5. Enhancing Law-Enforcement Audio Transcription: A LoRA-Based Adaptation of Whisper for BWC Footage

    Jul 27, 2026Vivek Senthil, Ernest FokouéSpeech Foundation ModelsAudio Understanding

  6. Improving Zero-Shot Phonetic Classification through Language-Agnostic Articulatory Features

    Jul 26, 2026Ryo Magoshi, Jaeyoung Lee, Shinsuke Sakai +1Zero-Shot LearningSpeech Foundation Models

  7. Rethinking Speech Foundation Model Fine-tuning: Better SFT or Better Match?

    Jul 15, 2026Wangjin Zhou, Yizhou Zhang, Yichi Wang +1Supervised Fine-TuningFine-Tuning

  8. GigaAM Multilingual: Foundation Model for Underrepresented Languages

    Jul 11, 2026Andrei Kuzmenko, Alexandr Maximenko, Aleksandr Kutsakov +5Speech Foundation ModelsSpeech Processing

  9. wav2VOT: Automatic estimation of voice onset time, closure duration, and burst realisation with wav2vec2

    Jun 27, 2026James Tanner, Morgan Sonderegger, Jane Stuart-Smith +2Speech Foundation ModelsSpeech Processing

  10. Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering

    Jun 10, 2026Haoning Xu, Zhaoqing Li, Huimeng Wang +4Speech Foundation ModelsClustering

  11. Fast Speech Foundation Model Distillation Using Interleaved Stacking

    Jun 10, 2026Eungbeom Kim, Kyogu LeeSpeech Foundation ModelsSpeech Processing

  12. dots.tts Technical Report

    Jun 5, 2026Shi Lian, Changtao Li, Bohan Li +6TTS SynthesisSpeech Foundation Models

  13. VoxCPM2 Technical Report

    Jun 5, 2026Yixuan Zhou, Guoyang Zeng, Xin Liu +15Speech Foundation ModelsMultilingual TTS Synthesis

  14. Do speech foundation models perceive speaker similarity as humans do?

    Jun 4, 2026Minoru Kishi, Hayato Yagi, Shinnosuke Takamichi +1Audio Representation LearningSpeech Foundation Models

  15. HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding

    May 28, 2026Bohan Li, Shi Lian, Hankun Wang +6Audio Representation LearningSpeech Foundation Models

  16. A Study of ASR Adaptation and Representation Dimensionality Reduction in Persian Speech Emotion Recognition Using Whisper

    May 26, 2026Ali Shendabadi, Parnia Izadirad, Mostafa SalehiSpeech Emotion RecognitionDimensionality Reduction

  17. WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling

    May 7, 2026Guanrou Yang, Tian Tan, Qian Chen +12Audio Representation LearningTTS Synthesis

  18. Alethia: A Foundational Encoder for Voice Deepfakes

    Apr 30, 2026Yi Zhu, Brahmi Dwivedi, Jayaram Raghuram +1Audio Representation LearningAudio Deepfake Detection

  19. Raon-Speech Technical Report

    Apr 8, 2026Beomsoo Kim, Changho Choi, Dohyun Kim +23Speech Language ModelsSpeech Foundation Models

  20. HARNESS: Lightweight Distilled Arabic Speech Foundation Models

    Mar 31, 2026Vrunda N. Sukhadia, Shammur Absar ChowdhurySpeech Foundation ModelsSpeech Processing

  21. Investigation on the Robustness of Acoustic Foundation Models on Post Exercise Speech

    Mar 29, 2026Xiangyuan Xue, Yuyu Wang, Ruijie Yao +3Robust ASRAutomatic Speech Recognition

  22. Ara-Best-RQ: Multi Dialectal Arabic SSL

    Mar 23, 2026Haroun Elleuch, Ryan Whetten, Salima Mdhaffar +2Arabic DialectsSpeech Processing

  23. MOSS-TTS Technical Report

    Mar 18, 2026Yitian Gong, Botian Jiang, Yiwei Zhao +23TTS SynthesisSpeech Foundation Models