Overview

Speech foundation models learn from large speech or audio-text collections, often using self-supervised methods such as masked prediction, then support tasks including automatic speech recognition, speaker analysis, emotion recognition, and text-to-speech. Common approaches include fine-tuning models such as wav2vec 2.0, HuBERT, WavLM, and Whisper; parameter-efficient adapters; and generative models built around discrete audio tokens or continuous speech latents. Researchers evaluate them on suites such as SUPERB, multilingual recognition tests, and benchmarks for robustness and speech generation. Models now span many tasks and languages, but performance can falter under noise, codec changes, domain shifts, and limited training data, leaving an open tension between broad generality, reliable adaptation, and efficient deployment.

Latest papers

CardsList

71 papers5 papers in the last four weeks, down 64% on the four weeks before. 0.0% of all new papers.

  1. Walrus Technical Report

    Nov 10, 2026Alkis Koudounas, Osamu Take, Hayato Futami +2Speech Foundation ModelsAudio-Language Models

  2. Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models

    Nov 5, 2026Yanru Huo, Zhikang Niu, Wenming Tu +7Speech Foundation ModelsDistribution Matching Distillation

  3. Audio System One Model: Typed Probabilistic Perception of Emotion in Speech

    Oct 14, 2026Jaime Bellver-Soler, Iván Martín-Fernández, Anmol Guragain +1Speech Emotion RecognitionSpeech Foundation Models

  4. Fish Audio Drama 3 Technical Report

    Oct 13, 2026Dawei Han, Shidong Li, Shijia Liao +13TTS SynthesisControllable Speech Generation

  5. Nufo: A Benchmark for Ewe Automatic Speech Recognition

    Oct 12, 2026Christ Kekeli Keli, Steve Sanogo, Patrice Sebastiano +2Automatic Speech RecognitionLow-Resource Speech Recognition

  6. Backdooring Acoustic Foundation Models for Physically Realizable Triggers

    Oct 7, 2026Zebin Yun, Eyal Ronen, Mahmood SharifBackdoor AttacksAdversarial Attacks

  7. Neural Representations, Natural Connections: What Transfers From Human Speech Foundation Models to Animal Vocalizations?

    Oct 5, 2026Tomás Arias-Vergara, Christopher Hauer, Héloïse Brotier +3Audio Representation LearningSpeech Foundation Models

  8. Transferable Adversarial Robustness for Speech Foundation Models via Hierarchical Stabilization

    Oct 4, 2026Aref Mousavi, Shahab Sherafat, Kiarash Kiani Feriz +4Adversarial Representation LearningSpeech Foundation Models

  9. Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition

    Sep 24, 2026Asmee Mishra, Mengjie Qian, Brechtje Post +1Speech Foundation ModelsLow-Resource Speech Recognition

  10. Temporal Taxation Compounds Under Post-Training Compression of Whisper Models

    Sep 23, 2026Srishti Ginjala, Eric Fosler-Lussier, Christopher W. Myers +1ASR EvaluationSpeech Foundation Models

  11. Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery

    Sep 23, 2026Rasmus Aagaard, Nicki Skafte DetlefsenSpeech Foundation ModelsAutomatic Speech Recognition

  12. Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction

    Sep 21, 2026Lujia Bao, Qian Chen, Luyao Cheng +15Tool-Augmented Language Model AgentsSpeech Foundation Models

  13. AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models

    Sep 21, 2026Natarajan Balaji Shankar, Zilai Wang, Zihan Wang +3Speech Foundation Models

  14. Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation

    Sep 16, 2026Simon Pals, Cristian Tejedor-GarciaParkinson's Disease Detection from SpeechParkinson's Disease

  15. Listening for Airway Stenosis: A Foundation Model-Based Method for Rapid and Accessible Detection

    Sep 14, 2026Jean Groeninger, Zihao Zhao, Juliana de Castilhos +2Speech Foundation ModelsAudio Classification

  16. CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models

    Sep 14, 2026Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva +5Speech Foundation ModelsSpeech Quality Assessment

  17. Do speech foundation models really learn words?

    Sep 9, 2026Robin Huo, Ewan DunbarRepresentation LearningSpeech Foundation Models

  18. Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models

    Sep 9, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer InterpretabilityCircuit Discovery

  19. BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

    Sep 9, 2026Shivam Singh, Aditya Yadavalli, Catherine Arnett +1Speech Foundation ModelsAutomatic Speech Recognition

  20. KanAdapter: A Kolmogorov-Arnold Network-based Plug-and-Play Module for Efficient Fine-tuning of Foundation Speech Models

    Sep 4, 2026Phuong Tuan Dat, Phuong Khai Minh, Tran Huy DatAdapter TuningParameter-Efficient Fine-Tuning

  21. Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection

    Sep 3, 2026Maryam Abbasihafshejani, Murtuza JadliwalaSpeech Foundation ModelsAutomatic Speech Recognition

  22. Fine-Tuning Whisper for Automatic Speech Recognition in Baniwa: A Preliminary Study

    Aug 26, 2026Leonardo Duart, Tiago Fonseca, Thiago ChacónAutomatic Speech RecognitionLow-Resource Speech Recognition