Speech Foundation Models
Overview
Speech foundation models learn from large speech or audio-text collections, often using self-supervised methods such as masked prediction, then support tasks including automatic speech recognition, speaker analysis, emotion recognition, and text-to-speech. Common approaches include fine-tuning models such as wav2vec 2.0, HuBERT, WavLM, and Whisper; parameter-efficient adapters; and generative models built around discrete audio tokens or continuous speech latents. Researchers evaluate them on suites such as SUPERB, multilingual recognition tests, and benchmarks for robustness and speech generation. Models now span many tasks and languages, but performance can falter under noise, codec changes, domain shifts, and limited training data, leaving an open tension between broad generality, reliable adaptation, and efficient deployment.
Latest papers
66 papers9 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.