Overview

Speech processing studies how computers can recognize words, identify speakers, understand vocal cues, and generate or improve speech. Common approaches include self-supervised speech encoders and Transformer or CTC models, alongside diffusion and flow-matching methods for synthesis, conversion, enhancement, and separation. Researchers evaluate these systems on tasks such as conversational transcription, speaker diarization, pronunciation assessment, and clinical speech analysis, increasingly including noisy, multilingual, and low-resource settings. A central challenge is making systems reliable across speakers, languages, and recording conditions while balancing accuracy and naturalness against latency, computation, privacy, and fairness.

Latest papers

422 papers18 papers in the last four weeks, down 70% on the four weeks before. 0.1% of all new papers.

Oct 19, 2021

A mathematical model of the vowel space

The articulatory-acoustic relationship is many-to-one and non linear and this is a great limitation for studying speech production. A simplification is proposed to set a bijection between the vowel space (f1, f2) and the parametric space of different vocal tract models. The generic area function model is based on mixtures of cosines allowing the generation of main vowels with two formulas. Then the mixture function is transformed into a coordination function able to deal with articulatory parameters. This is shown that the coordination function acts similarly with the Fant's model and with the 4-Tube DRM derived from the generic model.
Aug 12, 2019

Personal VAD: Speaker-Conditioned Voice Activity Detection

In this paper, we propose "personal VAD", a system to detect the voice activity of a target speaker at the frame level. This system is useful for gating the inputs to a streaming on-device speech recognition system, such that it only triggers for the target user, which helps reduce the computational cost and battery consumption, especially in scenarios where a keyword detector is unpreferable. We achieve this by training a VAD-alike neural network that is conditioned on the target speaker embedding or the speaker verification score. For each frame, personal VAD outputs the probabilities for three classes: non-speech, target speaker speech, and non-target speaker speech. Under our optimal setup, we are able to train a model with only 130K parameters that outperforms a baseline system where individually trained standard VAD and speaker recognition networks are combined to perform the same task.