Paper ID: 2202.07273

SpeechPainter: Text-conditioned Speech Inpainting

Zalán Borsos, Matt Sharifi, Marco Tagliasacchi

We propose SpeechPainter, a model for filling in gaps of up to one second in speech samples by leveraging an auxiliary textual input. We demonstrate that the model performs speech inpainting with the appropriate content, while maintaining speaker identity, prosody and recording environment conditions, and generalizing to unseen speakers. Our approach significantly outperforms baselines constructed using adaptive TTS, as judged by human raters in side-by-side preference and MOS tests.

Submitted: Feb 15, 2022