cs.SDNov 15, 2026

MOUSSE: Multilingual Overlapped Speech Separation with Predictive Uncertainty Estimation

Authors: Su Yeon Shim, Yejin Jeon, David Ifeoluwa Adelani, Young-Joo Suh

Organizations: KT Corporation · Mila - Quebec AI Institute · McGill University · Canada CIFAR AI Chair · 

Abstract

Recent generative speech separation methods have shown promising performance, yet mixtures of overlapping speech in different languages remain underexplored. To this end, we first curate MuSSeD, an English-Korean benchmark comprising two- to four- speaker mixtures across varying overlap ratios. We further propose MOUSSE, a flow matching framework that incorporates boundary-aware training and frame-wise predictive uncertainty through a permutation-invariant Gaussian negative log-likelihood. The predicted scale is further used to adapt classifier-free guidance during sampling. MOUSSE achieves 13.18 dB SI-SDRi and 0.175 ESTOIi, outperforming discriminative and flow matching-based separation baselines, with a 1.58 dB SI-SDRi improvement over the strongest model.

Figures & tables

Explore similar work

CardsList
  1. SLM-SS: Token-Based Speech Separation with Speech Language Models

    Oct 20, 2026Chenda Li, Tianhua Li, Yiming Chen +4Speech SeparationSpeech Language Models

  2. PromptSep-Speaker: Generative Target Speaker Extraction via Multimodal Prompting and Semantic Alignment

    Oct 16, 2026Yutong Wen, Ke Chen, Prem Seetharaman +5Target Speaker Extraction

  3. MeCo: One-Step MeanFlow-based Corrector for Multi-Channel Speech Separation

    Jun 8, 2026Dohwan Kim, Jung-Woo ChoiFlow MatchingMeanFlow