cs.SDApr 27, 2026

Speech Enhancement Based on Drifting Models

Authors: Liang XuDiego Caviedes-NozalW. Bastiaan KleijnLongfei Felix YanRasmus Kongsgaard Olsson

Organizations: Victoria University of Wellington, 3 Lincoln University, New Zealand · GN Advanced Science, Denmark

Abstract

We propose Speech Enhancement based on Drifting Models (DriftSE), a novel generative framework that formulates denoising as an equilibrium problem. Rather than relying on iterative sampling, DriftSE natively achieves one-step inference by evolving the pushforward distribution of a mapping function to directly match the clean speech distribution. This evolution is driven by a Drifting Field, a learned correction vector that guides samples toward the high-density regions of the clean distribution, which naturally facilitates training on unpaired data by matching distributions rather than paired samples. We investigate the framework under two formulations: a direct mapping from the noisy observation, and a stochastic conditional generative model from a Gaussian prior. Experiments on the VoiceBank-DEMAND benchmark demonstrate that DriftSE achieves high-fidelity enhancement in a single step, outperforming multi-step diffusion baselines and establishing a new paradigm for speech enhancement.

Explore similar work

CardsList
  1. DriftSE: Speech Enhancement with Generative Drifting

    Sep 14, 2026Liang Xu, Diego Caviedes-Nozal, W. Bastiaan Kleijn +2Speech EnhancementAcoustic Representation

  2. SE-MSB: End-to-End Unpaired Speech Enhancement using Mamba Schrödinger Bridges

    Sep 22, 2026Andreas Bagge, Andreas Nymand, Michael Riis Andersen +1Speech Enhancement