Absorbing Discrete Diffusion for Speech Enhancement

Philippe Gonzalez

cs.SDFeb 25, 2026

Absorbing Discrete Diffusion for Speech Enhancement

Authors: Philippe Gonzalez

Organizations: Department of Health Technology, Technical University of Denmark

Abstract

Inspired by recent developments in neural speech coding and diffusion-based language modeling, we tackle speech enhancement by modeling the conditional distribution of clean speech codes given noisy speech codes using absorbing discrete diffusion. The proposed approach, which we call ADDSE, leverages both the expressive latent space of neural audio codecs and the non-autoregressive sampling procedure of diffusion models. To efficiently model the hierarchical structure of residual vector quantization codes, we propose RQDiT, which combines techniques from RQ-Transformer and diffusion Transformers for non-autoregressive modeling. Results show competitive performance in terms of non-intrusive objective metrics on two datasets, especially at low signal-to-noise ratios and with few sampling steps. Code and audio examples are available online.

Figures & tables

Figure 1: Proposed ADDSE framework.

Panel of Figure 2 — Figure 2: RQDiT architecture. Two DiTs are applied along the NAC frame and depth dimensions, respectively.

		Libri-TUT										Clarity-FSD50K
	Params. (M)	PESQ $\uparrow$	ESTOI $\uparrow$	SDR $\uparrow$	MCD $\downarrow$	DNSMOS $\uparrow$	NISQA $\uparrow$	UTMOS $\uparrow$	SCOREQ $\uparrow$	LPS $\uparrow$	SBS $\uparrow$	PESQ $\uparrow$	ESTOI $\uparrow$	SDR $\uparrow$	MCD $\downarrow$	DNSMOS $\uparrow$	NISQA $\uparrow$	UTMOS $\uparrow$	SCOREQ $\uparrow$	LPS $\uparrow$	SBS $\uparrow$
Noisy	–	1.31	0.61	4.96	6.76	2.92	2.03	2.44	2.47	0.82	0.73	1.34	0.74	4.96	7.52	2.90	2.46	2.42	2.46	0.82	0.69
Clean	–	4.64	1.00	$\infty$	0.00	3.73	3.31	3.06	3.98	1.00	1.00	4.64	1.00	$\infty$	0.00	3.79	3.69	3.38	4.53	1.00	1.00
Clean-NAC	45	2.93	0.89	3.51	2.88	3.75	3.33	3.01	3.90	0.97	0.96	2.85	0.91	5.50	3.08	3.80	3.66	3.31	4.47	0.97	0.96
Conv-TasNet	5	2.44	0.85	15.88	5.03	3.58	3.27	2.68	3.49	0.93	0.85	2.54	0.90	18.18	5.00	3.47	3.32	2.92	3.69	0.91	0.84
BSRNN	13	2.77	0.88	16.77	5.23	3.79	3.59	3.16	3.94	0.96	0.89	3.11	0.94	20.97	4.39	3.72	3.74	3.32	4.22	0.95	0.88

Table 1: Average objective metrics for each model and each dataset.

Figure 3: Top and middle rows: Non-intrusive metrics as a function of

N_{\text{steps}}

. Bottom left: Number of function evaluations as a function of

N_{\text{steps}}

. Bottom right: DCE on test data.

Figure 4: Non-intrusive metrics as a function of input SNR.

Speech Enhancement

Speech enhancement improves intelligibility and listening quality by suppressing noise, reverberation, and competing speakers in recorded or live audio.

See all 123 papers

Bridging Self-Supervised Learning and Speech Enhancement: A Wav2Vec2-Conditioned Framework
Jun 21, 2026Shuubham Ojha, Carol Espy-WilsonAudio Diffusion Models Speech Enhancement
Institute for Systems Research, University of Maryland, College Park
Speech Enhancement Based on Drifting Models
Apr 27, 2026Liang Xu, Diego Caviedes-Nozal, W. Bastiaan Kleijn +2One-Step Generative Modeling Speech Enhancement
Victoria University of Wellington, 3 Lincoln University, New Zealand · GN Advanced Science, Denmark
SLICE: Speech Enhancement via Layer-wise Injection of Conditioning Embeddings
Mar 5, 2026Seokhoon Moon, Kyudan Jung, Jaegul ChooSpeech Enhancement Audio Diffusion Models
KAIST · KAIST AI

Jun 21, 2026J/K move · Enter open · S save

Bridging Self-Supervised Learning and Speech Enhancement: A Wav2Vec2-Conditioned Framework

Shuubham Ojha, Carol Espy-Wilson

Institute for Systems Research, University of Maryland, College Park

Audio Diffusion Models Speech Enhancement Conditional Diffusion Models

Open paper AskPDF arXiv

Diffusion models show potential for speech enhancement but lack linguistic guidance. We condition a diffusion-based model on wav2vec 2.0 features from noisy input, injected at the U-Net bottleneck via Feature-wise Linear Modulation (FiLM). Phonetic representations from wav2vec 2.0 features of degraded speech, anchor the reverse diffusion process. While a frozen wav2vec 2.0 encoder extracts features, a learned FiLM generator produces scale and shift parameters modulating the bottleneck with minimal overhead. Motivated by the optimal Bayesian causal estimator under a linear-Gaussian state-space model, FiLM coefficients are aggregated via exponential smoothing for temporal compression. Evaluation on VoiceBank-DEMAND and LibriMix shows competitive performance against the unconditioned baseline in PESQ, STOI, SI-SDR and DNSMOS. We consistently record an improvement of 0.4 on PESQ score, suggesting self-supervised representations effectively condition diffusion-based speech enhancement.

Abstract

Speech Enhancement

Explore similar work

Bridging Self-Supervised Learning and Speech Enhancement: A Wav2Vec2-Conditioned Framework

Speech Enhancement Based on Drifting Models

SLICE: Speech Enhancement via Layer-wise Injection of Conditioning Embeddings