Audio Source Separation in Reverberant Environments using $β$-divergence based Nonnegative Factorization

Mahmoud Fakhry; Piergiorgio Svaizer; Maurizio Omologo

cs.SDApr 14, 2026

Audio Source Separation in Reverberant Environments using $β$ -divergence based Nonnegative Factorization

Authors: Mahmoud Fakhry, Piergiorgio Svaizer, Maurizio Omologo

Organizations: SHINE research unit - Center for Information and Communication Technology (ICT) - Fondazione Bruno Kessler (FBK), Via Sommarive 18, 38123 Trento, Italy.

Abstract

In Gaussian model-based multichannel audio source separation, the likelihood of observed mixtures of source signals is parametrized by source spectral variances and by associated spatial covariance matrices. These parameters are estimated by maximizing the likelihood through an Expectation-Maximization algorithm and used to separate the signals by means of multichannel Wiener filtering. We propose to estimate these parameters by applying nonnegative factorization based on prior information on source variances. In the nonnegative factorization, spectral basis matrices can be defined as the prior information. The matrices can be either extracted or indirectly made available through a redundant library that is trained in advance. In a separate step, applying nonnegative tensor factorization, two algorithms are proposed in order to either extract or detect the basis matrices that best represent the power spectra of the source signals in the observed mixtures. The factorization is achieved by minimizing the $β$ -divergence through multiplicative update rules. The sparsity of factorization can be controlled by tuning the value of $β$ . Experiments show that sparsity, rather than the value assigned to $β$ in the training, is crucial in order to increase the separation performance. The proposed method was evaluated in several mixing conditions. It provides better separation quality with respect to other comparable algorithms.

Figures & tables

Fig. 1: Distance measured by different definitions of divergence, with $a=1$ .

Fig. 2: Examples of controlling the sparsity of $\mathbf{W}_{n}$ of the corrupted power spectrum by selecting the value of $\beta$ , from the left to the right, respectively, original $\mathbf{W}_{n}$ (training) with $\beta=0.9$ , estimated with $\beta=0.1$ , estimated with $\beta=0.3$ , estimated with $\beta=0.6$ , and estimated with $\beta=0.9$ .

Fig. 3: Normalized power spectra of true, corrupted, and reconstructed signals, from the left to the right respectively.

Fig. 4: Flowchart of the proposed method

Fig. 5: Normalized Likelihoods $(\mathbf{d})$ of each spectral basis vector in the library $\mathbf{U}_{lib}$ as a function of the separation iterations. First column on the left for the observed mixtures, then the five columns for iterations (1, 10, 25, 40 and 65 respectively) and rows for $3$ speech source signals.

Fig. 6: Normalized Likelihoods of each spectral basis matrix $\mathbf{U}_{z}$ as a function of the separation iterations. First column on the left for the observed mixtures, then the five columns for iterations (1, 10, 25, 40 and 65 respectively) and rows for $3$ speech source signals.

$T_{60}$ $(ms)$	$130$			$250$			$380$			Average
$K$	15	25	40	15	25	40	15	25	40	15	25	40
$\beta^{s}=$ 0.9	8.53	9.18	9.04	5.50	5.61	5.32	3.58	3.46	3.23	5.87	6.08	5.86
$\beta^{s}=$ 0.6	9.03	9.71	9.64	6.57	6.51	5.92	4.73	4.48	4.08	6.78	6.90	6.55
$\beta^{s}=$ 0.3	8.64	9.26	9.66	6.60	6.85	6.84	5.02	4.95	4.44	6.75	7.02	6.98
$\beta^{s}=$ 0.1	7.88	8.48	9.09	6.26	6.52	6.96	4.63	4.67	4.81	6.26	6.56	6.95

TABLE I: The average SDR of the simulated data as a function of

K

T_{60}

and

\beta^{s}

\beta^{t}=0.9

Fig. 7: Iterative improvement of SDR for four different detection cases.

$K$	$10$	$20$	$30$	$40$
$\beta^{s}=$ 0.9	4.52	4.82	5.10	5.43
$\beta^{s}=$ 0.6	5.60	5.70	5.79	5.88
$\beta^{s}=$ 0.3	5.72	6.62	6.78	6.62
$\beta^{s}=$ 0.1	5.67	6.46	6.42	6.43

TABLE II: The average SDR of the live-recorded data as a function of

K

and

\beta^{s}

\beta^{t}=0.9

$\mathbf{U}_{n}$	Informed			Blind
Div. Factors	$\beta^{t}=0.9,\beta^{s}=0.3$			$\beta^{e}=0.6,\beta^{s}=0.6$
Source images	$\mathbf{\tilde{c}}_{1}$	$\mathbf{\tilde{c}}_{2}$	$\mathbf{\tilde{c}}_{3}$	$\mathbf{\tilde{c}}_{1}$	$\mathbf{\tilde{c}}_{2}$	$\mathbf{\tilde{c}}_{3}$
SDR	11.27	10.42	10.22	9.63	8.67	9.17
ISR	15.13	14.56	17.76	12.90	12.82	16.50
SIR	17.26	18.02	13.12	15.28	13.97	11.95
SAR	14.99	12.99	13.68	14.32	12.33	12.76

TABLE III: The performance of informed and blind source separation of 3-Females of live-recorded mixtures from the SISEC dataset.

$T_{60}$	$130~\textrm{ms}$						$250~\textrm{ms}$						Average
Source	Three females			Three males			Three females			Three males			Average
$K$	$20$	$35$	$50$	$20$	$35$	$50$	$20$	$35$	$50$	$20$	$35$	$50$	$20$	$35$	$50$
$\beta^{s}=$ 0.9	9.45	9.88	9.90	8.43	8.83	8.95	6.34	4.21	4.14	7.70	7.87	7.53	7.98	7.70	7.63
$\beta^{s}=$ 0.6	10.08	10.20	10.07	8.57	8.90	9.10	6.72	6.11	5.93	7.77	7.97	8.25	8.29	8.30	8.34
$\beta^{s}=$ 0.3	9.80	10.70	10.56	8.07	8.60	8.94	9.05	9.40	9.01	7.39	7.82	8.11	8.58	9.13	9.16
$\beta^{s}=$ 0.1	8.95	9.74	10.18	7.63	8.15	8.50	7.88	8.16	8.86	6.76	7.33	7.78	7.81	8.35	8.83

TABLE IV: The SDR of informed separation of live-recorded stereo mixtures from SISEC,

\beta^{t}=0.9

Env.	Synthetic convolutive mixtures									Live-recorded mixtures
$K$	$20$			$35$			$50$			$20$			$35$			$50$
$\beta^{e}$	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6
$\beta^{s}=$ 0.9	5.53	5.97	6.00	5.70	5.79	5.63	5.78	5.80	5.68	7.46	7.54	5.83	7.74	7.58	7.95	7.55	7.62	7.80
$\beta^{s}=$ 0.6	5.92	6.32	6.49	5.71	6.22	5.97	6.18	6.17	5.76	7.49	7.27	6.98	7.66	7.48	7.76	7.61	7.90	$\mathbf{8.16}$
$\beta^{s}=$ 0.3	5.51	6.79	6.75	5.66	6.01	6.81	5.71	6.90	6.08	6.07	7.06	6.76	6.31	7.20	7.31	7.23	7.62	7.98
$\beta^{s}=$ 0.1	5.07	5.55	5.75	5.44	6.50	$\mathbf{7.07}$	5.34	6.36	6.20	5.26	5.59	5.41	5.48	6.76	6.74	6.26	7.10	7.11

TABLE V: The average SDR of blind separation of stereo mixtures from SISEC,

T_{60}=130~\textrm{ms}

Source	Three females									Three males
$K$	$20$			$35$			$50$			$20$			$35$			$50$
$\beta^{e}$	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6
$\beta^{s}=$ 0.9	8.10	8.27	5.98	8.70	8.35	8.83	8.48	8.80	9.04	6.82	6.81	5.68	6.78	6.81	7.07	6.62	6.44	6.56
$\beta^{s}=$ 0.6	7.75	8.35	8.12	8.52	8.70	8.72	8.72	8.99	$\mathbf{9.20}$	7.23	6.19	5.84	6.80	6.26	6.80	6.50	6.81	$\mathbf{7.13}$
$\beta^{s}=$ 0.3	7.29	7.51	7.72	7.40	8.14	8.38	8.43	8.38	9.11	4.85	6.61	5.80	6.00	6.26	6.24	6.03	6.86	6.85
$\beta^{s}=$ 0.1	5.66	6.97	6.60	5.80	7.26	8.30	7.12	7.99	8.85	4.86	4.21	4.22	5.16	6.26	5.17	5.40	6.21	5.37

TABLE VI: The SDR of blind separation of live-recorded stereo mixtures from SISEC,

T_{60}=130

ms.

Source	Three females									Three males
$K$	$20$			$35$			$50$			$20$			$35$			$50$
$\beta^{e}$	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6	1.2	0.9	0.6
$\beta^{s}=$ 0.9	3.65	3.42	3.71	3.55	3.63	3.24	3.56	3.49	3.36	3.90	3.32	3.66	4.71	3.17	3.44	4.03	3.96	3.46
$\beta^{s}=$ 0.6	3.81	4.52	4.01	4.54	4.65	3.45	5.60	3.54	3.54	3.47	3.48	3.52	5.04	4.01	3.69	4.09	4.22	4.10
$\beta^{s}=$ 0.3	3.59	6.44	6.29	4.55	4.95	$\mathbf{6.50}$	3.67	4.66	4.36	3.59	3.74	4.20	4.48	$\mathbf{5.17}$	4.34	4.87	5.07	4.47
$\beta^{s}=$ 0.1	3.63	5.74	6.19	4.61	5.86	$\mathbf{6.50}$	4.67	5.42	4.88	3.37	3.40	3.58	4.85	4.33	4.31	5.11	5.00	4.80

TABLE VII: The SDR of blind separation of live-recorded stereo mixtures from SISEC,

T_{60}=250

ms.

Source	Three females					Three males
Method	Proposed			Nesta	Cho	Proposed			Nesta	Cho
$\mathbf{U}_{n}$	Inform	Blind	Initialize			Inform	Blind	Initialize
$K$	50	50				50	50
$\beta^{t}$ / $\beta^{e}$	0.9	0.6		[ 37 ]	[ 38 ]	0.9	0.6		[ 37 ]	[ 38 ]
$\beta^{s}$	0.6	0.6				0.6	0.6
SDR	10.07	9.20	3.92	7.70	8.40	9.10	7.13	2.58	6.50	6.50

TABLE VIII: Performance comparison of live-recorded stereo mixtures from SISEC,

T_{60}=130

ms.

Source	Three females					Three males
Method	Proposed			Nesta	Cho	Proposed			Nesta	Cho
$\mathbf{U}_{n}$	Inform	Blind	Initialize			Inform	Blind	Initialize
$K$	35	35				35	35
$\beta^{t}$ / $\beta^{e}$	0.9	0.6		[ 37 ]	[ 38 ]	0.9	0.6		[ 37 ]	[ 38 ]
$\beta^{s}$	0.3	0.3				0.3	0.3
SDR	9.40	6.50	2.32	6.00	6.10	7.82	4.34	1.95	5.20	6.00

TABLE IX: Performance comparison of live-recorded stereo mixtures from SISEC,

T_{60}=250

ms.

Fig. 8: The average separation performance of the informed case of the SISEC dataset, as a function of $K$ and $\beta^{s}$ ( $\beta^{t}=0.9$ ). The horizontal axis indicates the values of the tested $\beta^{s}=[0.1~0.3~0.6~0.9]$ .

Mahmoud Fakhry received the B.Sc. and M.Sc. degrees in Electrical Engineering from the Department of Electrical Engineering, Aswan University, Egypt and the PhD degree in Information and Communication Technology from the University of Trento, Italy, in 2016. From October 2012 to November 2016, he was with the Center for Information and Communication Technology, Fondazione Bruno Kessler, Trento, Italy, as PhD student. Since December 2016, he is a postdoc researcher at the Audio Analysis Lab, Aalborg University, Denmark. His research interests include source modeling using nonnegative matrix and tensor factorization, analysis of dynamic systems, speech separation and enhancement, and voice analysis for early diagnosis of diseases.

Table 18

Piergiorgio Svaizer received the degree in electronic engineering (communications) from the University of Padua, Padua, Italy, in 1989. He is a senior researcher at the Center for Information and Communication Technology, Fondazione Bruno Kessler, Trento, Italy. His main research interests include speech analysis, enhancement and separation, front-end processing and noise reduction algorithms for speech recognition in adverse conditions, and microphone arrays for acoustic scene analysis and acoustic surveillance.

Table 19

Maurizio Omologo (M’88) received the “Laurea” degree (with honors) in electrical engineering from the University of Padua in 1984. He is the head of the SHINE (Speech-Acoustic Scene Analysis and Interpretation) research unit of the Center for Information and Communication Technology, Fondazione Bruno Kessler, Trento, Italy. His current research interests include audio and speech processing, acoustic scene analysis, and automatic speech recognition, in particular for distant-talking scenarios. He is author of more than 100 papers in major international conferences and journals in the field.

Table 20

Audio Source Separation

Audio source separation isolates individual speakers, instruments, or sound events from mixtures for listening, analysis, and audio editing.

See all 66 papers

Learning Where to Split: Dynamic Band Splitting for Audio Source Separation
Oct 10, 2026Bingzhe Zhang, Kai Li, Xiaolin HuAudio Source Separation
Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua University, Beijing 100084, China · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Chinese Institute for Brain Research (CIBR), Beijing 100010, China
Iterative Audio Separation with Mixture Consistency via MIMO Model Extension
Sep 7, 2026Yukara Ikemiya, WeiHsiang Liao, Yuki MitsufujiAudio Source Separation
SonyAI
StrADiff: A Structured Source-Wise Adaptive Diffusion Framework for Linear and Nonlinear Blind Source Separation
Apr 4, 2026Yuan-Hao WeiDiffusion Models

Oct 10, 2026J/K move · Enter open · S save

Learning Where to Split: Dynamic Band Splitting for Audio Source Separation

Bingzhe Zhang, Kai Li, Xiaolin Hu

Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua University, Beijing 100084, China · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Chinese Institute for Brain Research (CIBR), Beijing 100010, China

Audio Source Separation

Open paper AskPDF arXiv

Audio source separation models must preserve fine spectral structure while keeping the cost of the separator tractable. Band-split architectures reduce this cost by manually partitioning the spectrum into predefined frequency bands and encoding each band as a token, but the same boundaries are used for every mixture. Because informative spectral structure varies with source content, fixed boundaries can merge discriminative components or spend tokens where little resolution is needed. We introduce Dynamic Band Splitting (DBS), an end-to-end framework that adapts boundary locations and band widths to each mixture under a fixed band-token budget. DBS predicts content-conditioned boundary scores within frequency blocks, selects a prescribed number of boundaries using hard Top-

K

, and applies content-dependent pooling to obtain compact band tokens. After separation, it smooths and expands these tokens to the original STFT grid, refines them along frequency, and decodes complex masks with a frequency-shared head. DBS improved separation over the corresponding static models on DnR v2, MUSDB18-HQ, and EchoSet, with DBS-BSRNN achieving relative gain of 29.0% in average SI-SDR on DnR v2. These results and boundary analyses on DnR v2 show that content-adaptive frequency partitioning benefits multiple separator backbones under a fixed band-token budget. Code and audio demos are available at https://cslikai.cn/Dynamic-Band-Splitting

Abstract

Audio Source Separation

Explore similar work

Learning Where to Split: Dynamic Band Splitting for Audio Source Separation

Iterative Audio Separation with Mixture Consistency via MIMO Model Extension

StrADiff: A Structured Source-Wise Adaptive Diffusion Framework for Linear and Nonlinear Blind Source Separation