cs.SDNov 1, 2026
SaveSpatially-Guided Speaker Extraction Based on Relative Position Cues
Organizations: Bar-Ilan University, Israel · OriginAI, Israel
Abstract
We present a multi-channel target speaker extraction (TSE) framework that is guided by relative spatial cues rather than explicit speaker-dependent references, such as clean enrollment utterances or precise directions of arrival (DOA). The proposed approach encodes the target speaker through discrete, interpretable relations between concurrent speakers, such as left and right or near and far, referred to as the Relative Position Cue (RPC). This relational conditioning effectively leverages spatial information while avoiding explicit DOA estimation, thereby enabling more robust operation under realistic conditions. We evaluate the proposed method across multiple datasets encompassing diverse acoustic conditions, including directional and diffuse babble noise, varying numbers of microphones, mixtures with different numbers of speakers, and dynamic scenarios involving moving speakers. Experimental results demonstrate that the RPC-based approach consistently achieves strong extraction performance and generalizes well across all tested configurations, establishing it as a practical and interpretable alternative for spatially informed speaker extraction.
Figures & tables
| # | RPC for two speakers scenario: |
|---|---|
| 1 | The target speaker is farther from the array than the interferer |
| 2 | The target speaker is closer to the array than the interferer |
| 3 | The target speaker is to the right of the interferer |
| 4 | The target speaker is to the left of the interferer |
| 5 | The target speaker is farther than and to the right of the interferer |
| 6 | The target speaker is closer than and to the left of the interferer |
TABLE I : Relative speaker positions and orientations for the two-source configuration.
| # | Three speakers RPC : |
|---|---|
| 1 | The target is the farthest and the left-most speaker |
| 2 | The target is the farthest speaker, between the other two |
| 3 | The target is the farthest and the right-most speaker |
| 4 | The target is the middle (in distance) and the left-most speaker |
| 5 | The target is the middle speaker in both distance and direction |
| 6 | The target is the middle (in distance) and the right-most speaker |
TABLE II : Relative speaker positions and orientations for the three-source configuration.
Fig. 3 : Block diagram of the proposed RPC -based target speaker extraction framework. RPC ( RPC ) are used to guide the network toward the desired (target) speaker. The lower panel shows an implementation based on SpatialNet, while the upper panel shows one based on a U-Net architecture. In the SpatialNet implementation, the command embedding is concatenated with the feature representations, whereas in the U-Net implementation it is incorporated using FiLM conditioning. These choices are not mandatory and are shown only as representative examples of how RPC information can be integrated into existing architectures.
Fig. 4 : Spectrograms (reference microphone) of a test mixture from test set (ii), processed by the four-microphone U-Net RPC model. Speaker A is at 1.68 m and , speaker B at 0.70 m and ; s, SNR dB, SIR dB. (a) Noisy mixture. (b) Speech-only mixture (ideal noise suppression). (c), (d) Reverberant references of speakers A and B. (e) Output with the command “farther”: SI-SDR of 10.1 dB w.r.t. A and dB w.r.t. B. (f) Output with the command “closer”: 10.2 dB w.r.t. B and dB w.r.t. A.
| 4 Microphones | 2 Microphones | |||||
| Model / Test Set | (i) | (ii) | (iii) | (i) | (ii) | (iii) |
| Unprocessed | -0.75/0.598 | -0.76/0.598 | -0.75/0.598 | -0.75/0.599 | -0.76/0.599 | -0.75/0.598 |
| Oracle MVDR | 8.80/0.839 | 8.07/0.822 | 8.80/0.840 | 5.44/0.737 | 4.57/0.714 | 5.47/0.737 |
| FasNet [ 41 ] | 7.47/0.733 | 6.90/0.678 | 7.60/0.735 | 6.80/0.744 | 5.40/0.711 | 6.90/0.745 |
| SpatialNet [ 16 ] | 13.20/0.922 | 12.20/0.912 | 13.20/0.922 | 12.39/0.875 | 11.80/0.866 | 12.43/0.877 |
| SpatialNet RPC | 14.20/0.911 | 13.40/0.901 | 14.10/0.909 | 10.71/0.848 | 9.00/0.811 | 10.61/0.846 |
TABLE III : SI-SDR and STOI results for 2- and 4-microphone ULA . Test sets: (i) large DOA and radial gaps; (ii) small DOA gap, large radial gap; (iii) small radial gap, large DOA gap. x: not applicable. Each entry: SI-SDR [dB] / STOI .
| 4 Microphones | 2 Microphones | |||||
|---|---|---|---|---|---|---|
| Model / Test Set | (i) | (ii) | (iii) | (i) | (ii) | (iii) |
| Unprocessed | -3.84/0.469 | -3.87/0.469 | -3.84/0.468 | -3.84/0.469 | -3.87/0.469 | -3.84/0.468 |
| Oracle MVDR | 5.70/0.747 | 4.99/0.725 | 5.65/0.746 | 1.55/0.614 | 1.03/0.595 | 1.48/0.612 |
| SpatialNet RPC | 6.70/0.768 | 5.70/0.736 | 6.07/0.751 | 6.40/0.772 | 5.40/0.744 | 5.70/0.754 |
| Unet DOA | 5.34/0.740 | x | 5.10/0.733 | 3.98/0.689 | x | 3.56/0.680 |
| Unet RPC | 6.31/0.768 | 4.77/0.719 | 5.90/0.759 | 5.87/0.751 | 4.20/0.699 | 5.42/0.740 |
TABLE IV : SI-SDR and STOI results for 3 speakers with 4 and 2 microphones. Test sets (i)–(iii) as in Table III ; x: not applicable. Each entry: SI-SDR [dB] / STOI .
| 4 Microphones | 2 Microphones | |||||
| Model / Test Set | (i) | (ii) | (iii) | (i) | (ii) | (iii) |
| Unprocessed | -0.75/0.598 | -0.76/0.598 | -0.62/0.541 | -0.75/0.598 | -0.76/0.598 | -0.62/0.541 |
| SpatialNet RPC | 12.35/0.914 | 7.65/0.741 | 12.97/0.901 | 9.20/0.823 | 5.50/0.713 | 10.50/0.885 |
| Unet RPC | 9.39/0.829 | 5.24/0.736 | 9.70/0.853 | 8.80/0.804 | 4.70/0.721 | 8.44/0.766 |
TABLE V : SI-SDR and STOI results for real data with 4 and 2 microphones.
| 4 Microphones | 2 Microphones | ||||||
|---|---|---|---|---|---|---|---|
| Model | Noise | (i) | (ii) | (iii) | (i) | (ii) | (iii) |
| Unprocessed | Dir. | -0.75/0.598 | -0.76/0.598 | -0.75/0.598 | -0.75/0.599 | -0.76/0.599 | -0.75/0.598 |
| Bab. | -0.39/0.610 | -0.40/0.610 | -0.39/0.610 | -0.39/0.610 | -0.39/0.610 | -0.39/0.610 | |
| Oracle MVDR | Dir. | 8.81/0.839 | 8.07/0.822 | 8.81/0.840 | 5.44/0.737 | 4.57/0.714 | 5.47/0.737 |
| Bab. | 8.67/0.824 | 7.95/0.807 | 8.67/0.825 | 5.63/0.737 | 4.78/0.715 | 5.66/0.738 | |
| SpatialNet RPC | Dir. | 14.20/0.911 | 13.40/0.901 | 14.10/0.909 | 10.71/0.848 | 9.00/0.811 | 10.61/0.846 |
TABLE VI : SI-SDR and STOI results for directional (Dir.) and diffuse babble (Bab.) noise with 4- and 2-microphone ULA . The models were trained with directional noise only. Each entry: SI-SDR [dB] / STOI .
Fig. 5 : Magnitude-squared coherence between two adjacent microphones ( cm), averaged over the test set, for the directional (DNS) noise and the 20-talker babble noise, compared with the theoretical spherically and cylindrically isotropic (diffuse) noise fields.
Fig. 6 : SI-SDR of the RPC -based models (two-speaker test sets) as a function of the DOA gap (test sets (i) and (ii)), the radial gap (test sets (i) and (iii)), the reverberation time , the clarity index of the target, and the SIR . Each point is the mean over all outputs within the bin.
| model / test set | (i) | (ii) | (iii) |
|---|---|---|---|
| Unprocessed | -1.78/0.605 | -1.77/0.622 | -1.86/0.625 |
| FasNet [ 41 ] | 4.30/0.727 | 2.71/0.707 | 5.70/0.765 |
| SpatialNet [ 16 ] | 6.06/0.764 | 3.86/0.735 | 8.94/0.831 |
| SpatialNet RPC | 6.83/0.771 | 3.60/0.712 | 8.91/0.825 |
| SpatialNet RPC oracle window | 7.90/0.776 | 6.10/0.686 | 9.98/0.816 |
| Unet RPC | 10.72/0.846 | 8.34/0.802 | 11.90/0.867 |
TABLE VII : SISDR and STOI results for two moving speakers for 2 microphones. Each entry: SI-SDR [dB] / STOI .
Fig. 7 : (a) Speaker trajectory. (b) Example of SI-SDRi as a function of the window size.
More in this topic
Target Speaker Extraction
Target speaker extraction isolates a chosen voice from overlapping speech using speaker identity, visual, spatial, textual, or neural cues.
See all 42 papers