Microphone Geometry Aware Regional Target Speaker Extraction
Organizations: Department of Computer Science, University of Oxford, United Kingdom
Abstract
Regional target speaker extraction (RSE) aims to extract speech originating from a specified angular region, relaxing the requirement for precise target speaker location when performing location based speech extraction. However, existing RSE methods typically assume a fixed microphone array configuration and microphone number, limiting their applicability to unseen microphone geometries and varying numbers of microphones. In this work, we propose a geometry-aware RSE framework capable of extract with varying microphone array geometry and microphone numbers. Specifically, we introduce an attention-based multichannel front-end to encode inter-microphone phase information, together with a spatial audio reconstruction-based auxiliary objective to improve the front-end spatial representations quality. Experimental results demonstrate that the proposed framework improves SI-SDRi by 5.1 dB over geometry-aware baselines across microphone arrays with four-microphones, while maintaining a performance gap of less than 0.6 dB compared with a tetrahedral geometry-specific baseline when evaluated on the tetrahedral microphone array.
Figures & tables
| Model | Microphone | 0 Target | 1 Target | 2 Target | ||||||
| Geometry | Decay | SI-SDR | SI-SDRi | PESQ | STOI | SI-SDR | SI-SDRi | PESQ | STOI | |
| Geometry Specific Models | ||||||||||
| DSENet Tetra | Dynamic-4 | -23.64 | -32.61 | -26.83 | 1.09 | 0.31 | -32.56 | -37.10 | 1.08 | 0.42 |
| Tetrahedral | -31.64 | 5.03 | 11.36 | 1.77 | 0.75 | 10.09 | 5.63 | 2.40 | 0.84 | |
| DSENet NAO | Dynamic-12 | -23.45 | -33.10 | -26.86 | 1.11 | 0.30 | -30.56 | -35.03 | 1.12 | 0.40 |
| NAO Robot | -27.55 | 6.04 | 12.31 | 1.85 | 0.78 | 10.58 | 6.08 | 2.47 | 0.86 | |
| Model | Tetrahedral | 12 Mic NAO Robot | ||
|---|---|---|---|---|
| simulated | real | simulated | real | |
| DSENet | 11.36 | 8.40 | 12.31 | 9.57 |
| DPARNet-RSE | 3.52 | 5.19 | 3.84 | 6.11 |
| LCMV | 0.25 | 1.12 | -0.03 | -2.58 |
| FoVNet | 4.83 | 3.85 | 5.21 | 4.96 |
| Ours | 10.79 | 9.15 | 11.97 | 9.08 |
Target Speaker Extraction
Target speaker extraction isolates a chosen voice from overlapping speech using speaker identity, visual, spatial, textual, or neural cues.
See all 42 papers