eess.ASMar 17, 2026

HRTF-guided Binaural Target Speaker Extraction with Real-World Validation

Authors: Yoav Ellinson, Sharon Gannot

Organizations: Faculty of Engineering, Bar Ilan University, Israel

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents a Head-Related Transfer Function (HRTF)-guided framework for binaural Target Speaker Extraction (TSE) from mixtures of concurrent sources. Unlike conventional TSE methods based on Direction of Arrival (DOA) estimation or enrollment signals, which often distort perceived spatial location, the proposed approach leverages the listener's HRTF as an explicit spatial prior. The proposed framework is built upon a multi-channel deep blind source separation backbone, adapted to the binaural TSE setting. It is trained on measured HRTFs from a diverse population, enabling cross-listener generalization rather than subject-specific tuning. By conditioning the extraction on HRTF-derived spatial information, the method preserves binaural cues while enhancing speech quality and intelligibility. The performance of the proposed framework is validated through simulations and real recordings obtained from a head and torso simulator (HATS).

Figures & tables

Figure 1: Overview of the proposed binaural target speaker extraction model ( BB denotes batch size). Based on the NBSS framework, the architecture processes the STFT mixture by conditioning encoded features on direct-path HRTF via latent-space modulation. A stack of P=8P=8 NBC2 blocks produces the final complex-valued spectral estimate. See [ 16 , Fig. 2] for a detailed NBC2 block illustration.
MethodSI-SDR i ↑\uparrowPESQ ↑\uparrowΔ\Delta ITD ↓\downarrowΔ\Delta ILD ↓\downarrow
Mixture–1.181.4640.417
DOA-BDE [ 11 ]13.8812.740.9820.479
Proposed15.7703.030.0440.349
Table 1: Extraction performance in terms of SI-SDR i [dB], PESQ , and spatial consistency metrics Δ\Delta ITD [ms] and Δ\Delta ILD [dB]. Arrows indicate whether higher or lower values are better. Reported values correspond to mean scores computed over a test set of 1000 mixtures. Since extraction is performed for both speakers in each mixture, the effective number of evaluated samples is 2000.
Method20∘20^{\circ}30∘30^{\circ}40∘40^{\circ}50∘50^{\circ}60∘60^{\circ}70∘70^{\circ}80∘80^{\circ}90∘90^{\circ}Avg.
Mixture2.052.072.092.152.142.232.192.182.14
DOA-BDE [ 11 ]3.003.033.053.173.163.253.263.203.14
Proposed3.023.123.173.313.293.343.393.353.22
Table 2: Mean NISQA scores ( ↑\uparrow ) versus angular distance between speakers, evaluated over 30 real-world recordings per distance, totaling 480 samples (two speakers per recording).
More in this topic

Target Speaker Extraction

Target speaker extraction isolates a chosen voice from overlapping speech using speaker identity, visual, spatial, textual, or neural cues.

See all 38 papers

Explore similar work

CardsList

  1. Training-Free Multi-Step Inference for Target Speaker Extraction

    Mar 11, 2026Zhenghai You, Ying Shi, Lantian Li +1Target Speaker Extraction

    Beijing University of Posts and Telecommunications, China · Center for Speech and Language Technologies, BNRist, Tsinghua University, China

  2. SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings

    Jul 16, 2026Shuai Wang, Zihan Qian, Ke Zhang +9Target Speaker ExtractionSpeech Processing

    Nanjing University, China · Chinese University of Hong Kong (Shenzhen), China · Brno University of Technology, Czechia +3

  3. PromptSep-Speaker: Generative Target Speaker Extraction via Multimodal Prompting and Semantic Alignment

    Oct 16, 2026Yutong Wen, Ke Chen, Prem Seetharaman +5Target Speaker Extraction

    Adobe Research, USA · Massachusetts Institute of Technology, USA