VorTEX: Various overlap ratio for Target speech EXtraction
Organizations: Department of Artificial Intelligence, Chung-Ang University, Republic of Korea
Abstract
Target speech extraction (TSE) aims to recover a target speaker's voice from a mixture. While recent text-prompted approaches have shown promise, most approaches assume fully overlapped mixtures, limiting insight into behavior across realistic overlap ratios. We introduce VorTEX (Various overlap ratio for Target speech EXtraction), a text-prompted TSE architecture with a Decoupled Adaptive Multi-branch (DAM) Fusion block that separates primary extraction from auxiliary regularization pathways. To enable controlled analysis, we construct PORTE, a two-speaker dataset spanning overlap ratios from 0% to 100%. We further propose Suppression Ratio on Energy (SuRE), a diagnostic metric that detects suppression behavior not captured by conventional measures. Experiments show that existing models exhibit suppression or residual interference under overlap, whereas VorTEX achieves the highest separation fidelity across 20-100% overlap (e.g., 5.50 dB at 20% and 2.04 dB at 100%) while maintaining zero SuRE, indicating robust extraction without suppression-driven artifacts.
Figures & tables
| Type - | Prompt template |
|---|---|
| Speaker’s Gender - | Extract only the {male/female} voice from this audio. |
| - | Please remove the {male/female} voice from this audio. |
| Speech Order - | Extract the voice of the speaker who spoke {first/later}. |
| Relative Length - | Extract the speech that contains a {shorter/longer} duration of speech. |
| (1) SISDR | Avg. | 0% | 20% | 40% | 60% | 80% | 100% |
|---|---|---|---|---|---|---|---|
| AudioSep | 1.03 | 13.01 | 2.47 | -0.52 | -1.55 | -2.83 | -4.37 |
| DGMO | -4.56 | -2.92 | -4.29 | -4.52 | -5.06 | -5.30 | -5.25 |
| StyleTSE | 0.24 | 2.45 | -0.22 | -0.18 | -0.27 | -0.18 | -0.16 |
| LLM-TSE | -0.51 | -0.51 | -0.71 | -0.55 | -0.47 | -0.43 | -0.40 |
| VorTEX | 4.50 | 7.07 | 5.50 | 4.53 | 4.29 | 3.56 | 2.04 |
| (2) SISDRi | |||||||
| (1) SISDR | Avg. | 0% | 20% | 40% | 60% | 80% | 100% |
|---|---|---|---|---|---|---|---|
| VorTEX | 4.50 | 7.07 | 5.50 | 4.53 | 4.29 | 3.56 | 2.04 |
| w/o DAM | 3.11 | 6.20 | 4.55 | 2.77 | 2.63 | 1.83 | 0.71 |
| w/o MF | 3.46 | 5.60 | 4.78 | 2.87 | 3.60 | 2.69 | 1.24 |
| w/o AF | 4.41 | 7.00 | 5.55 | 4.52 | 4.01 | 2.96 | 2.42 |
| w/o DP | 4.46 | 7.00 | 5.53 | 4.69 | 4.08 | 3.16 | 2.35 |
| (2) PESQ | |||||||
Target Speaker Extraction
Target speaker extraction isolates a chosen voice from overlapping speech using speaker identity, visual, spatial, textual, or neural cues.
See all 39 papers