Target speech extraction (TSE) typically relies on pre-recorded high-quality enrollment speech, which disrupts user experience and limits feasibility in spontaneous interaction. In this paper, we propose Enroll-on-Wakeup (EoW), a novel framework where the wake-word segment, captured naturally during human-machine interaction, is automatically utilized as the enrollment reference. This eliminates the need for pre-collected speech to enable a seamless experience. We perform the first systematic study of EoW-TSE, evaluating advanced discriminative and generative models under real diverse acoustic conditions. Given the short and noisy nature of wake-word segments, we investigate enrollment augmentation using LLM-based TTS. Results show that while current TSE models face performance degradation in EoW-TSE, TTS-based assistance significantly enhances the listening experience, though gaps remain in speech recognition accuracy.
Figures & tables
Figure 1: Illustration of EoW-TSE system.
Scenario
Condition
#spk/#utt
Average duration (s)
enrollment
mixture
CloseNoise-10
d=1, RT=0.4
45/503
1.08
1.78
FarNoise-10
d=3, RT=0.4
45/502
1.08
1.78
FarNoiseReverb-10
d=3, RT=0.6
30/278
1.17
1.79
FarNoise-5
d=3, RT=0.4
45/503
1.08
1.78
FarNoiseReverb-5
d=3, RT=0.6
15/226
0.97
1.76
Table 1: Acoustic conditions of five EoW-TSE test scenarios. ‘ d ’ denotes distance to the microphone (m), ‘ RT ’ is the reverberation time ( RT60 in seconds), and ‘*-10/-5’ indicates SNR levels of 10dB and 5dB respectively.
Models
SI-SDR
PESQ
STOI
Params
MACs
Mixture
-1.96
1.08
64.73
-
-
SEF-PNet [ 9 ]
8.18
1.55
82.67
6.08M
15.87G
CIE-mDPTNet [ 10 ]
9.47
1.78
85.35
2.9M
48.3G
⋆ LExt [ 11 ]
10.47
1.88
87.26
3.9M
61.0G
SoloSpeech [ 18 ]
11.12
1.89
-
590.9M
-
Table 2: Results on Libri2Mix 2-Speaker+Noise. ⋆ are the results reproduced by us.
Conditions
DNSMOS ↑
WER ↓
Noisy
xTTS
IndexTTS2
CosyVoice3
Noisy
xTTS
IndexTTS2
CosyVoice3
CloseNoise-10
2.029
2.472
2.564
2.609
14.61
39.76
11.23
40.36
FarNoise-10
1.590
2.464
2.237
2.428
27.74
40.04
19.27
54.88
FarNoiseReverb-10
1.262
2.549
1.917
2.114
19.69
33.45
16.10
59.71
FarNoise-5
1.440
2.274
2.063
2.400
45.13
46.57
36.58
80.62
FarNoiseReverb-5
1.251
2.473
1.743
2.107
70.58
55.86
44.47
102.54
Table 3: Results on enrollments synthesized using different TTS models.
Conditions
DNSMOS ↑
WER ↓
Noisy
xTTS (CR)
IndexTTS2 (CR)
xTTS(EC)
IndexTTS2(EC)
Noisy
xTTS(CR)
IndexTTS2(CR)
xTTS(EC)
IndexTTS2(EC)
CloseNoise-10
1.904
2.163
2.119
2.197
2.170
3.10
4.37
3.24
3.91
3.48
FarNoise-10
1.371
1.511
1.481
1.527
1.500
4.54
6.41
5.38
6.81
5.14
FarNoiseReverb-10
1.165
1.202
1.200
1.214
1.195
3.52
7.15
5.06
7.00
6.08
FarNoise-5
1.286
1.379
1.366
1.396
1.371
13.31
16.01
29.55
15.92
25.61
FarNoiseReverb-5
1.173
1.216
1.223
1.217
1.212
15.95
22.89
31.32
23.12
28.36
Table 4: EoW-TSE results with different enrollment augmentation methods on CIE-mDPTNet.
Figure 2: OVRL scores on five scenarios: Original EoW-TSE vs. TTS-augmented (CR).
Figure 3: WERs on five scenarios: Original EoW-TSE vs. TTS-augmented (CR).