Toward Complete Speaker Anonymization: a Streaming Cascaded System for Voice Privacy and PII Removal
Authors: Stanisław Kacprzak, Thomas Thebaud, Najim Dehak
Organizations: Center for Language and Speech Processing, Johns Hopkins University, Baltimore, Maryland, USA · AGH University of Krakow, Institute of Electronics, Krakow, Poland · AMIAD, Palaiseau, FR
We present a cascade streaming voice privacy system, consisting of ASR, NLP, and TTS modules. By leveraging recent improvements in all three aspects of speech and language processing, we show that such a system is capable of achieving a result comparable to state-of-the-art offline voice privacy systems, while providing controllable capabilities of content protection beyond voice. With an estimated utterance-onset end-to-end latency of 717 ms, the presented system preserves speech content with 11% WER on Librispeech, while detecting PII with a 69% SLU F1 score and achieving up to 47.66% EER against a semi-informed attacker.
Figures & tables
ASR [ms]
TTS (TTFA) [ms]
E2E [ms]
Tout/Tin
669
41
711
1.33
Table 1: Median of measurements for latency, TTFA and the duration ratio of output and input audio.
System
WER ↓
UAR ↑
dev
test
dev
test
Orig.
1.80
1.84
69.08
71.06
T30 [ 13 ]
5.02
4.78
28.45
29.44
T8-4 [ 30 ]
3.74
3.75
30.10
30.59
Ours (Streaming)
11.11
11.44
29.37
29.22
Table 2: Utility metrics evaluating linguistic preservation (WER) on LibriSpeech and emotion preservation (UAR) on IEMOCAP, on the original speech.
Split
Orig.
T30 [ 13 ]
T8-4 [ 30 ]
B5 [ 28 ]
Ours
dev-f ↑
10.51
44.32
47.58
35.82
44.18
dev-m ↑
0.93
47.52
49.23
32.92
47.66
test-f ↑
8.76
45.07
47.44
33.95
44.37
test-m ↑
0.42
44.77
48.75
34.73
45.25
Table 3: Privacy results are shown as an EER, evaluated on the Librispeech subsets.
SLURP
DialogPII
Model
Clean transcript
ASR chunks
Simulated Streaming
Entity F1 ↑
SLU F1 ↑
Entity F1 ↑
SLU F1 ↑
Entity F1 ↑
SLU F1 ↑
Latency [ms] ↓
Qwen 1.5B
87.50
89.10
51.48
66.51
56.27
64.22
46
Qwen 0.5B
83.92
85.42
48.33
62.73
46.55
56.62
39
spaCy
87.19
88.98
54.82
69.01
53.27
60.60
4
GLiNER Small v2.5
82.55
87.55
50.59
66.93
61.22
69.14
6
Table 4: DialogPII and SLURP PII detection results for fine-tuned models.
Music and Audio Research Group (MARG) · Department of Intelligence and Information, Seoul National University · Interdisciplinary Program in Artificial Intelligence, Seoul National University