Organizations: Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. · Shanghai Artificial Intelligence Laboratory, Shanghai 200032, China. · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China. · Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing 100084, China. · Vanke School of Public Health, Tsinghua University, Beijing 100084, China. · Peking Union Medical College Hospital, Beijing 100730, China. · Peking University Huilongguan Clinical Medical School, Beijing 100096, China. · Department of Clinical Psychology, Beijing Huilongguan Hospital, Capital Medical University, Beijing 100096, China. · WHO Collaborating Center for Research and Training in Suicide Prevention, Beijing 100096, China. · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong (SAR), China. · Chinese Academy of Medical Sciences, Beijing 100730, China. · Department of Speech, Hearing and Phonetic Sciences, University College London (UCL), London WC1N 1PF, United Kingdom.
Clinical speech is increasingly used for disease screening and clinical assessment, but directly distributing speech recordings poses privacy risks because they contain both speaker-identifying voiceprints and sensitive spoken content. Existing speech anonymisation approaches are primarily developed for general speech applications or task-specific clinical speech and overlook that different clinical tasks rely on distinct speech characteristics, necessitating task-adaptive anonymisation strategies to protect patient privacy while preserving clinical utility. Here we introduce CASA, a clinically adaptive speech anonymisation agent that transforms anonymisation from a fixed processing procedure into a task-guided optimisation process, dynamically tailoring voiceprint and content anonymisation to task-specific privacy and clinical utility requirements. Through recursive self-improvement (RSI), CASA iteratively evaluates and refines anonymisation strategies against task-specific privacy and clinical utility requirements, while learning from these outcomes to guide more efficient planning in subsequent clinical tasks. CASA is validated across 14 clinical tasks encompassing five languages, diverse elicitation paradigms, and mental health, cognitive and voice disorders. CASA preserved disease-related classification performance while meeting speaker-anonymisation requirements across all clinical tasks, versus only 8/14 for the strongest fixed method. Assessments of resistance to speaker re-identification by external attackers and identification and redaction of sensitive personal information in clinical speech further demonstrated CASA's dual protection of speaker identity and sensitive spoken content. Clinical expert assessments further supported the preservation of clinically relevant information and the acceptability of anonymised speech for controlled research sharing.
Figures & tables
Figure 1: ∣ Overview of CASA and comparison with baselines. a, Architecture of the CASA agent. An LLM-based controller receives the clinical task, speech paradigm and privacy–utility requirements, plans an anonymisation strategy and invokes tools for voiceprint and content anonymisation. Evaluation feedback guides adjustment to methods and parameters. Cross-task memory retains coarse-grained summaries and fine-grained experimental records to guide subsequent planning. b, Number of datasets or task subsets meeting all applicable privacy and clinical-utility criteria, comparing five individual methods used as fixed strategies (grey), a SecureSpeech pipeline (orange) and CASA (blue). Success is defined as that the anonymised speech achieve an EER of at least 20%, sensitive-content localisation F1 of at least 0.75 when required, and a downstream BACC decrease of less than three percentage points relative to original speech. c, Delta BACC in percentage points after anonymisation with CASA and SecureSpeech. The dashed green circle marks the -3-percentage-point boundary, and the green shaded outer region indicates changes satisfying the utility requirement. The datasets are grouped into mental health, cognitive and voice disorders.
Figure 2: ∣ Voiceprint anonymisation evaluation across methods and datasets. Boxplots show six evaluation metrics across datasets for each voiceprint anonymisation method: ΔSNR (change in SNR, anonymised vs. original), ΔMOS , PCC F0 , CER/WER, Emo, and EER. Each dot represents a single subset/task. Box plots show the median (centre line), the first and third quartiles (box bounds), and whiskers extending to 1.5× the interquartile range. The dashed horizontal line in ΔSNR and ΔMOS at zero indicates no change relative to the original audio. The heatmap shows ΔBACC . Negative values (blue) indicate performance degradation and positive values (red) indicate improvement. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions. No adjustment was made for multiple comparisons. Exact P values are provided in Supplementary Tables Supplementary Information – Supplementary Information . Asterisks denote significance levels: ∗P<0.05 , ∗∗P<0.01 , ∗∗∗P<0.001 .
Figure 3: ∣ Content anonymisation evaluation. a,b, Content anonymisation on two in-house datasets, comparing CASA’s locally deployed Qwen2.5-32B-Instruct model under default and task-adaptive prompts with the NER-based localisation used in SecureSpeech. c, Comparison of Qwen2.5-32B-Instruct and GPT-5-nano on precision, recall and F1 for sensitive-content localisation. Privacy-sensitive entities were replaced with synthetic names and locations for closed-source model evaluation. d,e, Downstream task performance with and without content anonymisation. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions, and no statistically significant reductions were observed across all evaluations. No adjustment was made for multiple comparisons.
Figure 4: ∣ Experience-guided self-improvement improves the efficiency of task adaptation. The upper right is a comparison of CASA without and with memory. Success@1 and Success@2 (left axis) denote the proportions of runs satisfying all specified privacy and utility requirements within one and two evaluation steps, respectively. Mean hit steps (right axis) denotes the average number of steps required to first reach a configuration satisfying these requirements. Higher success rates and fewer steps indicate more efficient adaptation. The other part shows a representative execution trace for Androids-R, showing how experiences from previous runs guide CASA in subsequent run.
Figure 5: ∣ Clinical expert evaluation of anonymised speech. a, Agreement between assessments of original and anonymised recordings, using crisis-level labels for Crisis and nasality-severity ratings for PUMCH. b, Preservation rating of acoustic and semantic information on the Suicide-YF dataset. c, Anonymisation quality and sharing acceptability rating. Clinicians rated three aspects: voiceprint anonymisation, content anonymisation, and sharing acceptability. All subjective ratings were collected on a scale of 1 to 5, with higher scores indicating better performance. Evaluators were blinded to the original reference labels.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Fig. 1: ∣ Detailed workflow of CASA anonymisation. Original recordings are transcribed using an ASR module to obtain transcripts with speaker annotations and word-level timestamps. Available voiceprint anonymisation methods include pitch shifting ( Pitch ), spectral formant scaling ( Formant spec ), LPC-based formant transformation ( Formant LPC ), neural voice conversion ( Neural VC ) and transcription–resynthesis ( Resyn ). For content anonymisation, a locally deployed LLM identifies sensitive spans in the transcripts and produces annotations to guide redaction. For Resyn , which doesn’t preserve temporal alignment, sensitive content is masked in the transcript before synthesis; while for other methods, the corresponding speech segments are replaced with noise. The resulting recordings are evaluated for speaker privacy (EER), downstream clinical utility (BACC), signal cleanliness (SNR), perceptual quality (MOS), pitch-contour preservation (PCC F0 ), linguistic preservation (CER/WER) and emotion preservation (Emo).
Supplementary Fig. 2: ∣ Downstream model structure.
Supplementary Fig. 3: ∣ Web interface for crisis relabelling.
Supplementary Fig. 4: ∣ Web interface for crisis anonymisation rating.
Supplementary Fig. 5: ∣ Web interface for Suicide-YF scoring.
Supplementary Fig. 6: ∣ Web interface for PUMCH evaluation.
Clinical speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.
Speech anonymization is evaluated with automatic speaker verification attackers whose performance depends on the attribute distribution of both their training and evaluation data, which may bias privacy estimates for some speaker groups. To alleviate this issue, we introduce VoxAnon, a dataset built from VoxCeleb1 and 2 with training and evaluation splits balanced across gender, age, accent, and pitch. We extract and cross-check attributes using two complementary methods, balance trial lists per subgroup, and draw test/enrollment utterances from distinct video sources in order to limit linguistic and session bias. Comparing VoxAnon with unbalanced training and test data across four attackers and three anonymizers, we show that attribute imbalance changes which subgroups appear hardest to re-identify, in directions that depend on the attribute. In the balanced setting, residual disparities persist, mostly tied to accent and pitch, and depend on the anonymizer/attacker pair. Code and trial lists are publicly available.
Orane Dufour, Paul Magron, Mickael Rouvier
Université de Lorraine, CNRS Inria, LORIA, F-54000 Nancy, France · LIA, Avignon University F-84911 Avignon, France
Voice anonymization aims to protect a speaker's identity while preserving the linguistic content of speech, enabling voice data to be shared and processed with reduced privacy risk. This work focuses on reversible voice anonymization, where the original voice can be recovered by an authorized user rather than being permanently discarded. However, existing approaches can achieve apparent speaker anonymization by degrading the speech itself, while their effectiveness may also depend on the speaker-verification representation used during training. This creates a need for methods that protect speaker identity without sacrificing speech intelligibility and that remain effective against unseen speaker verifiers. We introduce content preserving training constraints to improve the privacy-utility trade-off by preventing anonymization from being achieved primarily through speech degradation. We further introduce a verifier-generalization strategy that incorporates a stronger speaker-verification representation during training, improving anonymization robustness beyond the verifier used in the original training setup. The resulting approach maintains intelligible speech, strengthens anonymization, and preserves reliable voice restoration, with evaluation on unseen speakers and held-out speaker verification models. On 38 unseen speakers, the approach raises the attacker's equal error rate from a measured floor of 2.8% to 26.3%, and to 27.9% against a held-out verifier, with 11.8% word error rate and 94.9% speaker similarity on key-based restoration. These improvements advance reversible voice anonymization toward practical privacy-preserving speech applications, where anonymized speech can be safely processed or shared while the original voice remains recoverable under authorized conditions.
Nirjon Mobarak Talukder, Rafid Ahmed, Mohammad Ishrak Abedin +1
Department of Computer Science and Engineering, Islamic University of Technology, Dhaka, Bangladesh