Organizations: Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. · Shanghai Artificial Intelligence Laboratory, Shanghai 200032, China. · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China. · Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing 100084, China. · Vanke School of Public Health, Tsinghua University, Beijing 100084, China. · Peking Union Medical College Hospital, Beijing 100730, China. · Peking University Huilongguan Clinical Medical School, Beijing 100096, China. · Department of Clinical Psychology, Beijing Huilongguan Hospital, Capital Medical University, Beijing 100096, China. · WHO Collaborating Center for Research and Training in Suicide Prevention, Beijing 100096, China. · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong (SAR), China. · Chinese Academy of Medical Sciences, Beijing 100730, China. · Department of Speech, Hearing and Phonetic Sciences, University College London (UCL), London WC1N 1PF, United Kingdom.
Clinical speech is increasingly used for disease screening and clinical assessment, but directly distributing speech recordings poses privacy risks because they contain both speaker-identifying voiceprints and sensitive spoken content. Existing speech anonymisation approaches are primarily developed for general speech applications or task-specific clinical speech and overlook that different clinical tasks rely on distinct speech characteristics, necessitating task-adaptive anonymisation strategies to protect patient privacy while preserving clinical utility. Here we introduce CASA, a clinically adaptive speech anonymisation agent that transforms anonymisation from a fixed processing procedure into a task-guided optimisation process, dynamically tailoring voiceprint and content anonymisation to task-specific privacy and clinical utility requirements. Through recursive self-improvement (RSI), CASA iteratively evaluates and refines anonymisation strategies against task-specific privacy and clinical utility requirements, while learning from these outcomes to guide more efficient planning in subsequent clinical tasks. CASA is validated across 14 clinical tasks encompassing five languages, diverse elicitation paradigms, and mental health, cognitive and voice disorders. CASA preserved disease-related classification performance while meeting speaker-anonymisation requirements across all clinical tasks, versus only 8/14 for the strongest fixed method. Assessments of resistance to speaker re-identification by external attackers and identification and redaction of sensitive personal information in clinical speech further demonstrated CASA's dual protection of speaker identity and sensitive spoken content. Clinical expert assessments further supported the preservation of clinically relevant information and the acceptability of anonymised speech for controlled research sharing.
Figures & tables
Figure 1: ∣ Overview of CASA and comparison with baselines. a, Architecture of the CASA agent. An LLM-based controller receives the clinical task, speech paradigm and privacy–utility requirements, plans an anonymisation strategy and invokes tools for voiceprint and content anonymisation. Evaluation feedback guides adjustment to methods and parameters. Cross-task memory retains coarse-grained summaries and fine-grained experimental records to guide subsequent planning. b, Number of datasets or task subsets meeting all applicable privacy and clinical-utility criteria, comparing five individual methods used as fixed strategies (grey), a SecureSpeech pipeline (orange) and CASA (blue). Success is defined as that the anonymised speech achieve an EER of at least 20%, sensitive-content localisation F1 of at least 0.75 when required, and a downstream BACC decrease of less than three percentage points relative to original speech. c, Delta BACC in percentage points after anonymisation with CASA and SecureSpeech. The dashed green circle marks the -3-percentage-point boundary, and the green shaded outer region indicates changes satisfying the utility requirement. The datasets are grouped into mental health, cognitive and voice disorders.
Figure 2: ∣ Voiceprint anonymisation evaluation across methods and datasets. Boxplots show six evaluation metrics across datasets for each voiceprint anonymisation method: ΔSNR (change in SNR, anonymised vs. original), ΔMOS , PCC F0 , CER/WER, Emo, and EER. Each dot represents a single subset/task. Box plots show the median (centre line), the first and third quartiles (box bounds), and whiskers extending to 1.5× the interquartile range. The dashed horizontal line in ΔSNR and ΔMOS at zero indicates no change relative to the original audio. The heatmap shows ΔBACC . Negative values (blue) indicate performance degradation and positive values (red) indicate improvement. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions. No adjustment was made for multiple comparisons. Exact P values are provided in Supplementary Tables Supplementary Information – Supplementary Information . Asterisks denote significance levels: ∗P<0.05 , ∗∗P<0.01 , ∗∗∗P<0.001 .
Figure 3: ∣ Content anonymisation evaluation. a,b, Content anonymisation on two in-house datasets, comparing CASA’s locally deployed Qwen2.5-32B-Instruct model under default and task-adaptive prompts with the NER-based localisation used in SecureSpeech. c, Comparison of Qwen2.5-32B-Instruct and GPT-5-nano on precision, recall and F1 for sensitive-content localisation. Privacy-sensitive entities were replaced with synthetic names and locations for closed-source model evaluation. d,e, Downstream task performance with and without content anonymisation. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions, and no statistically significant reductions were observed across all evaluations. No adjustment was made for multiple comparisons.
Figure 4: ∣ Experience-guided self-improvement improves the efficiency of task adaptation. The upper right is a comparison of CASA without and with memory. Success@1 and Success@2 (left axis) denote the proportions of runs satisfying all specified privacy and utility requirements within one and two evaluation steps, respectively. Mean hit steps (right axis) denotes the average number of steps required to first reach a configuration satisfying these requirements. Higher success rates and fewer steps indicate more efficient adaptation. The other part shows a representative execution trace for Androids-R, showing how experiences from previous runs guide CASA in subsequent run.
Figure 5: ∣ Clinical expert evaluation of anonymised speech. a, Agreement between assessments of original and anonymised recordings, using crisis-level labels for Crisis and nasality-severity ratings for PUMCH. b, Preservation rating of acoustic and semantic information on the Suicide-YF dataset. c, Anonymisation quality and sharing acceptability rating. Clinicians rated three aspects: voiceprint anonymisation, content anonymisation, and sharing acceptability. All subjective ratings were collected on a scale of 1 to 5, with higher scores indicating better performance. Evaluators were blinded to the original reference labels.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Fig. 1: ∣ Detailed workflow of CASA anonymisation. Original recordings are transcribed using an ASR module to obtain transcripts with speaker annotations and word-level timestamps. Available voiceprint anonymisation methods include pitch shifting ( Pitch ), spectral formant scaling ( Formant spec ), LPC-based formant transformation ( Formant LPC ), neural voice conversion ( Neural VC ) and transcription–resynthesis ( Resyn ). For content anonymisation, a locally deployed LLM identifies sensitive spans in the transcripts and produces annotations to guide redaction. For Resyn , which doesn’t preserve temporal alignment, sensitive content is masked in the transcript before synthesis; while for other methods, the corresponding speech segments are replaced with noise. The resulting recordings are evaluated for speaker privacy (EER), downstream clinical utility (BACC), signal cleanliness (SNR), perceptual quality (MOS), pitch-contour preservation (PCC F0 ), linguistic preservation (CER/WER) and emotion preservation (Emo).
Supplementary Fig. 2: ∣ Downstream model structure.
Supplementary Fig. 3: ∣ Web interface for crisis relabelling.
Supplementary Fig. 4: ∣ Web interface for crisis anonymisation rating.
Supplementary Fig. 5: ∣ Web interface for Suicide-YF scoring.
Supplementary Fig. 6: ∣ Web interface for PUMCH evaluation.