Speaker Disentanglement

Ask about this topic

Overview

Speaker disentanglement studies how to represent who is speaking separately from what they say and how they say it, so systems can preserve or suppress identity as needed. Common approaches include adversarial identity removal, orthogonality and covariance constraints, paired-utterance consistency, normalization, and sparse or bottleneck representations. These methods are used in speech synthesis and conversion, tokenization, verification, separation, deepfake detection, and speech-based health monitoring. A central challenge is balancing identity removal with intelligibility, naturalness, and control, while checking for information leakage and generalization across speakers, accents, languages, and recording conditions.

Latest papers

CardsList

17 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Disentangling Linguistic and Paralinguistic Information with Routed Sparse Autoencoders

    Oct 7, 2026Beimnet Bekele Guta, Xiaoyu Yang, Guangzhi Sun +1Disentangled Representation LearningSelf-Supervised Speech Representation Learning

    Department of Engineering, University of Cambridge, Cambridge, UK

  2. DEFINE: Exemplar-Guided Accent Control for Zero-Shot TTS

    Sep 26, 2026Ambuj Mehrish, Abhinaba Roy, Alex Ivanov +2TTS SynthesisZero-Shot TTS

    Ca’ Foscari University of Venice · Singapore University of Technology and Design · Kandinsky Lab +1

  3. A Comprehensive Study of Content Representations for Speech Synthesis

    Sep 25, 2026Diego Torres, Axel Roebel, Nicolas ObinAudio Representation LearningDiscrete Speech Representation Learning

    STMS Lab IRCAM, CNRS, Sorbonne Universit´e Paris, France

  4. Content is What Remains: Invariant Speech Tokenization from Parallel Utterances

    Jul 21, 2026Laurin Wagner, Bernhard Thallinger, Miroslav Stankovic +1Audio TokenizationSelf-Supervised Speech Representation Learning

    nyra labs · nyra labs, Austria

  5. Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

    Jul 5, 2026Ryota Komatsu, Kota Kawakita, Takuma Okamoto +1Audio TokenizationSelf-Supervised Speech Representation Learning

    Institute of Science Tokyo, Meguro, Tokyo 152-8550, Japan · National Institute of Information and Communications Technology, Kyoto 619-0289, Japan

  6. Beyond Cross-Reconstruction: Probing-Based Disentanglement Evaluation for Acoustic Teleportation Codecs

    Jun 30, 2026Philipp Grundhuber, Emanuël A. P. HabetsDisentangled Representation LearningNeural Audio Codecs

    Fraunhofer Institute for Integrated Circuits (IIS), Erlangen, Germany · International Audio Laboratories Erlangen∗, Erlangen, Germany

  7. SDP-Codec: A Speaker-Decoupled Speech Codec with Pitch Injection for Low-Bitrate Coding and Zero-Shot Voice Conversion

    Jun 19, 2026Hounsu Kim, Juhan NamNeural Audio CodecsSpeech Processing

    Graduate School of Culture Technology, KAIST, Daejeon, South Korea

  8. Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detection

    Jun 15, 2026Zhuodong Liu, Hugen Lv, Xiangyu Li +1Representation LearningAudio Deepfake Detection

    Beijing Jiaotong University, China · Shanghai Jiao Tong University, China · ITMO University, Russia

  9. Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations

    Jun 4, 2026Naman Kothari, Arjun Gangwar, Adarsh Arigala +1TTS SynthesisMultilingual TTS Synthesis

    National Institute of Technology, Trichy · Indian Institute of Technology, Madras

  10. Speaker-Disentangled Remote Speech Detection of Asthma and COPD Exacerbations

    May 16, 2026Yuyang Yan, Sami O. Simons, Visara UroviSpeech ProcessingClinical Prediction

    Institute of Data Science, Maastricht University, Paul-Henri Spaaklaan 1, Maastricht, 6229 EN, the Netherlands · Department of Respiratory Medicine, NUTRIM Research Institute of Nutrition and Translational Research in Metabolism, Faculty of Health Medicine and Life Sciences, Maastricht University, P. Debyelaan 25, Maastricht, 6229 HX, the Netherlands · Department of Respiratory Medicine, Maastricht University Medical Centre, P. Debyelaan 25, Maastricht, 6229 HX, the Netherlands

  11. Mixed-Precision Information Bottlenecks for On-Device Trait-State Disentanglement in Bipolar Agitation Detection

    May 4, 2026Joydeep ChandraMixed-Precision QuantizationInformation Bottleneck

    Department of Computer Science & Technology, Tsinghua University, Beijing, China.

  12. Asymmetric Encoder-Decoder Based on Time-Frequency Correlation for Speech Separation

    Mar 31, 2026Ui-Hyeop Shin, Hyung-Min ParkSpeech SeparationSpeaker Disentanglement

    The authors are with the Department of Electronic Engineering, Sogang University, Seoul 04107, Republic of Korea

  13. Disentangling Speaker Traits for Deepfake Source Verification via Chebyshev Polynomial and Riemannian Metric Learning

    Mar 23, 2026Xi Xuan, Wenxin Zhang, Zhiyu Li +3Speaker DisentanglementMetric Learning

    University of Eastern Finland, Finland · University of Illinois Urbana-Champaign, USA · University of Chinese Academy of Sciences, China +2

  14. SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection

    Mar 21, 2026Kyudan Jung, Jihwan Kim, Minwoo Lee +4Audio Deepfake DetectionSpeaker Disentanglement

    KAIST AI · NAVER Cloud

  15. Universal Speech Content Factorization

    Mar 9, 2026Henry Li Xinyuan, Zexin Cai, Lin Zhang +5Voice ConversionSpeaker Disentanglement

    Human Language Technology Center of Excellence (COE), Johns Hopkins University, USA · Center for Language and Speech Processing, Johns Hopkins University, USA

  16. Koopman Regularized Deep Speech Disentanglement for Speaker Verification

    Mar 5, 2026Nikos Chazaridis, Mohammad Belal, Rafael Mestre +2Speaker VerificationSpeaker Disentanglement

    School of Electronics and Computer Science, University of Southampton, SO17 1BJ Southampton, U.K. · Marine Geosciences, National Oceanography Centre, SO14 3ZH Southampton, U.K.

  17. Unified Architecture and Unsupervised Speech Disentanglement for Speaker Embedding-Free Enrollment in Personalized Speech Enhancement

    May 18, 2025Ziling Huang, Haixin Guan, Yanhua LongSpeech EnhancementSpeaker Disentanglement

    Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University · Unisound AI Technology Co., Ltd., Beijing, China