Audio Representation Learning

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 233% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 141

All topics

Do speech foundation models perceive speaker similarity as humans do?
Jun 4, 2026Minoru Kishi, Hayato Yagi, Shinnosuke Takamichi +1Audio Representation Learning Speech Foundation Models
Keio University, Japan · The University of Tokyo, Japan
InfoShield: Privacy-Preserving Speech Representations for Mental Health Screening via Information-Theoretic Optimization
Jun 4, 2026Xueyang Wu, Siyuan Liu, Kezhuo Yang +1Audio Representation Learning Depression Detection
Shenzhen NeurStar Inc., China · University of York, United Kingdom · Shanghai Jiao Tong University, China
Probing Spatial Structure in Pretrained Audio Representations
Jun 4, 2026Chuyang Chen, Sivan Ding, Adrian S. Roman +1Audio Representation Learning Representation Probing
Music and Audio Research Laboratory, New York University, USA
CleanCodec: Efficient and Robust Speech Tokenization via Perceptually Guided Encoding
Jun 3, 2026Eugene Kwek, Feng Liu, Rui Zhang +1Audio Token Compression Audio Representation Learning
Pennsylvania State University · Drexel University
EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement
Jun 1, 2026Hui Li, Yangfan Gao, Junlin Shang +4Audio Representation Learning Neural Audio Codecs
Fudan University
Echo: A Joint-Embedding Predictive Architecture for Speaker Diarization and Speech Recognition in a Shared Latent Space
Jun 1, 2026Louis MouchonAudio Representation Learning Joint-Embedding Predictive Architecture
Independent Research
UniAudio-Token: Empowering Semantic Speech Tokenizers with General Audio Perception
May 29, 2026Yuhan Song, Linhao Zhang, Aiwei Liu +6Audio Representation Learning Audio Understanding
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Basic Model Technology Center, WeChat AI, Tencent Inc.
Extracting accent features in spoken Brazilian Portuguese without sociolinguistic labels
May 28, 2026Pedro H. L. Leite, Pedro Benevenuto Valadares, Luiz W. P. BiscainhoAudio Representation Learning Speech Processing
PEE/COPPE, UFRJ, Rio de Janeiro-RJ · Faculdade de Engenharia Elétrica e Computação (FEEC), UNICAMP, Campinas-SP · DEL/Poli & PEE/COPPE, UFRJ, Rio de Janeiro-RJ
HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding
May 28, 2026Bohan Li, Shi Lian, Hankun Wang +6Audio Representation Learning Speech Foundation Models
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, China · hi lab, Xiaohongshu Inc, China
MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables
May 28, 2026Sung-Lin Yeh, Wei Zhou, Gil Keren +6Audio Representation Learning Autoregressive Language Modeling
University of Edinburgh · *Work done at Meta · 2Google DeepMind +1
LoSATok: Low-dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation
May 27, 2026Zhisheng Zhang, Xiang Li, Yixuan Zhou +3Audio Token Compression Audio Representation Learning
1Shenzhen International Graduate School, Tsinghua University, China · *Work conducted while interning at ModelBest. · 2ModelBest Inc., China
Dasheng AudioGen: A Unified Model for Generating Coherent Audio Scenes from Text
May 27, 2026Jiahao Mei, Heinrich Dinkel, Yadong Niu +7Text-to-Audio Generation Audio Representation Learning
1X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · 2MiLM Plus, Xiaomi Inc., Beijing, China
MERIT: Learning Disentangled Music Representations for Audio Similarity
May 26, 2026Abhinaba Roy, Junyi Liang, Dorien HerremansDisentangled Representation Learning Audio Representation Learning
Singapore University of Technology and Design
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
May 26, 2026Minh K. Quan, Pubudu N. PathiranaContrastive Learning Audio Representation Learning
School of Engineering, Deakin University Waurn Ponds, Australia
Rethinking Continual Learning for Speech and Audio: A Representation-Centric Taxonomy and Open Problems
May 24, 2026Yang Xiao, Siyi Wang, Eun-Jung Holden +1Audio Representation Learning Speech Foundation Models
University of Melbourne, Melbourne, Australia.
SAME: A Semantically-Aligned Music Autoencoder
May 18, 2026Julian D. Parker, Zach Evans, CJ Carr +4Audio Representation Learning Neural Audio Codecs
Stability AI
SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning
May 14, 2026KiHyun Nam, Jungwoo Heo, Siu Bae +2Audio Representation Learning Speaker Verification
Korea Advanced Institute of Science and Technology (KAIST) · University of Seoul
AudioMosaic: Contrastive Masked Audio Representation Learning
May 14, 2026Hanxun Huang, Qizhou Wang, Xingjun Ma +3Audio Representation Learning Self-Supervised Pre-Training
School of Computing and Information Systems, The University of Melbourne, Australia · Institute of Trustworthy Embodied AI, Fudan University, China · Baskin School of Engineering, University of California, Santa Cruz, USA
Exploring Token-Space Manipulation in Latent Audio Tokenizers
May 11, 2026Francesco Paissan, Luca Della Libera, Mirco Ravanelli +1Audio Representation Learning Neural Audio Codecs
1Mila – Québec AI Institute · 2Université Laval · 3Concordia University
AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
May 11, 2026Jiacheng Shi, Hongfei Du, Xinyuan Song +3Audio Representation Learning Neural Audio Codecs
College of William & Mary · Emory University · George Mason University
Multi-layer attentive probing improves transfer of audio representations for bioacoustics
May 11, 2026Marius Miron, David Robinson, Masato Hagiwara +15Audio Representation Learning Bioacoustics
Earth Species Project
PoDAR: Power-Disentangled Audio Representation for Generative Modeling
May 11, 2026Alejandro Luebs, Mithilesh Vaidya, Ishaan Kumar +5Audio Representation Learning Latent Diffusion Models
Descript
Voice Biomarkers for Depression and Anxiety
May 11, 2026Oleksii Abramenko, Noah D. Stein, Colin VazAudio Representation Learning Depression Detection
machine learning team at Kintsugi Mindful Wellness, Inc.
BeeVe: Unsupervised Acoustic State Discovery in Honey Bee Buzzing
May 8, 2026Hamze Hammami, Nidhal AbdulazizAudio Representation Learning Unsupervised Clustering
TARNet: A Temporal-Aware Multi-Scale Architecture for Closed-Set Speaker Identification
May 8, 2026Yassin Terraf, Youssef IraqiAudio Representation Learning Temporal Convolutional Networks
College of Computing, University Mohammed VI Polytechnic, Benguerir, Morocco · CID Development, Rabat, Morocco
WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
May 7, 2026Guanrou Yang, Tian Tan, Qian Chen +12Audio Representation Learning TTS Synthesis
Shanghai Jiao Tong University · Independent Researcher · Shanghai Innovation Institute +4
Benchmarking LLMs on the Massive Sound Embedding Benchmark (MSEB)
May 6, 2026Cyril Allauzen, Tom Bagby, Georg Heigold +2Audio-Language Model Evaluation LLM Evaluation
Google USA & Germany
PHALAR: Phasors for Learned Musical Audio Representations
May 5, 2026Davide Marincione, Michele Mancusi, Giorgio Strano +4Audio Representation Learning Representation Learning
Department of Computer Science, Sapienza University of Rome, Italy · Moises Systems, Inc. · Paradigma, Inc.
Shao: Scaling Acoustic Token Language Models Toward High-Fidelity Music Generation
May 3, 2026Jiafeng Liu, Yuanliang Dong, Hongjia Liu +8Text-to-Music Generation Audio Representation Learning
1Central Conservatory of Music · 2Tsinghua University
LASE: Language-Adversarial Speaker Encoding for Indic Cross-Script Identity Preservation
May 1, 2026Venkata Pushpak Teja MentaAdversarial Training Adversarial Representation Learning
Praxel Ventures

Jun 4, 2026, cs.SDJ/K move · Enter open · S save

Do speech foundation models perceive speaker similarity as humans do?

Minoru Kishi, Hayato Yagi, Shinnosuke Takamichi, Yuki Saito

Keio University, Japan · The University of Tokyo, Japan

Audio Representation Learning Speech Foundation Models Speech Processing

Open paper AskPDF arXiv

This study presents a comparative analysis between the speaker embeddings of speech foundation models and human subjective perception of speaker similarity. Human listeners have the ability to judge speaker similarity on a continuous scale discerning how similar two voices are. In contrast, speech foundation models embed speaker characteristics into numerical representation. However, a question remains: does the numerical distance between speaker embeddings in these models truly align with the similarity perceived by humans? To address this, we conduct a comprehensive investigation using more than 40 models to compare model-derived distances with human-perceived similarity scores. Furthermore, we identify which factors in model configuration contribute most to a speaker embedding that mirrors human perception. Our findings provide insights for the development of more perceptually grounded speech foundation models.

Momentum

Related topics

Latest papers 141