Self-Supervised Speech Representation Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysis
Jun 24, 2026Shu Shang, Fuliang Weng, Zeqian Hu +1Speech Processing Representation Probing
Fudan University, China · Logos & Dialogos, US
Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?
Jun 24, 2026Tomoya Mizumoto, Yusuke FujitaAudio Representation Learning Self-Supervised Speech Representation Learning
SB Intuitions
Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English
Jun 22, 2026Hamid Mojarad, Kevin TangSelf-Supervised Speech Representation Learning
Department of English Language and Linguistics, Institute of English and American Studies, Faculty of Arts and Humanities, Heinrich Heine University Dösseldorf, Germany · Department of Linguistics, University of Florida, United States of America
How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures
Jun 20, 2026Abhijit Sinha, Hemant Kumar Kathania, Mohit Joshi +3Speech Processing Self-Supervised Speech Representation Learning
aNational Institute of Technology Sikkim, India · bSignal Analysis and Interpretation Lab (SAIL), University of Southern California, USA
Online Predictive Coding for Dual-Mode Self-Supervised Speech Model
Jun 19, 2026Keita Goto, Takashi Maekaku, Jin Sakuma +3Predictive Coding Streaming ASR
LY Corporation, Tokyo, Japan · Carnegie Mellon University, PA, USA
A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition
Jun 18, 2026Nabil Mosharraf Hossain, Riasat Islam, Unaizah ObaidellahFine-Tuning Automatic Speech Recognition
Greentech Apps Foundation, United Kingdom · Greentech Apps Foundation, United Kingdom. · Queen Mary University of London, United Kingdom +3
S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning
Jun 17, 2026Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5Clustering Joint-Embedding Predictive Architecture
Carnegie Mellon University · James Silberrad Brown Center for AI · Amazon GenAI +4
Perceptual compensation for tonal context in self-supervised speech models
Jun 16, 2026James Kirby, Ioana Krehan, Michele GubianSupervised Fine-Tuning Speech Prosody
Institute for Phonetics and Speech Processing, LMU Munich, Germany
SpeechDx: A Multi-Task Benchmark for Clinical Speech AI
Jun 15, 2026Sejal Bhalla, Larry Kieu, Aina Merchant +2Speech Processing Self-Supervised Speech Representation Learning
University of Toronto, Canada
ArtNet: A JEPA-Like Articulatory Predictive Framework for Robust Zero-Shot Phoneme Recognition
Jun 15, 2026Zeqian Hu, Fuliang Weng, Shu Shang +1Zero-Shot Learning Joint-Embedding Predictive Architecture
Fudan University, China · Pedawise, Shanghai, China
From Tokens to Faces: Investigating Discrete Speech Representations for 3D Facial Animation
Jun 11, 2026Pedro Correa, Olivier Perrotin, Samir Sadok +2Audio Representation Learning Neural Audio Codecs
Univ. Estadual de Campinas (UNICAMP), Brazil · Univ. Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, France · Inria at Univ. Grenoble Alpes, CNRS, LJK, France
From Physics to Representation: Audio Learning with Synthetic Pre-training via Procedural Generation
Jun 11, 2026Fengrui Liu, Ruiyang Huang, Qijian Zheng +2Synthetic Data Pretraining Audio Representation Learning
East China Normal University · Southeast University · Fudan University +1
Dolph2Vec: Self-Supervised Representations of Dolphin Vocalizations
Jun 10, 2026Chiara Semenzin, Faadil Mustun, Roberto Dessi +5Audio Representation Learning Audio Understanding
École Normale Supérieure, Paris, France · Not Diamond, San Francisco, USA · Institut du Cerveau, Paris, France +1
Pretrained self-supervised speech models can recognize unseen consonants
Jun 10, 2026Chihiro Taguchi, Éric Le Ferrand, Hirosi Nakagawa +4Automatic Speech Recognition Self-Supervised Speech Representation Learning
University of Notre Dame, USA · University at Buffalo, USA · Tokyo University of Foreign Studies, Japan +3
Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domains
Jun 9, 2026Zilai Wang, Natarajan Balaji Shankar, Mohan Shi +2Domain Adaptation for ASR Domain Adaptation
University of California, Los Angeles, USA
Speaker Group Encoding in Self-supervised Speech Recognition Models
Jun 9, 2026Felix Herron, Solange Rossato Alexandre Allauzen, Benoit Favre +1Audio Representation Learning Demographic Bias in ASR
MILES Team, LAMSADE, Université Paris Dauphine-PSL, France (1) · GETALP Team, LIG, Université Grenoble Alpes, France (2) · NLP team, LIS, Aix-Marseille University, France (3)
ViP-VL: Vietnamese Self-supervised Speech Pretraining Model with Vector-Quantization Learning
Jun 9, 2026Khanh Le, Kiet Anh Hoang, Bao Nguyen +5Audio Representation Learning Speech Processing
VinUniversity, Vietnam · UNEY, Switzerland
Automated Pronunciation Evaluation for Korean Toddler Speech using Speech Diarization and Self-Supervised Learning
Jun 8, 2026Diane Myung-kyung Woodbridge, Jee Hyun SuhSpeech Processing Self-Supervised Speech Representation Learning
Masters in Data Science and Artificial Intelligence Program University of San Francisco · Department of Rehabilitation Medicine Seoul National University Bundang Hospital
A Comparison of SSL-Based Feature Extractors and Back-End Classifiers for Spoofing Detection: A Multi-Corpus Training and Cross-Linguistic Analysis
Jun 7, 2026Anh-Tuan Dao, Driss Matrouf, Mickael Rouvier +1Audio Deepfake Detection Domain Generalization
Laboratoire Informatique d’Avignon, Avignon Universite, France · EURECOM, Sophia Antipolis, France
Leveraging Soft Distributions of SSL-Derived Discrete Speech Tokens for Downstream Inference
Jun 5, 2026Kentaro Onda, Satoru Fukayama, Daisuke Saito +1Automatic Speech Recognition Audio Tokenization
The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan
Context-aware child-directed speech detection from long-form recordings
May 31, 2026Théo Charlot, Tarek Kunze, Kaveri K. Sheth +2Speech Processing Audio Understanding
LSCP, DEC, ENS, EHESS, CNRS, PSL University, France · Laboratoire d’Informatique et Syst`emes, Universit´e Aix-Marseille, CNRS, France
Beyond Binary: Speech Representations Across the Cognitive Score Hierarchy
May 26, 2026Serli Kopar, Roshan Prakash Rane, Christian Mychajliw +6Speech Processing Audio Understanding
Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany · Tübingen AI Center, University of Tübingen, Tübingen, Germany · Department of Psychology, Humboldt-Universität zu Berlin +8
MixFake: Benchmarking and Enhancing Audio Deepfake Detection in Diverse Real-world Mixed Audio
May 22, 2026Qingcao Li, Yipeng Lin, Weichen Lian +3Audio Deepfake Detection Prompt Tuning
School of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China
Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study
May 13, 2026Wuao Liu, Mustafa Chasmai, Subhransu Maji +1Masked Autoencoders Bioacoustics
University of Massachusetts Amherst
PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization
May 7, 2026Adhiraj Banerjee, Vipul AroraAudio Tokenization Self-Supervised Speech Representation Learning
Department of Electrical Engineering Indian Institute of Technology, Kanpur
WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
May 7, 2026Guanrou Yang, Tian Tan, Qian Chen +12Audio Representation Learning TTS Synthesis
Shanghai Jiao Tong University · Independent Researcher · Shanghai Innovation Institute +4
A Comprehensive Analysis of Tokenization and Self-Supervised Learning in End-to-End Automatic Speech Recognition applied on French Language
May 5, 2026Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa +1ASR Evaluation Automatic Speech Recognition
Nantes University LS2N Nantes, France · Avignon University LIA Avignon, France · Le Mans University LIUM Le Mans, France
Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models
May 4, 2026Sandra Arcos-Holzinger, Sarah M. Erfani, James Bailey +1Intrinsic Dimensionality Unsupervised Anomaly Detection
University of Melbourne, School of Computing and Information Systems, Australia · Monash University, Department of Data Science and Artificial Intelligence, Australia · Johns Hopkins University, Center for Language and Speech Processing, USA
Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition models
Apr 24, 2026Felix Herron, Solange Rossato, Alexandre Allauzen +1Demographic Bias in ASR Automatic Speech Recognition
MILES Team, LAMSADE, Université Paris Dauphine-PSL, France · GETALP Team, LIG, Université Grenoble Alpes, France
Advancing automatic speech recognition using feature fusion with self-supervised learning features: A case study on Fearless Steps Apollo corpus
Apr 24, 2026Szu-Jui Chen, John H. L. HansenAutomatic Speech Recognition Self-Supervised Speech Representation Learning
Center for Robust Speech Systems Erik Jonsson School of Engineering & Computer Science,2026 University of Texas at Dallas, Richardson, 75080, TX, USA

Jun 24, 2026, cs.CLJ/K move · Enter open · S save

Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysis

Shu Shang, Fuliang Weng, Zeqian Hu, Yaqian Zhou

Fudan University, China · Logos & Dialogos, US

Speech Processing Representation Probing Self-Supervised Speech Representation Learning

Open paper AskPDF arXiv

While self-supervised speech models have achieved strong performance across speech tasks, relatively little is known about how their internal phonetic representations behave under fine-grained dialect variation. Existing probing studies typically rely on curated corpora with manual phonetic annotations, limiting their applicability to naturally occurring dialect speech. We present a case study of articulatory feature representations in a Mandarin self-supervised speech model using an entirely unlabeled probing pipeline. Phone sequences are generated using a language-agnostic universal phone recognizer and mapped to articulatory feature vectors, enabling frame-level probing without manual annotation. Our results reveal a structured pattern in articulatory feature decodability across Mandarin sub-dialects. Acoustically salient features such as labiality and stridency remain comparatively stable, whereas features associated with finer spectral distinctions exhibit larger dialect-dependent variation. This variation is driven primarily by elevated decodability for Beijing speech relative to other Mandarin sub-dialects. Layer-wise analyses further show distinct representational dynamics for these feature groups. These findings suggest that language-agnostic articulatory probing can be applied to real-world dialect corpora and that dialect sensitivity in self-supervised speech representations is unevenly distributed across articulatory dimensions.

Momentum

Related topics

Latest papers 70