Speech Supervised Learning Model

Speech supervised learning models aim to create robust and efficient representations of speech data using self-supervised learning (SSL), primarily focusing on improving downstream tasks like speech recognition and quality assessment. Current research emphasizes developing more efficient SSL architectures (e.g., HuBERT, Wav2Vec 2.0) and exploring effective evaluation metrics beyond traditional resource-intensive methods, including investigating the representation of linguistic features and addressing biases in existing models. This field is significant because improved speech representations have broad applications in areas such as assistive technologies, human-computer interaction, and language understanding, while also advancing our understanding of how these models learn and represent complex acoustic and linguistic information.

Papers

September 16, 2024

Towards Automatic Assessment of Self-Supervised Speech Models using Rank
Zakaria Aldeneh, Vimal Thilak, Takuya Higuchi, Barry-John Theobald, Tatiana Likhomanenko
Stable Rank Self Supervised Speech Model Speech Supervised Learning Model Speech Domain Encoder Layer Automatic Assessment Unsupervised Metric

September 14, 2024

The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech
Kaito Baba, Wataru Nakata, Yuki Saito, Hiroshi Saruwatari
Transfer Learning Image Classification Synthesized Speech Speech Supervised Learning Model Naturalness Assessment

September 13, 2024

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
Minglun Han, Ye Bai, Chen Shen, Youjia Huang, Mingkun Huang, Zehua Lin, Linhao Dong, Lu Lu, Yuxuan Wang
Next Token Prediction Speech Supervised Learning Model Non Streaming Automatic Speech Recognition Non Causal Nest Agent Random Projection Quantizer

August 26, 2024

Self-supervised Speech Representations Still Struggle with African American Vernacular English
Kalvin Chang, Yi-Hui Chou, Jiatong Shi, Hsuan-Ming Chen, Nicole Holliday, Odette Scharenborg, David R. Mortensen
Zero Shot Automatic Speech Recognition System Speech Model Automatic Speech Recognition Performance Self Supervised Speech Representation Speech Supervised Learning Model African American

August 24, 2024

A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models
Antón de la Fuente, Dan Jurafsky
Chinese Character Contextual Representation Layer Wise Monolingual Model Self Supervised Speech Model Speech Supervised Learning Model Categorical Structure

June 13, 2024

May 30, 2024

Fill in the Gap! Combining Self-supervised Representation Learning with Neural Audio Synthesis for Speech Inpainting
Ihab Asaad, Maxime Jacquelin, Olivier Perrotin, Laurent Girin, Thomas Hueber
Speech Analysis Speech Representation Self Supervised Representation Learning Speech Signal Speaker Recognition Neural Vocoder Speech Supervised Learning Model Neural Audio Synthesis

March 13, 2024

Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations
Amit Meghanani, Thomas Hain
Speech Model Self Supervised Speech Representation Speech Supervised Learning Model Acoustic Word Embeddings Based Correspondence

February 14, 2024

UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models
Ruchao Fan, Natarajan Balaji Shanka, Abeer Alwan
Speech Supervised Learning Model Non Autoregressive Automatic Speech Recognition

December 14, 2023

STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models
Kangwook Jang, Sungnyun Kim, Hoirin Kim
Self Supervised Learning Temporal Relation Speech Supervised Learning Model Speech Segment Speech Transformer Speech Frame

September 25, 2023

Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning
Guanrou Yang, Ziyang Ma, Zhisheng Zheng, Yakun Song, Zhikang Niu, Xie Chen
Speech Recognition Librispeech Speech Recognition Speech Supervised Learning Model Self Supervised Speech Representation Learning Hidden Unit BERT

September 22, 2023

A Study on Incorporating Whisper for Robust Speech Assessment
Ryandhimas E. Zezario, Yu-Wen Chen, Szu-Wei Fu, Yu Tsao, Hsin-Min Wang, Chiou-Shann Fuh
Study Feature Speech Intelligibility State of the Art Whisper Speech Supervised Learning Model

August 18, 2023

Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Model
Ryandhimas E. Zezario, Bo-Ren Brian Bai, Chiou-Shann Fuh, Hsin-Min Wang, Yu Tsao
Multi Task Pseudo Label Speech Quality Speech Supervised Learning Model Speech Distortion Non Intrusive Speech

July 11, 2023

On the Effectiveness of Speech Self-supervised Learning for Music
Yinghao Ma, Ruibin Yuan, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Ruibo Liu, Gus Xia, Roger Dannenberg, Yike Guo, Jie Fu
Self Supervised Speech Model Music Industry Music Information Retrieval Speech Supervised Learning Model Recent Language Model

June 14, 2023

SpeechGLUE: How Well Can Self-Supervised Speech Models Capture Linguistic Knowledge?
Takanori Ashihara, Takafumi Moriya, Kohei Matsuura, Tomohiro Tanaka, Yusuke Ijima, Taichi Asami, Marc Delcroix, Yukinori Honma
Self Supervised Speech Representation Spoken Language Understanding Speech Model Speech Supervised Learning Model Linguistic Knowledge

May 30, 2023

MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models
Yu-Hsiang Wang, Huang-Yu Chen, Kai-Wei Chang, Winston Hsu, Hung-yi Lee
Self Supervised Learning Speech Model Self Supervised Speech Model Speech Supervised Learning Model

May 28, 2023

DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models
Yifan Peng, Yui Sudo, Shakeel Muhammad, Shinji Watanabe
Knowledge Distillation Self Supervised Edge Pruning Self Supervised Speech Model Speech Supervised Learning Model Speech Processing Task Task Specific Structured Pruning Joint Distillation

May 19, 2023

Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation
Kangwook Jang, Sungnyun Kim, Se-Young Yun, Hoirin Kim
Speech Supervised Learning Model Speech Processing Task Speech Transformer Functional Compression Masked Distillation Speech Frame

May 18, 2023

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark
Jiatong Shi, Dan Berrebbi, William Chen, Ho-Lam Chung, En-Pei Hu, Wei Ping Huang, Xuankai Chang, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Shinji Watanabe
Automatic Speech Recognition Multilingual Model Speech Supervised Learning Model Speech Processing Task Universal Performance Benchmark

Speech Supervised Learning Model

Papers

Towards Automatic Assessment of Self-Supervised Speech Models using Rank

The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training

Self-supervised Speech Representations Still Struggle with African American Vernacular English

A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models

LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks

SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models

Fill in the Gap! Combining Self-supervised Representation Learning with Neural Audio Synthesis for Speech Inpainting

Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations

UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models

STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models

Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning

A Study on Incorporating Whisper for Robust Speech Assessment

Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Model

On the Effectiveness of Speech Self-supervised Learning for Music

SpeechGLUE: How Well Can Self-Supervised Speech Models Capture Linguistic Knowledge?

MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models

DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models

Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark