Self Supervised Speech Representation Model

Self-supervised speech representation models learn powerful representations of speech from vast amounts of unlabeled audio data, aiming to improve various downstream tasks like speech recognition and synthesis. Current research focuses on adapting these models for low-resource languages, enhancing noise robustness, and disentangling speaker characteristics from content for improved speech coding and paralinguistic analysis. These advancements are significantly impacting fields like hearing aid technology, cross-lingual speech processing, and animal vocalization analysis, demonstrating the broad applicability of these models.

Papers

September 30, 2024

AfriHuBERT: A self-supervised speech representation model for African languages
Jesujoba O. Alabi, Xuechen Liu, Dietrich Klakow, Junichi Yamagishi
Automatic Speech Recognition Speech Data Automatic Speech Recognition Model Language Identification African Language Self Supervised Speech Representation Model

July 18, 2024

Using Speech Foundational Models in Loss Functions for Hearing Aid Speech Enhancement
Robert Sutherland, George Close, Thomas Hain, Stefan Goetze, Jon Barker
Loss Function Speech Enhancement Speech Intelligibility Self Supervised Speech Representation Speech Foundation Model Speech Enhancement Model Self Supervised Speech Representation Model

April 29, 2024

Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification
Artem Abzaliev, Humberto Pérez Espinosa, Rada Mihalcea
Speech Recognition Speech Processing Human Speech Self Supervised Speech Representation Model Bark Removal

January 10, 2024

Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapters
Kenichi Fujita, Hiroshi Sato, Takanori Ashihara, Hiroki Kanagawa, Marc Delcroix, Takafumi Moriya, Yusuke Ijima
Critical Synthesis Adapter Module Zero Shot Text to Speech Noise Robustness Self Supervised Speech Representation Model

October 17, 2023

Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audio
Antoni Dimitriadis, Siqi Pan, Vidhyasaharan Sethu, Beena Ahmed
Self Supervised Speech Representation Learning Single Channel Audio Single Speaker Acoustic Domain Speaker Localization Multi Channel Audio Self Supervised Speech Representation Model Hidden Unit BERT

May 18, 2023

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Heng-Jui Chang, Alexander H. Liu, James Glass
Speech Recognition Speech Representation Semantic Representation Acoustic Unit Self Supervised Speech Representation Model

May 9, 2023

Exploration of Language Dependency for Japanese Self-Supervised Speech Representation Models
Takanori Ashihara, Takafumi Moriya, Kohei Matsuura, Tomohiro Tanaka
Self Supervised Environment Exploration Automatic Speech Recognition Performance Monolingual Model Cross Lingual Model Self Supervised Speech Representation Model Language Driven

April 24, 2023

Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation model
Kenichi Fujita, Takanori Ashihara, Hiroki Kanagawa, Takafumi Moriya, Yusuke Ijima
Zero Shot Speech Representation Critical Synthesis Zero Shot Text to Speech Voice Style Transfer Self Supervised Speech Representation Model

November 22, 2022

Disentangled Feature Learning for Real-Time Neural Speech Coding
Xue Jiang, Xiulian Peng, Yuan Zhang, Yan Lu
Neural Audio Neural Speech Low Bitrate Self Supervised Speech Representation Model

October 21, 2022

Combining Contrastive and Non-Contrastive Losses for Fine-Tuning Pretrained Models in Speech Analysis
Florian Lux, Ching-Yi Chen, Ngoc Thang Vu
Fine Tuning Contrastive Loss Speech Analysis Contrastive Example Emotional Speech Unlabeled Speech Paralinguistic Feature Self Supervised Speech Representation Model

December 27, 2021

Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech
Gaoussou Youssouf Kebe, Luke E. Richards, Edward Raff, Francis Ferraro, Cynthia Matuszek
Natural Language Language Grounding Language Acquisition Speech Input Perceptual Information Self Supervised Speech Representation Model

Self Supervised Speech Representation Model

Papers

AfriHuBERT: A self-supervised speech representation model for African languages

Using Speech Foundational Models in Loss Functions for Hearing Aid Speech Enhancement

Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification

Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapters

Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audio

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

Exploration of Language Dependency for Japanese Self-Supervised Speech Representation Models

Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation model

Disentangled Feature Learning for Real-Time Neural Speech Coding

Combining Contrastive and Non-Contrastive Losses for Fine-Tuning Pretrained Models in Speech Analysis

Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech