Acoustic Context

Acoustic context, the influence of surrounding sounds on the perception and processing of a target sound, is a crucial area of research impacting diverse fields like speech recognition, audio captioning, and sound synthesis. Current research focuses on developing models, often employing neural networks (including transformers and attention-based architectures), to effectively incorporate contextual information across various timescales and modalities (audio-visual, audio-textual). This work aims to improve the accuracy and efficiency of audio processing tasks, leading to advancements in applications such as more natural-sounding speech synthesis, improved speech recognition in noisy environments, and more realistic audio rendering in virtual and augmented reality.

Papers

June 8, 2024

Mmm whatcha say? Uncovering distal and proximal context effects in first and second-language word perception using psychophysical reverse correlation
Paige Tuttösí, H. Henny Yeung, Yue Wang, Fenqi Wang, Guillaume Denis, Jean-Julien Aucouturier, Angelica Lim
Prosodic Feature Speech Perception Acoustic Context Context Effect Psychophysical Task

November 14, 2023

Zero-shot audio captioning with audio-language model guidance and audio context keywords
Leonard Salewski, Stefan Fauth, A. Sophia Koepke, Zeynep Akata
Zero Shot Audio Captioning Captioning Method Audio Language Model Acoustic Context Descriptive Caption

October 24, 2023

How Much Context Does My Attention-Based ASR System Need?
Robert Flynn, Anton Ragni
Speech Recognition Acoustic Model Context Length Speech Recognition Performance Acoustic Context

September 27, 2023

Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fields
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, Chenliang Xu
Neural Field Room Impulse Response High Fidelity Audio Acoustic Context Acoustic Property Sound Propagation

June 30, 2023

Empirical Interpretation of the Relationship Between Speech Acoustic Context and Emotion Recognition
Anna Ollerenshaw, Md Asif Jalal, Rosanna Milner, Thomas Hain
Emotion Recognition Speech Emotion Recognition Human Relationship Empirical Analysis Speech Emotion Acoustic Context

May 3, 2023

M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis
Jinlong Xue, Yayue Deng, Fengping Wang, Ya Li, Yingming Gao, Jianhua Tao, Jianqing Sun, Jiaen Liang
End to End Prosodic Feature Acoustic Context

March 7, 2023

Face: Fast, Accurate and Context-Aware Audio Annotation and Classification
M. Mehrdad Morsali, Hoda Mohammadzade, Saeed Bagheri Shouraki
Classification Code Human Face Environmental Sound Sound Classification Acoustic Context

January 4, 2023

Object Segmentation with Audio Context
Kaihui Zheng, Yuqing Ren, Zixin Shen, Tianxu Qin
Object Segmentation Video Instance Segmentation Modal Feature Acoustic Context Audio Visual Instance

November 4, 2022

Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contexts
Detai Xin, Sharath Adavanne, Federico Ang, Ashish Kulkarni, Shinnosuke Takamichi, Hiroshi Saruwatari
Synthesized Speech Prosodic Feature Acoustic Context Textual Context Japanese Corpus Audiobook Speech Synthesis

October 28, 2022

Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders
Jason Fong, Yun Wang, Prabhav Agrawal, Vimal Manohar, Jilong Wu, Thilo Köhler, Qing He
Synthesized Speech Speaker Embeddings Speaker Identity Utterance Level Acoustic Context Audio Editing Text Based Speech Editing

July 3, 2022

Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASR
Kun Wei, Yike Zhang, Sining Sun, Lei Xie, Long Ma
Automatic Speech Recognition Cross Modal Contextual Representation Acoustic Context

June 8, 2022

Few-Shot Audio-Visual Learning of Environment Acoustics
Sagnik Majumder, Changan Chen, Ziad Al-Halah, Kristen Grauman
Acoustic Environment Impulse Response Audio Visual Learning Acoustic Context Acoustic Measurement

April 1, 2022

WavFT: Acoustic model finetuning with labelled and unlabelled data
Utkarsh Chauhan, Vikas Joshi, Rupesh R. Mehta
Fine Tuning Acoustic Model Self Supervised Learning Method Acoustic Representation Acoustic Context

December 26, 2021

AIDA: An Active Inference-based Design Agent for Audio Processing Algorithms
Albert Podusenko, Bart van Erp, Magnus Koudahl, Bert de Vries
Experimental Design Active Inference Audio Processing Personalized Speech Enhancement Acoustic Context

Acoustic Context

Papers

Mmm whatcha say? Uncovering distal and proximal context effects in first and second-language word perception using psychophysical reverse correlation

Zero-shot audio captioning with audio-language model guidance and audio context keywords

How Much Context Does My Attention-Based ASR System Need?

Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fields

Empirical Interpretation of the Relationship Between Speech Acoustic Context and Emotion Recognition

M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis

Face: Fast, Accurate and Context-Aware Audio Annotation and Classification

Object Segmentation with Audio Context

Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contexts

Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders

Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASR

Few-Shot Audio-Visual Learning of Environment Acoustics

WavFT: Acoustic model finetuning with labelled and unlabelled data

AIDA: An Active Inference-based Design Agent for Audio Processing Algorithms