Speech Foundation Models

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 42

All topics

Backdooring Acoustic Foundation Models for Physically Realizable Triggers
Oct 7, 2026Zebin Yun, Eyal Ronen, Mahmood SharifBackdoor Attacks Adversarial Attacks
Tel Aviv University, Tel Aviv, Israel
BanglaBox: A Phonetically-Balanced Corpus and Data-Efficient Foundation-Model Adaptation for Bangla Text-to-Speech with Zero-Shot Voice Cloning
Oct 6, 2026Emtiaz Uddin Ahmed, Araf Mahmud, Sajib Hossain +3Text-to-Speech Low-Resource TTS Synthesis
Neural Representations, Natural Connections: What Transfers From Human Speech Foundation Models to Animal Vocalizations?
Oct 5, 2026Tomás Arias-Vergara, Christopher Hauer, Héloïse Brotier +3Audio Representation Learning Speech Foundation Models
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Faculty of Electrical Engineering, Media and Computer Science, OTH Amberg-Weiden, Germany · Faculty of Life Science, The Gonda Multidisciplinary Brain Research Center, Bar Ilan University, Israel
Transferable Adversarial Robustness for Speech Foundation Models via Hierarchical Stabilization
Oct 4, 2026Aref Mousavi, Shahab Sherafat, Kiarash Kiani Feriz +4Adversarial Representation Learning Speech Foundation Models
Sharif University of Technology · University of Tehran · New Uzbekistan University +1
Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition
Sep 24, 2026Asmee Mishra, Mengjie Qian, Brechtje Post +1Speech Foundation Models Low-Resource Speech Recognition
Department of Engineering, University of Cambridge, United Kingdom · Phonetics Laboratory, University of Cambridge, United Kingdom
Temporal Taxation Compounds Under Post-Training Compression of Whisper Models
Sep 23, 2026Srishti Ginjala, Eric Fosler-Lussier, Christopher W. Myers +1ASR Evaluation Speech Foundation Models
The Ohio State University, Columbus, OH, USA · Air Force Research Laboratory, USA
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
Sep 23, 2026Rasmus Aagaard, Nicki Skafte DetlefsenSpeech Foundation Models Automatic Speech Recognition
Technical University of Denmark · Laerdal Medical
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
Sep 21, 2026Lujia Bao, Qian Chen, Luyao Cheng +15Tool-Augmented Language Model Agents Speech Foundation Models
Alibaba Token Foundry, Alibaba Group
AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models
Sep 21, 2026Natarajan Balaji Shankar, Zilai Wang, Zihan Wang +3Speech Foundation Models
Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA
Listening for Airway Stenosis: A Foundation Model-Based Method for Rapid and Accessible Detection
Sep 14, 2026Jean Groeninger, Zihao Zhao, Juliana de Castilhos +2Speech Foundation Models Audio Classification
Department of Diagnostic and Interventional Radiology, University Hospital Aachen
CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models
Sep 14, 2026Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva +5Speech Foundation Models Speech Quality Assessment
AKCIT, Brazil · Federal University of Rio Grande do Norte (UFRN), Brazil · Federal University of Technology (UTFPR), Brazil +1
Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models
Sep 11, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer Interpretability Circuit Discovery
Department of Computer Science University of Toronto Toronto, Canada
Do speech foundation models really learn words?
Sep 9, 2026Robin Huo, Ewan DunbarRepresentation Learning Speech Foundation Models
University of Toronto, Canada
BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models
Sep 9, 2026Shivam Singh, Aditya Yadavalli, Catherine Arnett +1Speech Foundation Models Automatic Speech Recognition
UC San Diego · EleutherAI
AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing
Sep 8, 2026Ziyang Ma, Zhikang Niu, Wenming Tu +30Speech Foundation Models Speech Editing
Clean Accuracy Does Not Guarantee Provenance Robustness: A Prospective Codec-Stress Evaluation of Audio Attribution
Sep 7, 2026Gang ShiSpeech Foundation Models
Independent Researcher
Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection
Sep 7, 2026Maryam Abbasihafshejani, Murtuza JadliwalaSpeech Foundation Models Automatic Speech Recognition
The university of Texas at San Antonio
Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioning
Aug 12, 2026Xulin Fan, Jialu Li, Mohammad Nur Hossain Khan +4Speech Foundation Models Domain Generalization
University of Illinois Urbana-Champaign, USA · University of Arizona, USA · Worcester Polytechnic Institute, USA
MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages
Aug 5, 2026Qiongqiong Wang, Ai Ti Aw, Nancy F. Chen +18Speech Foundation Models Speech Processing
Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore
dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model
Aug 2, 2026Hankun Wang, Bohan Li, Shi Lian +7Speech Foundation Models Speech Editing
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University · Xiaohongshu Inc.
JoyAI-Talker: Full-Duplex Speech Interactive Large Model Built for Empathetic Voice Agents
Aug 2, 2026Yinhao Bai, Jinming Chen, Yafeng Chen +26Speech Foundation Models Controllable Speech Generation
Normal-Anchored First-Order Model-Agnostic Meta-Learning based Whisper Fine-Tuning for Enhancing Fairness of Cleft Lip and Palate Speech Recognition
Jul 31, 2026Susmita Bhattacharjee, Jagabandhu Mishra, H. S. Shekhawat +2Speech Foundation Models Meta-Learning
Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati, Guwahati, 781039, Assam, India · School of Computing, University of Eastern Finland, Joensuu, Finland · Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States +1
Enhancing Law-Enforcement Audio Transcription: A LoRA-Based Adaptation of Whisper for BWC Footage
Jul 27, 2026Vivek Senthil, Ernest FokouéSpeech Foundation Models Audio Understanding
Rochester Institute of Technology, Rochester, NY, USA
Improving Zero-Shot Phonetic Classification through Language-Agnostic Articulatory Features
Jul 26, 2026Ryo Magoshi, Jaeyoung Lee, Shinsuke Sakai +1Zero-Shot Learning Speech Foundation Models
Graduate School of Informatics, Kyoto University, Japan · NTT, Inc., Japan
DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages
Jul 23, 2026Paul Azunre, Naafi Ibrahim, Joel Budu +1Speech Foundation Models Automatic Speech Recognition
Khaya AI
Rethinking Speech Foundation Model Fine-tuning: Better SFT or Better Match?
Jul 15, 2026Wangjin Zhou, Yizhou Zhang, Yichi Wang +1Supervised Fine-Tuning Fine-Tuning
Graduate School of Informatics, Kyoto University, Kyoto, Japan
GigaAM Multilingual: Foundation Model for Underrepresented Languages
Jul 11, 2026Andrei Kuzmenko, Alexandr Maximenko, Aleksandr Kutsakov +5Speech Foundation Models Speech Processing
SaluteDevices, Russia
wav2VOT: Automatic estimation of voice onset time, closure duration, and burst realisation with wav2vec2
Jun 27, 2026James Tanner, Morgan Sonderegger, Jane Stuart-Smith +2Speech Foundation Models Speech Processing
University of Glasgow, United Kingdom · McGill University, Canada · University of Oregon, United States +1
Learning task-specific subspaces via interventional post-training of speech foundation models
Jun 16, 2026Jack Cox, Jon BarkerContrastive Learning Disentangled Representation Learning
University of Sheffield, United Kingdom
Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering
Jun 10, 2026Haoning Xu, Zhaoqing Li, Huimeng Wang +4Speech Foundation Models Clustering
The Chinese University of Hong Kong, Hong Kong SAR, China · National Research Council Canada, Canada

Oct 7, 2026, cs.SDJ/K move · Enter open · S save

Backdooring Acoustic Foundation Models for Physically Realizable Triggers

Zebin Yun, Eyal Ronen, Mahmood Sharif

Tel Aviv University, Tel Aviv, Israel

Backdoor Attacks Adversarial Attacks Speech Foundation Models

Open paper AskPDF arXiv

Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with minimal resources. However, the security of applications based on AFMs remains largely underexplored. Our work addresses this gap by proposing the Foundation Acoustic model Backdoor (FAB) attack, demonstrating that state-of-the-art AFMs are susceptible to backdooring under practical settings. Despite making minimal assumptions about adversary capabilities (e.g., no access to pre-training data), we show that FAB preserves benign performance while inducing backdoors that survive fine-tuning and cause significant degradation across diverse downstream tasks when activated. Notably, FAB utilizes task-agnostic, physically realizable, inconspicuous, and sync-free triggers (e.g., a background siren). We evaluate FAB using two leading AFMs, nine downstream tasks, and four different triggers. We further demonstrate its effectiveness against established defenses and across both digital and physical domains. While extensive end-to-end fine-tuning can mitigate FAB, such a defense is resource-intensive and task-specific. Our work highlights critical risks to AFMs and calls for advanced defenses.

Momentum

Related topics

Latest papers 42