Speech Processing

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 433% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 295

All topics

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
Jul 7, 2026Albert Zeyer, Ralf Schlüter, Hermann NeyGradient-Based Attribution Speech Processing
Machine Learning and Human Language Technology Group, RWTH Aachen University, Aachen, Germany · AppTek GmbH, Aachen, Germany
Flow Matching-Based Speech Source Separation with Best-of-N Biometric Sampling
Jul 7, 2026Anastasia Zorkina, Alexandr Anikin, Nikita Khmelev +5Flow Matching Speech Processing
ITMO University, Speech Processing Group, Russia · Speech Technology Center Ltd., R&D department, Russia.
Listen to the Features: Voice Anonymization Driven by Content Embedding Matching over Signal Reconstruction
Jul 7, 2026Adrien Schneider, Kacper Zabkowski, Anderson Augusma +3Speech Processing Speaker Anonymization
M-PSI · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France · SAM, SVH +1
Fréchet Distance Loss on Speech Representations for Text-to-Speech Synthesis
Jul 7, 2026Ho-Lam Chung, Kuan-Po Huang, Bo-Ru Lu +1Flow Matching TTS Synthesis
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Graduate Institute of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan · Amazon
REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editing
Jul 6, 2026Cheng-Kang Chou, Ming-To Chuang, Ke-Han Lu +2ASR Evaluation Speech Processing
National Taiwan University · Carnegie Mellon University · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
DuplexChat: Constructing Speaker-Separated Full-Duplex Dialogue Speech at Scale for Spoken Dialogue Language Modeling
Jul 6, 2026Wataru Nakata, Yuki Saito, Hiroshi SaruwatariSpeech Processing Language Modeling
The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology, Japan
Information-Geometric Superposed Vowel Evaluation: Part 1. Moraic Syllabary (Japanese)
Jul 5, 2026Yusei Tamura, Shigekazu Ishihara, Ken ItoAudio Deepfake Detection Speech Processing
Graduate School of Interdisciplinary Information Studies The University of Tokyo · Department of Psychology, Faculty of Health and Wellness Sciences Hiroshima International University · Interfaculty Initiative in Information Studies The University of Tokyo
S-DiverSe: Spanish Diverse Speech
Jul 3, 2026Fernando López, Fernando Ibañez, Ana Martínez +4ASR Evaluation Speech Processing
Scientific Research, Telefónica Innovación Digital, Spain · Universidad Autónoma de Madrid, Spain · Brno University of Technology, Czech Republic
Deriving Benchmarking Datasets from Long-Form Recordings: Challenges and Opportunities
Jul 3, 2026Kaveri K. Sheth, Lawrence Borst, Tarek Kunze +6Speech Processing
LAAC, LSCP, DEC, ENS, EHESS, CNRS, PSL University, Paris, France · Laboratoire d’Informatique et Syst`emes, Universit´e Aix-Marseille, CNRS, France · Signal Processing Research Centre, Tampere University, Finland
Using embeddings to predict spoken word duration and pitch in Mandarin monosyllabic words
Jul 2, 2026Xiaoyun Jin, Mirjam Ernestus, R. Harald BaayenSpeech Processing Speech Prosody
Quantitative Linguistik, University of Tuebingen, Germany · Center for Language Studies, Radboud University, 6525 HT Nijmegen, The Netherlands
RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices
Jul 2, 2026Z. Benslimane, P. Chouteau, M. Poreba +4Speech Processing Speech Enhancement
Universit´e Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Universit´e de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France
Speech Playground: An Interactive Tool for Speech Analysis and Comparison
Jul 1, 2026Stephen McIntosh, Daisuke Saito, Nobuaki MinematsuSpeech Processing
The University of Tokyo, Japan
LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish
Jun 30, 2026Nina Hosseini-Kivanani, Sandipana DowerahSpeech Processing Low-Resource TTS Synthesis
Radio T´el´evisioun L¨etzebuerg (RTL) & University of Luxembourg · Esch-sur-Alzette, Luxembourg · Tallinn University of Technology +1
How Bilingual Are SSL Speech Models? Cross-Lingual Probing of Articulatory Encoding with Finnish and Russian EMA
Jun 30, 2026Ailín Pollio San Pedro, Tomi Kinnunen, Alexandre Nikolaev +1Speech Processing Self-Supervised Speech Representation Learning
University Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, France · University of Eastern Finland, Computational Speech Group, Finland
FlexiSLM: A Spoken Language Model with Dynamic and Controllable Frame Rates
Jun 30, 2026Jiaqi Li, Chaoren Wang, Xiaohai Tian +9Speech Processing Speech Language Models
1The Chinese University of Hong Kong, Shenzhen · 2ByteDance
UniSAE: Unified Speech Attribute Editing on Speaker, Emotion and Low-Level Content via Discrete Phonetic Posteriorgram Modelling
Jun 30, 2026Chuanbo Zhu, Wuyou Zhou, Rongxiu Zhong +4Audio Editing Speech Editing
The Hong Kong University of Science and Technology · China Mobile · Beijing Institute of Technology
Child-Centric Voice Anonymization in Single and Multi-Speaker Speech via Domain-Adapted SSL Models
Jun 29, 2026Pranav Tushar, Xiao Xiao Miao, Rong TongDomain Adaptation Speech Processing
Singapore Institute of Technology, Singapore · Duke Kunshan University, China
TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation
Jun 28, 2026Qinzhe Hu, Chenda Li, Wangyou Zhang +3Sparse Mixture-of-Experts Speech Processing
Auditory Cognition and Computational Acoustics Lab Shanghai Jiao Tong University, Shanghai, China · Microsoft Research Asia, China
Position-Aware Target Speaker Extraction for Long-Form Multi-Party Conversations: A Diarization-Free Framework for ASR
Jun 28, 2026Yichi Wang, Junzhe Chen, Wangjin Zhou +1Target Speaker Extraction Speech Processing
Graduate School of Informatics, Kyoto University, Kyoto, Japan
AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification
Jun 28, 2026Chuxiao Zuo, Yao Zhu, Minqiang Xu +3Multimodal Robustness Speech Processing
Honor Device Co., Ltd Nanjing, China · Honor Device Co., Ltd Shanghai, China
wav2VOT: Automatic estimation of voice onset time, closure duration, and burst realisation with wav2vec2
Jun 27, 2026James Tanner, Morgan Sonderegger, Jane Stuart-Smith +2Speech Foundation Models Speech Processing
University of Glasgow, United Kingdom · McGill University, Canada · University of Oregon, United States +1
wav2tok 2.0: Scalable Audio Tokenization Maintaining Explicit Pairwise Token Alignment for Efficient Audio Retrieval
Jun 25, 2026Adhiraj Banerjee, Vipul AroraAudio Representation Learning Speech Processing
Department of Electrical Engineering, Indian Institute of Technology Kanpur, India · Department of Electrical Engineering, KU Leuven, Belgium
FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
Jun 25, 2026Zhihang Xie, Marco Gaido, Sara Papi +2Audio-Language Model Evaluation Speech Processing
Fondazione Bruno Kessler · University of Trento
SE-AGCNet: An End-to-End Framework for Joint Speech Enhancement and Loudness Control in Meeting Scenarios
Jun 24, 2026Jinming Zhang, Wei Rao, Xionghu Zhong +1Speech Processing Speech Enhancement
Zhejiang University, China · Nanyang Technological University, Singapore · Hunan University, China
Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysis
Jun 24, 2026Shu Shang, Fuliang Weng, Zeqian Hu +1Speech Processing Representation Probing
Fudan University, China · Logos & Dialogos, US
Phoneme-Level Mispronunciation Screening in Polish-Speaking Children with an Explainable Assistant
Jun 23, 2026Milosz Dudek, Daria Hemmerling, Kamil Kwarciak +9Speech Processing Automatic Speech Recognition
AGH University of Krakow, Cracow, Poland · SoftServe, Cracow, Poland · Department of Biomedical Engineering, Silesian University of Technology, Poland +1
What Does a Pathological Speech Assessment Model Know about Acoustic Features? A Case Study on Oral and Oropharyngeal Cancer Patients
Jun 23, 2026Tuan Nguyen, Corinne Fredouille, Alain Ghio +2Speech Quality Assessment Interpretable ML
LIA, AU · Avignon University, LIA, UPR 4128, Avignon, France · AU, LIA +5
NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction
Jun 23, 2026Wenhao Gao, Yifan Wang, Yijia Ma +3Conditional Flow Matching EEG Decoding
Stony Brook University, Stony Brook, NY, USA · University of Texas Health Center at Houston, Houston, TX, USA · Emory University, Atlanta, GA, USA
A Fusion-Aware Two-Stage Framework for Mispronunciation Detection and Diagnosis in Low-Resource Modern Standard Arabic
Jun 23, 2026Jing Yang, Shuqing Zhang, Yongyi Deng +5Domain Adaptation Speech Processing
School of Electronic Information, Wuhan University, Wuhan Hubei, 430072, China · School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · Northwestern Polytechnical University +1
VieSpeaker: A Large-Scale Vietnamese Speaker Recognition Dataset Beyond Visual Dependency
Jun 23, 2026Viet Hoang Pham, Tran Trung Nguyen, Bao Thu Ho +2Speech Processing Low-Resource Language Processing
Hanoi University of Science and Technology, Hanoi, Vietnam

Jul 7, 2026, cs.CLJ/K move · Enter open · S save

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs

Albert Zeyer, Ralf Schlüter, Hermann Ney

Machine Learning and Human Language Technology Group, RWTH Aachen University, Aachen, Germany · AppTek GmbH, Aachen, Germany

Gradient-Based Attribution Speech Processing Automatic Speech Recognition Speech Language Models

Open paper AskPDF arXiv

Speech-to-text alignment means finding the temporal boundaries of each word in the audio. Some models provide such an alignment directly and others do not. Connectionist temporal classification (CTC) and transducer models have an alignment by construction, whereas attention-based encoder-decoders (AED) and speech large language models (LLMs) do not, and their word timings are usually read off the attention weights instead. All of these signals live on the encoder frame grid, which bounds their temporal precision. We study a generic gradient-based alignment that applies to any differentiable ASR model. We take the gradient of each teacher-forced token log probability with respect to the input, reduce it to a per-frame saliency, and decode the resulting matrix into word boundaries with a single dynamic-programming pass. The method needs no training, no model modification and no alignment heads, works across all model families including the speech LLMs, and aligns on the input grid rather than on the coarser encoder grid. We evaluate it on sixteen models from four families, on read (TIMIT) and spontaneous (Buckeye) speech, each against the model's own native or attention-based alignment. We find that the gradient yields a usable alignment for every model, that it is usually somewhat behind a strong native aligner but better where the native alignment is weak, as for the streaming models, and that its main disadvantage is the cost of one backward pass per token.

Momentum

Related topics

Latest papers 295