Multi-Speaker ASR

ASR: Automatic Speech Recognition

Follow this topic Ask about this topic

Latest papers

14 papers1 paper in the last four weeks, down 75% on the four weeks before. 0.0% of all new papers.

Omnilingual-MsASR: Benchmarking Multimodal, Multilingual, Long-Form Multi-Speaker ASR of Omni Large Language Models
Oct 21, 2026Yuan Ge, Pei Zhang, Xize Cheng +8Multi-Speaker ASR Speech Recognition Benchmarks
Northeastern University, China · Alibaba Token Hub, Alibaba · Kunming University of Science and Technology +1
Inference-Time Target Speaker Unlearning in LLM-Based Automatic Speech Recognition
Sep 24, 2026Bo Su, Yueru Yan, Thai LePrivacy-Preserving Speech Processing Automatic Speech Recognition
Indiana University, Bloomington, USA
STAM-ASR: Speaker-Temporal Anchoring with Memory for Multi-Speaker ASR
Sep 24, 2026Victor Tolulope Olufemi, Syeda Faiza Ahmed Sara, Shammur Absar ChowdhuryMulti-Speaker ASR Audio-Language Models
Qatar Computing Research Institute (QCRI), Doha, Qatar
Pushing the Boundaries of Streaming Multi-Speaker ASR: A Systematic Study of Architectural Trade-offs
Sep 9, 2026Taejin Park, Ivan Medennikov, Kunal Dhawan +3Multi-Speaker ASR Streaming ASR
NVIDIA, Santa Clara, CA, United States
Zero-Shot Temporal Localisation of Audio Deepfakes in Multi-Speaker Conversations
Sep 9, 2026Soumyadeep RoyAudio Deepfake Detection Multi-Speaker ASR
Postgraduate Department of Data Science, St. Xavier’s College (Autonomous), Kolkata, India · Institute for Advancing Intelligence (IAI), TCG CREST, Kolkata
Soft Posterior Speaker Injection for Multi-Talker Speech Recognition
Sep 1, 2026Jian Zhu, Jun Sun, Jiang Yang +6Multi-Speaker ASR Automatic Speech Recognition
Zhejiang Lab, Hangzhou, China · Zhejiang International Studies University, Hangzhou, China · University of Science and Technology of China, Hefei, China
DiaScriber: A Speech LLM for Joint Diarization and Transcription in Multi-Speaker Scenarios
Aug 24, 2026Bingshen Mu, Xian Shi, Xiong Wang +6Multi-Speaker ASR Automatic Speech Recognition
Independent Researcher
Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correction
Aug 23, 2026Hermann Yepdjio Nkouanga, Minwei Luo, Maggie Wigness +1Multi-Speaker ASR Automatic Speech Recognition
Portland State University, USA · US Army Research Laboratory, USA
On the Role of Conversational Timing in Synthetic Training Data for ASR
Jul 9, 2026Máté Gedeon, Péter MihajlikMulti-Speaker ASR Automatic Speech Recognition
Dept. of Telecommunications and Artificial Intelligence, Budapest University of Technology and Economics, Hungary · Speechtex Ltd.
H-SAGE: Holistic Speaker-Aware Guided Experts for MoE-based Multi-Talker ASR
Jul 2, 2026Yujie Guo, Jiaming Zhou, Yuhang Jia +2Multi-Speaker ASR Automatic Speech Recognition
TMCC, College of Computer Science, Nankai University, Tianjin, China
Balancing ASR and diarization in end-to-end LLMs for multi-talker speech recognition
Jun 11, 2026Naijun Zheng, Yuke Lin, Sanli Tian +4Speaker-Attributed ASR Multi-Speaker ASR
Huawei Technologies, China
Beyond Acoustic Prefixes: Persistent Access to Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognition
Mar 28, 2026Hao Shi, Yuan Gao, Xugang Lu +1Multi-Speaker ASR Memory-Augmented Language Models
Graduate School of Informatics, Kyoto University, Kyoto, Japan · National Institute of Information and Communications Technology, Japan
Distilling LLM Semantic Priors into Encoder-Only Multi-Talker ASR with Talker-Count Routing
Mar 11, 2026Hao Shi, Yusuke Fujita, Roman Koshkin +4Multi-Speaker ASR Connectionist Temporal Classification
SB Intuitions, Tokyo, Japan
TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding
Jan 11, 2026Mingyue Huo, Yiwen Shao, Yuheng ZhangMulti-Speaker ASR Audio Understanding
University of Illinois Urbana-Champaign · Johns Hopkins University

Oct 21, 2026J/K move · Enter open · S save

Omnilingual-MsASR: Benchmarking Multimodal, Multilingual, Long-Form Multi-Speaker ASR of Omni Large Language Models

Yuan Ge, Pei Zhang, Xize Cheng, Lingzhou Mu+7

Northeastern University, China · Alibaba Token Hub, Alibaba · Kunming University of Science and Technology+1

Multi-Speaker ASR Speech Recognition Benchmarks Long-Form Speech Recognition Multimodal Large Language Models Audio-Visual Speech Recognition Multilingual ASR

Open paper AskPDF arXiv

Omni-modal large language models (LLMs) have achieved remarkable progress in audio and video understanding. However, existing models struggle with long-form, multi-speaker videos, especially in understanding the entire video and recognizing who says what and when. To characterize the capability limit of current omni LLMs and guide future model development, we introduce Omnilingual-MsASR, a multilingual, multimodal benchmark for long-form Multi-speaker Automatic Speech Recognition (MsASR) covering six languages, recordings up to 30 minutes, and scenarios with up to 20 speakers. Omnilingual-MsASR provides parallel audio and video inputs to evaluate models' cross-modal consistency and their ability to exploit complementary cues across modalities. Extensive evaluations of leading omni-modal LLMs reveal substantial performance bottlenecks, while their tcpWERs frequently exceed 100%. These findings highlight fine-grained temporal localization, speaker disambiguation, and cross-modal synergy as key challenges for future omni-modal foundation models. Our work is available at https://github.com/QwenLM/Omnilingua-Bench/tree/main/Omnilingua-MSpeaker.