Spoken Dialogue Systems

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 156% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 93

All topics

BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
Jun 12, 2026Qingkai Fang, Shoutao Guo, Yang FengSpoken Dialogue Systems Speech Language Models
Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · Key Laboratory of AI Safety, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Adaptive Turn-Taking for Real-time Multi-Party Voice Agents
Jun 11, 2026Soumyajit Mitra, Prabhat Pandey, Abhinav Jain +2Spoken Dialogue Systems Speech Language Models
Amazon AGI · IIT Kharagpur, India
Endpoint Anticipation for Low-Latency Spoken Dialogue
Jun 11, 2026Sathvik Udupa, Shinji Watanabe, Petr Schwarz +1Spoken Dialogue Systems
Brno University of Technology, Czechia · Carnegie Mellon University, United States
PRISM: Prosody-Integrated Multi-Agent Reasoning Framework for Empathetic Spoken Dialogue
Jun 11, 2026Wen Zhang, Xiaocui Yang, Zhuoyue Gao +3Multi-Agent LLM Systems Spoken Dialogue Systems
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System
Jun 10, 2026Zi Haur Pang, Yahui Fu, Koji Inoue +1Emotional Support Conversation Spoken Dialogue Systems
Graduate School of Informatics, Kyoto University, Japan
Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering
Jun 9, 2026Cheng-Kuang Chang, Kai-Wei Chang, Alexander H. Liu +1Language Model Steering Spoken Dialogue Systems
MIT CSAIL
Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
Jun 9, 2026Atsumoto Ohashi, Neil Zeghidour, Alexandre Défossez +1Spoken Dialogue Systems Multi-Objective Language Model Alignment
Kyutai, Paris, France · Gradium, Paris, France
ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models
Jun 9, 2026Yuxiang Wang, Qinke Ni, Shengbo Cai +3On-Policy Self-Distillation Spoken Dialogue Systems
1The Chinese University of Hong Kong, Shenzhen · 2Tencent Hunyuan · 5Tsinghua University +2
IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems
Jun 4, 2026Tao Zhong, Jiajun Deng, Nikita Kuzmin +6Spoken Dialogue Systems Multimodal Fusion
The Chinese University of Hong Kong, China · AudioLab Hong Kong, Huawei Leibniz Research Center, China · Nanyang Technological University, Singapore
Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning
May 30, 2026Sukru Samet Dindar, Riki Shimizu, Xilin Jiang +1Spoken Dialogue Systems Controllable Speech Generation
Department of Electrical Engineering Columbia University New York, NY 10025
Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking
May 28, 2026Songbo Hu, Yinhong Liu, Ej Zhou +5Spoken Dialogue Systems RAG Evaluation
Language Technology Lab, University of Cambridge, UK · Independent Researcher · School of Computation, Information and Technology, Technical University of Munich, Germany
Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems
May 25, 2026Yizhou Peng, Ziyang Ma, Changsong Liu +3Spoken Dialogue Systems Automatic Speech Recognition
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China
STREAM: A Data-Centric Framework for Mining High-Value Task-Oriented Dialogues from Streaming Media
May 24, 2026Liang Xue, Haoyu Liu, Cheng Wang +3Spoken Dialogue Systems Synthetic Data Generation for Language Models
Harbin Institute of Technology Byering Technology Harbin, China · Harbin Institute of Technology Harbin, China
A Proactive Multi-Agent Dialogue Framework for Assessing Social Language Disorder Traits in Autism
May 21, 2026Chuanbo Hu, Minglei Yin, Bin Liu +4Multi-Agent LLM Systems Autism Spectrum Disorder
Department of Computer Science, University at Albany, Albany, 12222, NY, USA. · Department of Management Information System, West Virginia University, Morgantown, 26506, WV, USA. · Humanities and Social Sciences, California Institute of Technology, Pasadena, 91125, CA,2026 USA. +1
ReacTOD: Bounded Neuro-Symbolic Agentic NLU for Zero-Shot Dialogue State Tracking
May 18, 2026Yanjun Lin, Zimo Xiao, Kartik Natarajan +7Spoken Dialogue Systems LLM Agents
Amazon
DuIVRS-2: An LLM-based Interactive Voice Response System for Large-scale POI Attribute Acquisition
May 18, 2026Le Zhang, Shengming Zhang, Rui Zha +3Spoken Dialogue Systems LLM Agent Training
Baidu Inc., Beijing, China
VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing
May 7, 2026Jiacheng Xu, Heting Gao, Liufei Xie +8Singing Voice Synthesis Spoken Dialogue Systems
QinYu Team & VITA-Team
PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue
May 7, 2026Hyunbae Jeon, Jinho D. ChoiVoice Agent Evaluation Spoken Dialogue Systems
Department of Computer Science Emory University Atlanta, GA, USA
GEM: Graph-Enhanced Mixture-of-Experts with ReAct Agents for Dialogue State Tracking
May 6, 2026Ziqi Zhu, Adithya Suresh, Tomal Deb +1Spoken Dialogue Systems Mixture-of-Experts Models
Generative AI Innovation Center, Amazon Web Services
Proactive Dialogue Model with Intent Prediction
Apr 30, 2026Yang LuoLLM Prompting Spoken Dialogue Systems
Department of Computer Science University of Hong Kong
Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations
Apr 20, 2026Eric Rudolph, Philipp Steigerwald, Jens AlbrechtSpoken Dialogue Systems
Technische Hochschule Nürnberg Georg Simon Ohm
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
Apr 20, 2026Yun Hong, Yan Zhou, Yang FengSpoken Dialogue Systems Efficient Language Model Training
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions
Apr 19, 2026Dongwook Lee, Eunwoo Song, Che Hyun Lee +2Spoken Dialogue Systems Audio Understanding
1Interdisciplinary Program in Artificial Intelligence, Seoul National University (SNU) · Department of EE, Yonsei University · Department of ECE, SNU +2
Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use
Apr 17, 2026Ramit Pahwa, Apoorva Beedu, Parivesh Priye +4Audio-Language Model Evaluation Voice Agent Evaluation
Rivian and Volkswagen Group Technologies
VoxMind: An End-to-End Agentic Spoken Dialogue System
Apr 17, 2026Tianle Liang, Yifu Chen, Shengpeng Ji +7Multi-Agent LLM Systems Spoken Dialogue Systems
Zhejiang University · China University of Petroleum-Beijing at Karamay · Xiamen University
WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training
Apr 16, 2026Yifu Chen, Shengpeng Ji, Qian Chen +9Spoken Dialogue Systems Multi-Objective Language Model Alignment
Zhejiang University · Tongyi Fun Team, Alibaba Group · Beijing University of Technology
Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models
Apr 16, 2026Yifu Chen, Shengpeng Ji, Zhengqing Liu +6Spoken Dialogue Systems Full-Duplex Spoken Dialogue Systems
Zhejiang University · Tongyi Fun Team, Alibaba Group · Beijing University of Technology
FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection
Apr 2, 2026Chengyou Wang, Hongfei Xue, Mingchen Shao +9Spoken Dialogue Systems Audio Understanding
Shengwang · QualiaLabs
RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue
Mar 24, 2026Long Mai, Junli LiangSpoken Dialogue Systems Speculative Decoding
Trinity College Dublin, Ireland · University College Dublin, Ireland
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Mar 17, 2026Tianyu Xie, Jinfa Huang, Yuexiao Ma +11Audio-Visual Understanding Spoken Dialogue Systems
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · Department of Computer Science, University of Rochester, Rochester, NY, USA +2

Jun 12, 2026, cs.CLJ/K move · Enter open · S save

BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM

Qingkai Fang, Shoutao Guo, Yang Feng

Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · Key Laboratory of AI Safety, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

Spoken Dialogue Systems Speech Language Models Full-Duplex Spoken Dialogue Systems Turn-Taking in Dialogue

Open paper AskPDF arXiv

Real-time, full-duplex speech interaction is a key feature of next-generation spoken chatbots, allowing the model to listen and speak at the same time and to handle natural phenomena such as overlap, hesitation, and barge-in. Existing speech language models (SpeechLMs) such as LLaMA-Omni and GLM-4-Voice are still turn-based and rely on an external Voice Activity Detection (VAD) module to mark the end of the user's turn, which fundamentally limits their interactive ability. In this paper, we introduce BayLing-Duplex, a native full-duplex SpeechLM where a single autoregressive LLM decides when to listen, when to speak, and when to stop, with no auxiliary turn-taking module. The design adds only a few special tokens to the standard vocabulary, so it transfers across LLMs and reuses existing training and serving stacks with no architectural adaptation. Starting from the public GLM-4-Voice checkpoint and using only 400K full-duplex samples for fine-tuning followed by a lightweight DPO stage, BayLing-Duplex reaches 92% turn-taking success and 100% interruption success on InstructS2S-Eval, while improving the speech-response score from 2.17 to 3.39 over Moshi. BayLing-Duplex also matches or surpasses its turn-based counterpart on Llama Questions, Web Questions, and Alpaca-Eval, showing that simultaneous listen-and-speak modeling does not sacrifice response quality.

Momentum

Related topics

Latest papers 93