Spoken Dialogue Systems

Ask about this topic

Overview

Spoken dialogue systems study how machines can understand speech and take part in natural, useful conversations. Approaches range from cascaded automatic speech recognition, language models, and speech synthesis to end-to-end speech-language models, including full-duplex systems that listen and speak at once; many also use dialogue-state tracking, tool use, and explicit turn-taking policies. Benchmarks and user studies assess task success, latency, interruptions, conversational naturalness, and robustness to accents, noise, and multiple speakers. A central challenge is balancing fast, well-timed interaction with accurate reasoning, coherent context use, and emotionally appropriate speech, especially when deciding when to speak, wait, or recover from an interruption.

Latest papers

CardsList

141 papers19 papers in the last four weeks, down 39% on the four weeks before. 0.1% of all new papers.

  1. From Voiceprints to User Entities: Online Speaker-Memory Construction for Multi-User Spoken Agents

    Oct 27, 2026Yucong Zhang, Cancan Li, Bo Xiao +2Conversational MemoryMemory-Augmented Language Model Agents

    School of Computer Science, Wuhan University · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen · Meituan

  2. Integrate the Reward, Don't Sample It: Window Likelihood for Interruptions and Backchannels in Full-Duplex Speech Models

    Oct 26, 2026Guoqiang Hu, Lichao Zhang, Yizhou Peng +5Full-Duplex Spoken Dialogue SystemsSpoken Dialogue Systems

    Nanyang Technological University, Singapore · Tencent

  3. Synthetic Child-Adult Interactions Are Not Developmentally Plausible

    Oct 26, 2026Bastian Bunzeck, Francesca PadovaniLanguage ModelingSynthetic Data Evaluation

    Computational Linguistics, Department of Linguistics, Bielefeld University · Center for Language and Cognition (CLCG), University of Groningen

  4. Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models

    Oct 26, 2026Junho Kim, CheHyun Lee, Donghun Kang +4Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

    IPAI, Seoul National University (SNU) · Department of ECE, SNU · Department of AI, University of Seoul +1

  5. Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems

    Oct 26, 2026Yizhou Peng, Chen Chen, Donghang Wu +2Spoken Dialogue SystemsSpoken Language Understanding

    Nanyang Technological University, Singapore · AI Singapore, Singapore

  6. Speech LLMs That Breathe: Modeling Non-Verbal Vocalizations in Dialogue

    Oct 19, 2026Heyang Liu, Lixin Zhang, Jiayi Huang +6Nonverbal Vocalization GenerationSpeech Language Models

    Shanghai Jiao Tong University · Ant Group

  7. Quality-Controlled Reply and Summary Data for Post-Training Small Language Models in AI Call-Screening Agents

    Oct 18, 2026Jiaqi Gan, Tsang Ng, Ankit Raj +5Small Language ModelsSpoken Dialogue Systems

  8. ASR-Style Transcript Normalization Removes Useful Pragmatic Signal for Dialogue Act Classification

    Oct 16, 2026Lucky Onyekwelu-Udoka, Md Shafiqul IslamSpoken Language UnderstandingSpoken Dialogue Systems

    Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa, United States

  9. CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models

    Oct 15, 2026Yingzhuo Deng, Chong Li, Bingxin Liu +6Spoken Dialogue SystemsSpeech Language Models

    Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Wuhan AI Research

  10. A Benchmark for Bangla Task-Oriented Dialogue Understanding

    Oct 13, 2026Afrina Alam Lamia, Sumiaya Sharif, Prottoy Modak +2State TrackingTask-Oriented Dialogue Systems

    East West University, Dhaka, Bangladesh

  11. DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?

    Oct 13, 2026Jiaheng Dong, Shihong Tan, Gongping Huang +3Spoken Dialogue SystemsSpeech Language Models

    The University of Melbourne · University of Birmingham · Wuhan University +1

  12. People Making Conversation: Evaluating How Naturally Speech Models Converse

    Oct 8, 2026Arya Bastani, Iman ModarressiSpeech Generation EvaluationSpoken Dialogue Systems

    People Make Things

  13. EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams

    Oct 8, 2026Heeseung KimSpoken Dialogue Systems

    Department of AI, University of Seoul

  14. Towards an Extensible Benchmark for Spoken Dialogue with Social Robots

    Oct 6, 2026Casey Kennington, Ross Mead, Saad Elbeleidy +1Spoken Dialogue SystemsMulti-Turn Dialogue Evaluation

    Department of Computer Science Boise State University Boise, Idaho, U.S.A. · Semio Los Angeles, California, U.S.A. · Peerbots Arlington, Virginia, U.S.A. +1

  15. Jarvis: A Proactive Speech Agent for Multi-Party Conversations

    Oct 5, 2026Seunghyun Oh, Hirotaka Hiraki, Shuyue Stella Li +2Conversational AgentsProactive Agents

    Paul G. Allen School of Computer Science and Engineering, University of Washington, WA, USA · Graduate School of Interdisciplinary Information Studies, The University of Tokyo, Bunkyo, Tokyo, Japan

  16. VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents

    Oct 3, 2026Yate Ge, Run Yuan, Yueran Qi +7Human-AI InteractionHuman-Computer Interaction

    Tongji University Shanghai, China · Grinnell College Grinnell, Iowa, United States · Hunan University Changsha, Hunan, China +1

  17. XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions

    Oct 3, 2026Zhanxun Liu, Yifan Duan, Hengtao Wu +7Turn-Taking PredictionSpoken Dialogue Systems

    MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Shanghai Innovation Institute · Shanghai AI Laboratory +3

  18. A study on human-agent teaming through spoken interaction: the impact of human individual traits and agent characteristics

    Oct 2, 2026Lara Gauder, Martin Bernardo Meza, Javier Krick +4Human-AI CollaborationTrust in AI

    Departamento de Computación, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Pabellón Cero+Infinito, Ciudad Universitaria, Buenos Aires, C1428EGA, Argentina · Instituto de Investigación en Ciencias de la Computación, UBA-CONICET, Pabellón Cero+Infinito, Ciudad Universitaria, Buenos Aires, C1428EGA, Argentina · Universidad Nacional de Córdoba, Pabellón Argentina, Av. Haya de la Torre s/n, Ciudad Universitaria, Córdoba, X5000HUA, Argentina +1

  19. TPBench: A Turning-Point Benchmark for Dialogue Compression

    Oct 2, 2026Minji Park, Seunghyun Yoon, Hyuk LimMulti-Turn Dialogue EvaluationSpoken Dialogue Systems

    Korea Institute of Energy Technology (KENTECH)

  20. Mixture of Decoders for Diverse Dialog Response Generation

    Sep 30, 2026Wenchao DuSpoken Dialogue SystemsDiverse Text Generation

    Microsoft Corporation Redmond, WA 98052

  21. Richard: Voice-First Mobile Interaction for Persistent Tasks

    Sep 30, 2026Xinyang ChenSpoken Dialogue SystemsConversational Agents

  22. Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs

    Sep 29, 2026Runqiu Xu, Zhisheng Zheng, David HarwathSpoken Dialogue Systems

    The University of Texas at Austin, Austin, TX, USA

  23. MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation

    Sep 29, 2026Ke Wang, Houxing Ren, Zimu Lu +4Spoken Dialogue SystemsMulti-Turn Dialogue Evaluation

    CUHK MMLab · CPII under InnoHK

  24. Almost Human, Except When It Matters: VoxParity and the Decisions a Voice Should Change

    Sep 28, 2026Bhavik ManglaSpoken Dialogue SystemsAudio Reasoning

    Independent Research

  25. Lost with a Map: Conversational State and Behavioral Reliability in Language Models

    Sep 27, 2026Atahan Dokme, Larry HeckSpoken Dialogue SystemsLLM Reliability

    Georgia Institute of Technology

  26. Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends

    Sep 27, 2026Seonghyeon Go, Yongwoo Kim, Hyeonjin Cha +1Knowledge Augmentation for Language ModelsSpoken Dialogue Systems

    Mindlogic

  27. RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue

    Sep 25, 2026Sathvik Udupa, Naveen Kumar, Ryan FolmsbeeSpoken Dialogue SystemsMulti-Turn Dialogue Management

    Disney Research Imagineering, Los Angeles, USA

  28. Learning Natural Conversational Behavior in Tandem Speech-to-Speech Models with Randomized Guidance

    Sep 25, 2026Manato Yaguchi, Yotaro Kubo, Hikaru Asano +1Spoken Dialogue SystemsSpeech Language Models

    Sakana AI, Tokyo, Japan · The University of Tokyo, Tokyo, Japan

  29. Controlling Backchannels in Streamable Full-duplex Models

    Sep 24, 2026Maike Züfle, Peter Polák, Sefik Emre Eskimez +3Spoken Dialogue SystemsSpeech Language Models

    Karlsruhe Institute of Technology, Germany · AppTek, Germany · Charles University, Czech Republic +2

  30. AdaptDuplex: From Static to Adaptive Full-Duplex Spoken Dialogue

    Sep 24, 2026Zhiyang Zhou, Yingxin Shang, Zhou Wang +9Spoken Dialogue SystemsSpeech Language Models

    Zuoyebang Education Technology (Beijing) Co., Ltd.