Overview

Spoken dialogue systems study how machines can understand speech and take part in natural, useful conversations. Approaches range from cascaded automatic speech recognition, language models, and speech synthesis to end-to-end speech-language models, including full-duplex systems that listen and speak at once; many also use dialogue-state tracking, tool use, and explicit turn-taking policies. Benchmarks and user studies assess task success, latency, interruptions, conversational naturalness, and robustness to accents, noise, and multiple speakers. A central challenge is balancing fast, well-timed interaction with accurate reasoning, coherent context use, and emotionally appropriate speech, especially when deciding when to speak, wait, or recover from an interruption.

Latest papers

CardsList

147 papers10 papers in the last four weeks, down 64% on the four weeks before. 0.1% of all new papers.

  1. Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling

    Nov 18, 2026Yifang Pan, Chris Landreth, Karan SinghTurn-Taking PredictionSpoken Dialogue Systems

  2. Latency, Speech Recognition and Turn-Taking in Voice Agents for Spanish Phone Negotiation: An Externally Measured Comparison of Pipecat, LiveKit and GPT-Live

    Nov 14, 2026Jovan Popoca Moran, Eduardo de Avila-Armenta, Raúl Alejandro Aquino-Ibarra +2Voice Agent EvaluationSpoken Dialogue Systems

  3. Real-Time Short-Response Selection for Spoken Dialogue Systems

    Nov 6, 2026Hien Ohnaka, Kazuyo Onishi, Koichiro YoshinoSpoken Dialogue Systems

  4. LibriSpeech-Real-QA: A Benchmark for Streaming Question Answering

    Nov 4, 2026Zewen Zhan, Hui Lu, Xueyuan Chen +6Speech Language ModelsAudio QA

  5. Do Multi-Turn Conversational Capabilities Carry Over from Text to Speech? A Parallel Multilingual Benchmark

    Nov 3, 2026Jiaming Zhou, Pei Zhang, Shiwan Zhao +3Audio-Language Model EvaluationSpeech Language Models

  6. From Voiceprints to User Entities: Online Speaker-Memory Construction for Multi-User Spoken Agents

    Oct 27, 2026Yucong Zhang, Cancan Li, Bo Xiao +2Conversational MemoryMemory-Augmented Language Model Agents

  7. Synthetic Child-Adult Interactions Are Not Developmentally Plausible

    Oct 26, 2026Bastian Bunzeck, Francesca PadovaniLanguage ModelingSynthetic Data Evaluation

  8. Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models

    Oct 26, 2026Junho Kim, CheHyun Lee, Donghun Kang +4Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

  9. Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems

    Oct 26, 2026Yizhou Peng, Chen Chen, Donghang Wu +2Spoken Dialogue SystemsSpoken Language Understanding

  10. Speech LLMs That Breathe: Modeling Non-Verbal Vocalizations in Dialogue

    Oct 19, 2026Heyang Liu, Lixin Zhang, Jiayi Huang +6Nonverbal Vocalization GenerationSpeech Language Models

  11. ASR-Style Transcript Normalization Removes Useful Pragmatic Signal for Dialogue Act Classification

    Oct 16, 2026Lucky Onyekwelu-Udoka, Md Shafiqul IslamSpoken Language UnderstandingSpoken Dialogue Systems

  12. CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models

    Oct 15, 2026Yingzhuo Deng, Chong Li, Bingxin Liu +6Spoken Dialogue SystemsSpeech Language Models

  13. A Benchmark for Bangla Task-Oriented Dialogue Understanding

    Oct 13, 2026Afrina Alam Lamia, Sumiaya Sharif, Prottoy Modak +2State TrackingTask-Oriented Dialogue Systems

  14. DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?

    Oct 13, 2026Jiaheng Dong, Shihong Tan, Gongping Huang +3Spoken Dialogue SystemsSpeech Language Models

  15. Towards an Extensible Benchmark for Spoken Dialogue with Social Robots

    Oct 6, 2026Casey Kennington, Ross Mead, Saad Elbeleidy +1Spoken Dialogue SystemsMulti-Turn Dialogue Evaluation

  16. Jarvis: A Proactive Speech Agent for Multi-Party Conversations

    Oct 5, 2026Seunghyun Oh, Hirotaka Hiraki, Shuyue Stella Li +2Conversational AgentsProactive Agents

  17. VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents

    Oct 3, 2026Yate Ge, Run Yuan, Yueran Qi +7Human-AI InteractionHuman-Computer Interaction

  18. XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions

    Oct 3, 2026Zhanxun Liu, Yifan Duan, Hengtao Wu +7Turn-Taking PredictionSpoken Dialogue Systems

  19. A study on human-agent teaming through spoken interaction: the impact of human individual traits and agent characteristics

    Oct 2, 2026Lara Gauder, Martin Bernardo Meza, Javier Krick +4Human-AI CollaborationTrust in AI

  20. TPBench: A Turning-Point Benchmark for Dialogue Compression

    Oct 2, 2026Minji Park, Seunghyun Yoon, Hyuk LimMulti-Turn Dialogue EvaluationSpoken Dialogue Systems

  21. Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs

    Sep 29, 2026Runqiu Xu, Zhisheng Zheng, David HarwathSpoken Dialogue Systems