Full-Duplex Spoken Dialogue Systems

Overview

Full-duplex spoken dialogue systems study how voice agents can listen and speak at the same time while deciding when to continue, yield, interrupt, or respond with a backchannel. Approaches include end-to-end streaming speech models, cascaded ASR–LLM–TTS pipelines, acoustic and semantic turn detectors, and reinforcement learning that separates timing choices from response content. Benchmarks test latency, turn-taking, interruption recovery, response quality, and increasingly tool use, grounding, multilingual interaction, and multi-party conversation. A central challenge is balancing fast, natural interaction with reliable understanding: overlap and interruptions can disrupt context, while stronger reasoning, safety, and grounding must work without making responses too slow or overly eager.

Latest papers

CardsList

129 papers18 papers in the last four weeks, down 55% on the four weeks before. 0.1% of all new papers.

  1. The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning

    Mar 18, 2026Donghang Wu, Tianyu Zhang, Yuxin Li +4Speech Language ModelsContinuous Latent Reasoning

  2. SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models

    Mar 17, 2026Tianyu Xie, Jinfa Huang, Yuexiao Ma +11Audio-Visual UnderstandingSpoken Dialogue Systems

  3. ττ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains

    Mar 14, 2026Soham Ray, Keshav Dhandhania, Victor Barres +1Voice Agent EvaluationAI Agent Benchmarks

  4. Privacy-Preserving End-to-End Full-Duplex Speech Dialogue Models

    Mar 9, 2026Nikita Kuzmin, Tao Zhong, Jiajun Deng +6Privacy-Preserving Speech ProcessingSpeaker Anonymization

  5. Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

    Feb 26, 2026Siyuan Liu, Jiahui Xu, Feng Jiang +6Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

  6. Phoenix-VAD: Streaming Semantic Endpoint Detection for Full-Duplex Speech Interaction

    Sep 24, 2025Weijie Wu, Wenhao Guan, Kaidi Wang +6Full-Duplex Spoken Dialogue Systems