Full-Duplex Spoken Dialogue Systems

Overview

Full-duplex spoken dialogue systems study how voice agents can listen and speak at the same time while deciding when to continue, yield, interrupt, or respond with a backchannel. Approaches include end-to-end streaming speech models, cascaded ASR–LLM–TTS pipelines, acoustic and semantic turn detectors, and reinforcement learning that separates timing choices from response content. Benchmarks test latency, turn-taking, interruption recovery, response quality, and increasingly tool use, grounding, multilingual interaction, and multi-party conversation. A central challenge is balancing fast, natural interaction with reliable understanding: overlap and interruptions can disrupt context, while stronger reasoning, safety, and grounding must work without making responses too slow or overly eager.

Latest papers

CardsList

129 papers18 papers in the last four weeks, down 55% on the four weeks before. 0.1% of all new papers.

  1. Robots That Talk Too Much and Don't Listen Enough

    Nov 12, 2026Graham Wilcock, Kristiina JokinenHuman-Robot InteractionSocial Robotics

  2. Omni-JEV: Text and Dual-Stream Speech Decisions

    Nov 10, 2026Shengzhou KongFull-Duplex Spoken Dialogue SystemsMulti-Turn Dialogue Management

  3. Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models

    Nov 4, 2026Yutong Bian, Yanglin Zhang, Yuxiang Wang +6Speech Language ModelsSpeculative Decoding

  4. ReFrame: Silence-Aware Compute Reclamation for Full-Duplex Speech Models

    Nov 1, 2026Zheng Zhou, Guanyi Chen, Jing Wei +3Full-Duplex Spoken Dialogue SystemsSpeech Language Models

  5. Local or Escalate? Post-Training Expert Routing for Real-Time Duplex Speech Models

    Nov 1, 2026Ruoyu He, Jisen Li, Anthony Ji +5LLM RoutingFull-Duplex Spoken Dialogue Systems

  6. Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models

    Oct 26, 2026Junho Kim, CheHyun Lee, Donghun Kang +4Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

  7. CounselPlex: A Stressor-Aware Full-Duplex Counseling Model

    Oct 23, 2026Yeonju Kim, Bella Godiva, Yong Man RoConversational AgentsFull-Duplex Spoken Dialogue Systems

  8. Why Full-Duplex Speech Models Collapse in Long Conversations

    Oct 23, 2026Yeonju Kim, Bella Godiva, Se Jin Park +1Full-Duplex Spoken Dialogue SystemsLong-Context Modeling

  9. DIVE: Benchmarking Full-Duplex Interaction with Visual Engagement in Continuous Streams

    Oct 20, 2026Yuhang Chen, Guangyue Liu, Shilong LiuMultimodal Model EvaluationMultimodal Large Language Models

  10. DuplexLedger: Benchmarking Transactional Memory in Full-Duplex Speech Agents

    Oct 20, 2026Yuxuan Li, Xiangchen Guan, Kunrong Li +7Memory Management for LLM AgentsMemory Agent Benchmarks

  11. Beyond Two Tracks: Speaker-Anchored Full-Duplex Dialogue Recovery

    Oct 19, 2026Yuxuan Chen, Yang Xiao, Jack Sun +3Full-Duplex Spoken Dialogue SystemsSpeaker Diarization

  12. Jarvis: Coordination Deserves a Brain of Its Own in Agentic Interactive Models

    Oct 16, 2026Yuxiang Wang, Shengbo Cai, Zhixian Zhao +7LLM Agent HarnessesMulti-Agent Coordination

  13. Full-Duplex Interaction with Speech, Expressions and Gestures

    Oct 15, 2026Siyin Wang, Wenyi Yu, Terumi Chiba +5Facial Expression SynthesisMultimodal Generation

  14. Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models

    Oct 14, 2026Yuxuan Chen, Yang Xiao, Jack Sun +3Speech Language ModelsFull-Duplex Spoken Dialogue Systems

  15. DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?

    Oct 13, 2026Jiaheng Dong, Shihong Tan, Gongping Huang +3Spoken Dialogue SystemsSpeech Language Models

  16. D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones

    Oct 12, 2026Cheng-Kuang Chang, Alexander H. Liu, Brian Kingsbury +5Speech Language ModelsFull-Duplex Spoken Dialogue Systems

  17. RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can

    Oct 9, 2026Yi-Jen Shih, Puyuan Peng, Haohe Liu +2Full-Duplex Spoken Dialogue SystemsPositional Encoding

  18. DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion

    Oct 8, 2026Heeseung KimFull-Duplex Spoken Dialogue SystemsDiffusion Model Inference Acceleration

  19. SteerablePlex: Can We Steer Full-Duplex Models?

    Oct 8, 2026Haolong Zheng, Maike Züfle, Dominik Macháček +6Full-Duplex Spoken Dialogue SystemsLanguage Model Steering

  20. DuplexAgent-RSI: Recursive Harness Improvement for Full-Duplex Voice Agent Collaboration

    Oct 8, 2026Yingda Shen, Yuxiang Wang, Kunyu Feng +7LLM Agent HarnessesAgent Harness Optimization