Full-Duplex Spoken Dialogue Systems

Full-duplex spoken dialogue systems study how voice agents can listen and speak at the same time while deciding when to continue, yield, interrupt, or respond with a backchannel. Approaches include end-to-end streaming speech models, cascaded ASR–LLM–TTS pipelines, acoustic and semantic turn detectors, and reinforcement learning that separates timing choices from response content. Benchmarks test latency, turn-taking, interruption recovery, response quality, and increasingly tool use, grounding, multilingual interaction, and multi-party conversation. A central challenge is balancing fast, natural interaction with reliable understanding: overlap and interruptions can disrupt context, while stronger reasoning, safety, and grounding must work without making responses too slow or overly eager.