Spoken Dialogue Systems

Ask about this topic

Overview

Spoken dialogue systems study how machines can understand speech and take part in natural, useful conversations. Approaches range from cascaded automatic speech recognition, language models, and speech synthesis to end-to-end speech-language models, including full-duplex systems that listen and speak at once; many also use dialogue-state tracking, tool use, and explicit turn-taking policies. Benchmarks and user studies assess task success, latency, interruptions, conversational naturalness, and robustness to accents, noise, and multiple speakers. A central challenge is balancing fast, well-timed interaction with accurate reasoning, coherent context use, and emotionally appropriate speech, especially when deciding when to speak, wait, or recover from an interruption.

Latest papers

CardsList

141 papers19 papers in the last four weeks, down 39% on the four weeks before. 0.1% of all new papers.

  1. ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models

    Apr 11, 2026Chi-Yuan Hsiao, Ke-Han Lu, Yu-Kuan Fu +3Speech Language ModelsFull-Duplex Spoken Dialogue Systems

    National Taiwan University · ASUS Open Cloud Infrastructure Software Center · NVIDIA AI Technology Center

  2. EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptions

    Apr 8, 2026Smit Nautambhai Modi, Gandharv Mahajan, Marc Wetter +1State TrackingVoice Agent Evaluation

    Applied Machine Learning Research

  3. Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

    Apr 6, 2026Guan-Ting Lin, Chen Chen, Zhehuai Chen +1Voice Agent EvaluationFull-Duplex Spoken Dialogue Systems

    National Taiwan University · NVIDIA

  4. PSY-STEP: Structuring Therapeutic Targets and Action Sequences for Proactive Counseling Dialogue Systems

    Apr 6, 2026Jihyun Lee, Yejin Min, Yejin Jeon +3Conversational AgentsCognitive Behavioral Therapy

    Graduate School of Artificial Intelligence, POSTECH · MILA · McGill University +1

  5. FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection

    Apr 2, 2026Chengyou Wang, Hongfei Xue, Mingchen Shao +8Spoken Dialogue SystemsAudio Understanding

    Shengwang

  6. CoDeTT: A Context-Aware Decision Benchmark for Turn-Taking Evaluation

    Mar 26, 2026Huan Shen, Yingao Wang, Shangkun Huang +2Turn-Taking PredictionSpoken Dialogue Systems

    BRVoice Team, Bairong, Inc., China

  7. Semantic-Aware Interruption Detection in Spoken Dialogue Systems: Benchmark, Metric, and Model

    Mar 25, 2026Kangxiang Xia, Bingshen Mu, Xian Shi +2Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

    Qwen Team, Alibaba · Independent Researcher

  8. RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue

    Mar 24, 2026Long Mai, Junli LiangSpoken Dialogue SystemsSpeculative Decoding

    Trinity College Dublin, Ireland · University College Dublin, Ireland

  9. TiCo: Time-Controllable Spoken Dialogue Model

    Mar 23, 2026Kai-Wei Chang, Wei-Chih Chen, En-Pei Hu +2Spoken Dialogue SystemsInstruction Following

    MIT · NTU · NTU AI-CoRE

  10. MOSS-TTSD: Text to Spoken Dialogue Generation

    Mar 20, 2026Yuqian Zhang, Donghua Yu, Zhengyuan Lin +15Multi-Speaker TTSLong-Form Speech Recognition

  11. From Seeing it to Experiencing it: Interactive Evaluation of Intersectional Voice Bias in Human-AI Speech Interaction

    Mar 19, 2026Shree Harsha Bokkahalli Satish, Maria Teleki, Christoph Minixhofer +3Intersectional Bias in Language ModelsLanguage Model Bias Evaluation

    KTH Royal Institute of Technology Stockholm, Sweden · Texas A&M University College Station, USA · University of Edinburgh Edinburgh, UK

  12. Proactive Knowledge Inquiry in Doctor-Patient Dialogue: Stateful Extraction, Belief Updating, and Path-Aware Action Planning

    Mar 18, 2026Zhenhai Pan, Yan Liu, Jia YouState TrackingElectronic Health Records

    The Hong Kong Polytechnic University, Hong Kong

  13. SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models

    Mar 17, 2026Tianyu Xie, Jinfa Huang, Yuexiao Ma +11Audio-Visual UnderstandingSpoken Dialogue Systems

    Xiamen University · University of Rochester · Nanyang Technological University +1

  14. SpokenUS: A Spoken User Simulator for Task-Oriented Dialogue

    Mar 17, 2026Jonggeun Lee, Junseong Pyo, Jeongmin Park +1Spoken Dialogue SystemsConversational Agents

    Graduate School of Data Science, Seoul National University · Department of Information Systems, Hanyang University · Department of Computer Science and Engineering, Seoul National University

  15. SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness

    Mar 16, 2026Jingyu Lu, Yuhan Wang, Fan Zhuo +7Reward ModelingSpoken Dialogue Systems

    Zhejiang University

  16. Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue

    Mar 12, 2026Kratika Bhagtani, Mrinal Anand, Yu Chen Xu +1Turn-Taking in DialogueTurn-Taking Prediction

    School of Electrical and Computer Engineering, Purdue University, United States · Ishiki Labs Inc.

  17. DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization

    Mar 10, 2026Jianing Yang, Yusuke Fujita, Yui SudoSpoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

    SB Intuitions Corp., Japan · The University of Tokyo, Japan

  18. Designing Explainable Conversational Agentic Systems for Guaraní Speakers

    Mar 5, 2026Samantha Adorno, Akshata Kishore Moharir, Ratna KandalaSpoken Dialogue SystemsHuman-Centered AI

    University of Kansas, Lawrence, Kansas, USA · Independent Researcher, Sherwood, Oregon, USA

  19. Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction

    Feb 27, 2026Xiang Li, Jiabao Gao, Sipei Lin +5Voice Agent EvaluationSpoken Dialogue Systems

    State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications · Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong, Shenzhen +2

  20. Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

    Feb 26, 2026Siyuan Liu, Jiahui Xu, Feng Jiang +6Spoken Dialogue SystemsFull-Duplex Spoken Dialogue Systems

    School of Computer Science and Technology, Tongji University · School of Artificial Intelligence, Chinese University of Hong Kong, Shenzhen · Department of Language Science and Technology, Hong Kong Polytechnic University +2

  21. Hello-Chat: Towards Realistic Social Audio Interactions

    Feb 16, 2026Yueran Hou, Peilei Jia, Zihan Sun +5Expressive Speech SynthesisAudio-Language Models