Turn-Taking Prediction

Follow this topic Ask about this topic

Turn-taking prediction studies when a speaker is likely to finish or continue, who may speak next, and whether an entry is a floor change, backchannel, or interruption. Systems use acoustic and prosodic cues such as pauses and intonation, often with voice activity projection, while other approaches add words, gaze, facial motion, speaker roles, or dialogue context. Current work combines streaming speech recognition with turn-state prediction and trains models using supervised, self-supervised, or synthetic conversational data. A central challenge is acting at the right time across noisy, overlapping, and multi-party conversations: early responses risk interruptions, while cautious systems may miss opportunities, and benchmarks do not always capture human judgments of appropriate timing.

Latest papers

37 papers5 papers in the last four weeks, down 55% on the four weeks before. 0.0% of all new papers.

VoxTurn: Answering Several Turn-Taking Questions in One Pass with One Model
Oct 15, 2026Yang XiaoTurn-Taking Prediction Speech Language Models
The University of Melbourne
MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue
Oct 14, 2026Puneet Mathur, Dinesh ManochaTurn-Taking in Dialogue Full-Duplex Spoken Dialogue Systems
University of Maryland College Park, USA
HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models
Oct 6, 2026Kyudan Jung, Hyunsin Park, Yoonhyung Lee +5RL for Language Models Speech Language Models
Qualcomm AI Research · KAIST AI
XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions
Oct 3, 2026Zhanxun Liu, Yifan Duan, Hengtao Wu +7Turn-Taking Prediction Spoken Dialogue Systems
MoE Key Lab of Artiﬁcial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Shanghai Innovation Institute · Shanghai AI Laboratory +3
When Does a Spoken Agent Have Enough Evidence to Act? The PACT-SLM Contract Test
Sep 28, 2026Mengzhe GengVoice Agent Evaluation Speech Language Models
National Research Council Canada
BanglaTurn: A Benchmark and Whisper-Based Model for End-of-Turn Detection in Bangla Speech
Sep 24, 2026Mizbaul Haque MarufTurn-Taking Prediction
Vivasoft Limited, Dhaka, Bangladesh
Multimodal Voice Activity Projection for Social Robot Mediation: Expected Behavior and Deployment Constraints
Sep 23, 2026Antonio Cano, Guillermo Perez, Luis Merino +1Turn-Taking Prediction Human-Robot Interaction
4i Intelligent Insights, Seville, Spain · Universidad de Sevilla, Seville, Spain · Universidad Pablo de Olavide, Seville, Spain +1
Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models
Sep 23, 2026Kian Shamsaie, Iman ModarressiMulti-Turn Dialogue Evaluation Turn-Taking Prediction
People Make Things
Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts
Sep 15, 2026Shuofeng Zhao, Hongwei Cai, Wenke Fan +9Spoken Dialogue Systems Multi-Turn Dialogue Evaluation
Zuoyebang Education Technology, Beijing, China
Audio-Visual Turn-taking Prediction in Cocktail Party Scenarios
Sep 15, 2026Long-Vu Hoang, Naomi HarteDomain Adaptation Turn-Taking Prediction
Sigmedia Group, School of Engineering, Trinity College Dublin, Ireland
Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projection
Sep 13, 2026Willem Berner, Julio Cesar Cavalcanti, Kalle Åström +1Turn-Taking Prediction Audio-Visual Understanding
Lund University · KTH Royal Institute of Technology
Less can be More: What Aspects of Speech Drive End-of-Turn Detection
Sep 10, 2026Rini Sharon, Manickavela A, Kadri Hacioglu +1Speech Prosody Turn-Taking Prediction
Uniphore, India
Using Semantic Uncertainty to Estimate Transition Relevance in Turn-taking
Sep 10, 2026Muhammad Umair, Jan P. de RuiterSpoken Dialogue Systems Turn-Taking Prediction
Department of Computer Science Tufts University Medford, Massachusetts, USA
TamilEOT: A Dataset and Model for Semantic End-of-Turn Detection in Tamil Telephone Speech
Sep 4, 2026Santhoshkumar VTurn-Taking Prediction Conversational Agents
Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting
Sep 3, 2026Muneeb Khan, Frederic Kirstein, Terry Ruas +1LLM Agent Evaluation Conversational Agents
University of Göttingen, Germany
Decoupling Turn-Taking from Semantics: A Decoupled Data Approach for Finite-State-Machine-Based Full-Duplex Dialogue
Sep 3, 2026Yihang Li, Chenhui ChuSpoken Dialogue Systems Speech Language Models
Kyoto University
Predicting Turn-Taking Outcomes in Multi-Party Conversation: Interpretable Modelling of Speech and Gaze Dynamics with Interpersonal Closeness
Aug 28, 2026Mark Dourado, Karim Haddad, Henrik G. Hassager +1Turn-Taking Prediction Turn-Taking in Dialogue
Research & Exploration, GN Store Nord, Ballerup, Denmark · Department of Architecture, Design and Media Technology, Aalborg University, Copenhagen, Denmark · Department of Engineering Technology and Didactics, Technical University of Denmark, Kongens Lyngby, Denmark
TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue
Aug 25, 2026Freeman Jiang, Ramon Sanabria, Soham Deshmukh +16Spoken Dialogue Systems Multi-Turn Dialogue Evaluation
Sesame AI · Mundo AI · Oto +4
Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction
Aug 22, 2026Ahmet Tuğrul Bayrak, Fatma Nur Korkmaz, Bekir Berker Türker +2Turn-Taking Prediction
Data Science and Innovation Ata Technology Platforms Istanbul, Turkey
X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Prediction
Aug 11, 2026Kaiqi Fu, Rime Wen, Altman Lin +4Streaming ASR Multi-Task Learning
X Square Robot
Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
Jul 30, 2026Xilin Jiang, Riki Shimizu, Sukru Samet Dindar +3Spoken Dialogue Systems Speech Language Models
Columbia University · University of Washington · University of Illinois Urbana-Champaign
On the Structure of Address in Multi-Party Dialogue: From Discrete Labels to Continuous Levels
Jul 17, 2026Taiga Mori, Koji Inoue, Divesh Lala +1Turn-Taking Prediction
Graduate School of Informatics, Kyoto University, Japan
Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders
Jul 8, 2026Antonio Cano, Guillermo Pérez, Luis Merino +1Conversational Agents Human-Robot Interaction
4i Intelligent Insights, Seville, Spain · Universidad de Sevilla, Seville, Spain · Universidad Pablo de Olavide, Seville, Spain +1
Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings
Jun 16, 2026Ryo Fukuda, Takatomo Kano, Siddhant Arora +7LLM Evaluation Multimodal Large Language Models
NTT, Inc., Japan · Language Technologies Institute, Carnegie Mellon University, USA
MuVAP: Multimodal Multiparty Voice Activity Projection for Turn-taking Prediction in the Wild
Jun 15, 2026Haotian Qi, Gabriel SkantzeMultimodal Grounding Audio-Visual Understanding
Department of Speech Music and Hearing, KTH Stockholm, Sweden
Fast When, Careful Who: Dual-Process Multiparty Turn-Taking with Diffusion Augmentation
Jun 15, 2026Rutherford A. Patamia, Ming Liu, Wei Luo +2Spoken Dialogue Systems Turn-Taking Prediction
Deakin University, Melbourne, Australia · Griffith University, Brisbane, Australia
Adaptive Turn-Taking for Real-time Multi-Party Voice Agents
Jun 11, 2026Soumyajit Mitra, Prabhat Pandey, Abhinav Jain +2Spoken Dialogue Systems Speech Language Models
Amazon AGI · IIT Kharagpur, India
Toward Signing Activity Projection in Sign Language Interaction
Jun 8, 2026Takao Obi, Wang Yusong, Koji Inoue +1Human-AI Interaction Turn-Taking Prediction
Institute of Science Tokyo, Yokohama 230-0045, Japan · Kyoto University, Yoshida-honmachi, Sakyoku, Kyoto, Japan
Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Models
May 19, 2026Pablo Riera, Pablo Brusco, Cristina Kuo +2Speech Language Models Turn-Taking Prediction
ASAPP Inc., USA
When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
May 7, 2026Vihaan Nama, Shreya Mendi, Zian Ye +1Synthetic Data Generation Synthetic Data Generation for Language Models
Pratt School of Engineering Duke University

Oct 15, 2026J/K move · Enter open · S save

VoxTurn: Answering Several Turn-Taking Questions in One Pass with One Model

Yang Xiao

The University of Melbourne

Turn-Taking Prediction Speech Language Models Voice Agent Evaluation

Open paper AskPDF arXiv

A cascaded voice agent must decide, throughout each silence of its user, whether the user has finished their turn: answering too early cuts the user off, answering too late makes the agent slow. It often needs related decisions too: will the user soon go on, is speech during its own turn an interruption or a backchannel, is a sound speech at all. With open detectors, each extra decision costs another model or another call, and on the 14-language eot-bench they cut users off in more than a quarter of mid-turn pauses. Because they are trained on segmented utterances, this mismatch is the largest single source of error. We present VoxTurn, a 0.8B speech LLM that casts these decisions as multiple-choice questions, scores the hidden state of each option, and answers any set of them in one forward pass with a block-isolating attention mask, exactly as if asked one at a time. Trained on real conversational pauses with labels derived from their timing, and re-queried while a silence lasts, VoxTurn reaches 11.5% false cut-offs in English at a 300 ms latency budget, comparable to two commercial detectors, and a 14-language mean of 19.9%, the lowest among open models, at 8.3 ms per decision. In Pipecat and LiveKit it cuts off 17% and 9% of turns, and it ranks fifth of 24 in TurnBench end-of-turn detection without any TurnBench training data. The model, code and framework plugins are released.