Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While existing research has explored multimodal approaches and large language models for turn-ending prediction, there is a lack of naturalistic conversational corpora specifically addressing turn-taking dynamics in Turkish. This study introduces a multimodal Turkish conversational dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels that allow overlapping speech to be attributed to individual speakers, and time-aligned transcriptions. Turn-taking prediction is formulated as a binary classification problem, and a Genetic Algorithm (GA) is employed to optimize interpretable decision rules derived from visual, acoustic, and linguistic features. A hybrid AND-OR rule representation is adopted in the proposed framework to represent the alternative cue combinations that precede a turn transition.
Figures & tables
Speaker Set
Dur. (min)
Speech (min)
Non-sp. (min)
Non-sp. (%)
# Words
user_01 & user_02
26.5
22.7
3.7
14.1
3,267
user_03 & user_04
16.0
13.7
2.3
14.4
2,236
user_01 & user_05
16.4
13.6
2.8
17.2
2,219
user_01 & user_04
24.4
20.9
3.5
14.5
3,290
user_03 & user_06
17.1
14.1
3.1
17.9
2,252
user_03 & user_07
25.5
22.9
2.6
10.1
3,856
TABLE I: Statistics by Conversation
Fig. 1: Turn labelling flow.
Speaker
# Words
% Words
Speak. (min)
% Speak. time
# Conv.
user_01
9,523
26.7
66.2
28.6
5
user_02
1,044
2.9
8.2
3.5
1
user_03
12,172
34.1
72.9
31.5
6
user_04
1,937
5.4
12.8
5.5
2
user_05
1,196
3.3
7.7
3.3
1
user_06
1,376
3.9
9.8
4.2
2
TABLE II: Speaker Contribution Statistics
Feature
Unit (range)
Visual (9), MediaPipe Face Mesh + Py-Feat, 16 fps
landmark_movement
norm. coord. (0–100)
au_intensity_change
AU intensity (0–1)
blink_rate
blinks per 2 s (0–10)
gaze_changes
normalized (0–1)
mouth_opening_change
MAR (0–50)
TABLE III: Feature Details
Fig. 2: Simplified chromosome representation containing two features for visualization purposes.
Parameter
Value
Population size / max. generations
500 / 900
Selection
tournament, size 5
Crossover
single-point on conditions, rate 0.85
Mutation
rate 0.10 per offspring, 1/3 each for
threshold, feature, logic operators
Elitism
top 5% carried over unchanged
TABLE IV: Genetic Algorithm Configuration
Generations
Precision
Recall
F1
0
32.4
49.3
39.1
100
38.6
60.4
47.1
300
45.1
66.0
53.6
500
45.9
71.2
55.8
700
46.0
73.6
56.6
900
46.1
74.6
57.0
TABLE V: Evolution of Model Performance Across Generations (5-fold averages)
Managing natural dialogue timing is a significant challenge for voice-based chatbots. Most current systems usually rely on simple silence detection, which often fails because human speech patterns involve irregular pauses. This causes bots to interrupt users, breaking the conversational flow. This problem is even more severe for languages like Turkish, which lack high-quality datasets for turn-taking prediction. This paper introduces Syn-TurnTurk, a synthetic Turkish dialogue dataset generated using various Qwen Large Language Models (LLMs) to mirror real-life verbal exchanges, including overlaps and strategic silences. We evaluated the dataset using several traditional and deep learning architectures. The results show that advanced models, particularly BI-LSTM and Ensemble (LR+RF) methods, achieve high accuracy (0.839) and AUC scores (0.910). These findings demonstrate that our synthetic dataset can have a positive affect for models understand linguistic cues, allowing for more natural human-machine interaction in Turkish.
Ahmet Tuğrul Bayrak, Mustafa Sertaç Türkel, Fatma Nur Korkmaz
Data Science and Innovation, Ata Technology Platforms, İstanbul, Turkey
Speakers in natural conversation take turns speaking and listening, deciding in real time when to take, hold, or yield the floor. However, turn-taking evaluation remains limited due to the lack of a consistent, linguistically grounded evaluation protocol and hand-annotated data covering diverse conversation types. To address this, we present TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection. We set conversation type as a controllable experimental variable, covering six distinct interaction styles, and triple-annotate each conversation. Benchmarking 14 heterogeneous turn-taking systems, we find end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in backchannel-dense interaction styles. Although in smooth floor transfers human listeners begin speaking a median 151 ms before the current turn ends, no current system performs equivalently without incurring excessive false positives. We release our corpus, a 104-hour training set, and a public leaderboard with an interactive dataset viewer at https://turnbench.sesame.com.
Speech-to-speech models handle turn-taking naturally but offer limited support for tool-calling or complex reasoning, while production ASR-LLM-TTS voice pipelines offer these capabilities but rely on silence timeouts, which lead to unnatural turn-taking. We present DualTurn, which narrows this gap through generative pretraining on dual-channel conversational audio. The model generates both speakers' future audio autoregressively, implicitly learning conversational dynamics without any labels, and is then fine-tuned to predict interpretable turn-taking signals that map directly to agent actions. DualTurn monitors both channels continuously, anticipating turn boundaries and producing five agent actions. On standard benchmarks, DualTurn (0.5B) outperforms both VAP on agent action prediction (wF1 0.633 vs. 0.389) and a 3.1B audio-text model on word-level turn prediction (AUC 0.930 vs. 0.880), while anticipating turn boundaries earlier with fewer interruptions.