Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel without taking it. Existing evaluations study dyadic turn-taking or end-to-end agent behavior, but lack a unified, controlled evaluation of genuine floor transitions against confusable non-events in multi-party settings. We introduce MultiPartyTurnBench (MPTB), a benchmark of 240 synthetically generated, channel-separated conversations spanning two to five speakers and six interaction types. MPTB evaluates two complementary tasks: End-of-Turn (EOT) detection distinguishes genuine turn completion from temporary holds, while Interruption (INT) detection distinguishes successful floor takeovers from non-floor-taking speech. Exact event-conditioned annotations enable causal evaluation using false-positive-constrained recall and signed latency. Across acoustic, semantic, and turn-projection baselines, we find a stark asymmetry: learned systems achieve 82.5-92.6% EOT recall at 4.3-11.8% false-positive rates, while INT detection suffers from missed takeovers or excessive false activations. This asymmetry extends to five full-duplex agents which achieve 86.1-95.8% balanced accuracy on turn entry, yet perform near chance (46.9-54.7%) on selective interruption handling. Gemini-3.8-Live yields to 97.7% of backchannels, whereas MiniCPM-o-4.5 yields to only 1.4% of genuine interruptions. These findings establish selective floor control as a distinct, unresolved challenge for reliable multi-party voice interaction.
Figures & tables
Figure 1: MultiPartyTurnBench evaluation tracks. The End-of-Turn (EOT) track distinguishes genuine floor relinquishment from mid-turn pauses where the speaker retains the floor. The Interruption (INT) track distinguishes successful floor takeovers from backchannels and non-content vocalizations that leave floor ownership unchanged. Non-floor-taking interruption attempts are excluded from causal INT scoring because their ambiguous outcomes become apparent only through subsequent interaction.
Benchmarks
Multi- party
EOT event detection
INT event detection
Event-specific negatives
Timestamped event scoring
FPR budget
Controlled Speaker Comparison
Agent behavior scoring
Turn-taking and interruption benchmarks
TurnBench ( Jiang and others, 2026 )
✗
✓
✓
✓
✓
✓
✗
✗
Talking Turns ( Arora et al., 2025a )
✗
✗
✗
✗
✗
✗
✗
✓
SID-Bench ( Xia et al., 2026 )
✗
✗
✓
✓
✓
✗
✗
✗
ECHO ( Zhao et al., 2026b )
✗
✗
✗
✓
✗
✗
✗
✓
Multi-party turn-taking evaluation resources
Table 1: Comparison of turn-taking and full-duplex evaluation benchmarks. Existing benchmarks address dyadic turn-taking events, natural multi-party next-speaker predictions, or agent interaction behavior. MultiPartyTurnBench jointly evaluates multi-party EOT and INT event detection with event-specific hard negatives, timestamped causal decisions, FPR-constrained scoring, controlled speaker-count comparisons, and complementary agent behavior validation.
Event
Floor outcome
EOT
INT
Turn : yield
Released
✓
–
hold
Retained
×
–
interrupted
Taken over
–
–
Interruption
Transferred
–
✓
Non-Floor-Taking Int.
Retained
–
Excl.
Backchannel
Retained
–
×
Table 2: Floor-state event taxonomy in MultiPartyTurnBench . Events are mapped to EOT and INT evaluation targets: ✓ denotes a positive event, × a hard negative, and – an event not scored in that track. Excl. denotes events explicitly excluded from scoring.
Figure 2: MultiPartyTurnBench conversation generation pipeline. Five-stage pipeline transforms event-conditioned dialogue scripts into synchronized multi-speaker audio with exact floor-transition annotations. A discrete-event orchestrator resolves floor ownership and temporal dynamics before speech synthesis, followed by speaker rendering, acoustic augmentation, and channel assembly. The example illustrates overlapping speech, backchannels, successful interruptions, and their event annotations.
Figure 3: MultiPartyTurnBench evaluation protocol. Causal per-speaker predictions are matched to reference events and scored against hard-negative spans. Development-selected thresholds are frozen for test evaluation, with performance measured through recall, false-positive rate, and latency.
End-of-Turn (EOT)
Interruption (INT)
System
Recall (%) ↑
FPR (%) ↓
p50 (ms) ↓
Recall (%) ↑
FPR (%) ↓
p50 (ms) ↓
Acoustic VAD
RMS Energy
49.8 [44.4, 57.3]
40.9
292
66.2 [61.5, 72.8]
25.0
859
WebRTC VAD ( Google WebRTC Project, )
91.8 [90.8, 92.8]
90.3
332
84.3 [80.6, 89.2]
83.8
128
Silero VAD ( Silero Team, 2024 )
97.2 [96.3, 97.8]
93.5
397
85.4 [81.4, 89.7]
89.2
119
Semantic endpointers
Table 3: EOT and INT detection on MultiPartyTurnBench . Recall is reported with 95% conversation-bootstrap CI; p50 denotes median signed latency (ms). Gray indicates operating points satisfying dev FPR ≤10% and test FPR ≤15% ; cyan highlights the highest qualified recall in each task. Budget-qualified systems achieve up to 92.6% EOT recall, yet the best INT recall reaches only 56.2%. Acoustic detectors recover more events but incur substantially higher FPR, highlighting the difficulty of distinguishing genuine floor transitions from speech activity.
START: enter the floor
STOP: yield selectively
Voice agent
Start ↑
False start ↓
bACC ↑
Yield ↑
False yield ↓
bACC ↑
ABias→0
GPT-Live-1 ( OpenAI, 2026 )
93.3
1.7
95.8 [90.8, 99.2]
50.0
40.6
54.7 [45.7, 63.5]
-0.09
Gemini-3.8-Live ( Google, 2026 )
88.9
0.0
94.4 [90.0, 97.8]
100.0
97.7
51.1 [50.0, 52.8]
+0.98
MiniCPM-o-4.5 ( Cui et al., 2026 )
86.7
5.6
90.6 [85.0, 95.6]
1.4
0.0
50.7 [50.0, 52.2]
-0.99
Realtime-Venus ( Zhao et al., 2026a )
81.1
8.9
86.1 [80.6, 91.1]
8.1
12.7
47.7 [43.1, 52.4]
-0.79
Step-Audio3-Realtime ( Lin et al., 2026a )
0.0
0.0
50.0 [50.0, 50.0]
9.1
15.4
46.9 [34.6, 60.0]
-0.76
Table 4: Full-duplex agents know when to start, but not when to stop. START distinguishes genuine turn endings from hold pauses; STOP distinguishes interruptions from backchannels. Four agents achieve 86.1–95.8% START balanced accuracy (bACC), yet all five remain near chance on STOP (46.9–54.7%). Gemini yields to 97.7% of backchannels, whereas MiniCPM-o yields to only 1.4% of interruptions, exposing opposite failure behaviors. Brackets denote 95% scenario-clustered bootstrap CIs; cyan highlights above-chance START performance; red marks STOP estimates whose CIs include 50%.
Figure 4: Speaker-count sensitivity of EOT and INT detection. Recall by speaker count ( N=2 – 5 ), with 95% conversation-bootstrap CI. EOT is broadly stable except for Kyutai, whereas INT trends differ. At five speakers, no plotted detector reaches 50% INT recall, highlighting the challenge of identifying successful floor takeovers among multiple speech claimants.
Figure 5: EOT / INT recall by interaction type on MultiPartyTurnBench test set. Darker means higher recall. † denotes over the FPR budget; its recall reflects indiscriminate firing. EOT recall remains comparatively stable across conversation types. INT recall swings up to 30 points. Multi-party floor-control difficulty depends on conversational dynamics (argumentative for VAP and Mimi EP, collaborative for Kyutai, narrative for MuVAP, and task-oriented for DualTurn), not merely on speaker count.
Generator label, event log, model predictions, condition metadata
Appendix
Table 5: Human annotation schema. Annotators independently identify event categories, floor holders, and temporal boundaries using the role-relative taxonomy (Table 13 ). Noncontent corresponds to listener laughter in the released corpus.
Event Class
# Events
Consensus (%)
Fleiss’ κ
Boundary F1
Gen.–Human Match (%)
TURN / Yield
3,422
83.4
0.80
0.94
86.5
TURN / Hold
241
85.6
0.84
0.93
87.3
INTERRUPTION
367
81.4
0.78
0.89
91.2
NON-FLOOR-TAKING INT.
605
88.6
0.82
0.91
92.5
BACKCHANNEL
1,176
93.6
0.92
0.95
97.8
NONCONTENT
305
90.5
0.83
0.90
94.5
Appendix
Table 6: Human validation of MPTB floor-state events. Event counts cover all 240 conversations, with three independent annotator judgments per candidate. Consensus denotes majority categorical agreement; Fleiss’ κ measures chance-corrected inter-annotator agreement; boundary F1 measures temporal agreement within 500 ms; and generator–human match is conditional on consensus (Eq. 1 ). Noncontent denotes listener laughter. Non-floor-taking interruption attempts are included in human validation but excluded from causal INT scoring. Pooled agreement statistics require event-level recomputation and are not estimated by averaging class-level coefficients.
Corpus
Human Validation
N
Conv.
Events
Cons. (%)
Fleiss’ κ
Bnd. F1
2
60
1,490
89.8
0.84
0.96
3
60
1,527
88.6
0.82
0.93
4
60
1,536
84.3
0.81
0.91
5
60
1,563
82.1
0.76
0.90
Appendix
Table 7: Human agreement across speaker counts. Event totals include the six generator-defined categories in Table 6 . Categorical consensus, Fleiss’ κ , and boundary F1 consistently decrease from two to five speakers.
Interaction Type
# Conv.
Hours
Turns/min
BC/min
INT/min
Overlap (%)
Mean Turn (s)
Casual
40
1.10
12.7
4.24
0.92
11.4
4.81
Task-Oriented
40
1.03
11.4
3.80
0.89
9.9
5.27
Instructional
40
1.17
7.9
3.81
0.70
10.7
7.79
Collaborative
40
1.07
11.4
3.43
1.33
11.0
5.32
Argumentative
40
1.11
9.6
2.95
1.08
10.7
6.29
Narrative
40
1.41
6.6
3.34
0.53
9.1
9.22
Appendix
Table 8: Corpus statistics by interaction type. Turns/min counts floor-holding turn segments; BC/min includes backchannels and laughter; INT/min counts successful interruptions; overlap denotes the fraction of conversation time with simultaneous vocal activity from at least two speakers. Interaction types induce distinct conversational regimes despite balanced conversation counts.
N
# Conv.
EOT
INT
BC/min
Overlap (%)
2
60
853
80
3.12
8.9
3
60
838
96
3.61
10.9
4
60
875
94
3.60
10.9
5
60
856
97
4.06
11.1
All
240
3,422
367
3.58
10.4
Appendix
Table 9: Corpus composition by speaker count. Each party-count condition contains 60 conversations. EOT and INT denote positive event anchors; overlap measures the percentage of conversation time containing simultaneous vocal activity.
Split
# Conv.
Hours
# Voices
EOT / INT
Dev
48
1.42
8
706 / 86
Test
192
5.47
16
2,716 / 281
Total
240
6.89
24
3,422 / 367
Appendix
Table 10: MPTB development and test partitions. The partitions use disjoint TTS voices and retain the complete speaker-count × interaction-type design, with two development and eight test conversations per condition. EOT and INT denote positive event counts.
Track
Positive: floor changes
Hard negative: floor unchanged
Prediction anchor
EOT
Current floor holder completes a turn and yields the floor.
Mid-turn pause after which the same speaker resumes while retaining the floor.
End of floor-holding speech
INT
Listener begins speaking while another speaker holds the floor and successfully takes ownership.
Backchannel or non-content listener activity that does not change floor ownership.
Onset of floor-taking speech
Appendix
Table 11: MultiPartyTurnBench evaluation tracks. Each track contrasts a floor-changing event with a conversationally similar event that leaves floor ownership unchanged. This paired formulation prevents systems from succeeding through indiscriminate silence or speech-activity detection.
EOT
INT
Split
Conv.
Pos.
Neg.
Pos.
Neg.
Dev
48
706
55
86
309
Test
192
2,716
186
281
1,172
Total
240
3,422
241
367
1,481
Appendix
Table 12: Scored denominators (positive anchors and hard-negative spans). Every test-split number in the paper is computed over these anchors. INT negatives comprise 1,176 backchannels and 305 laughter vocalizations (test: 919 and 253); the 605 non-floor-taking claims are excluded intervals.
Event
Operational meaning
Role
Count
turn –yield
Holder releases the floor to another speaker (or the conversation ends).
EOT +
3,422
turn –hold
Holder pauses mid-turn and resumes.
EOT −
241
turn –interrupted
Holder is cut off by a successful interruption.
EOT excl.
367
interruption
Listener takes the floor before the current turn naturally ends.
INT +
367
non-floor-taking int.
Listener attempts a claim but the holder keeps the floor.
INT excl.
605
backchannel
Listener vocalizes ( ≤ 3 words) without claiming the floor.
INT −
1,176
Appendix
Table 13: Role-relative event taxonomy used by the generator and scorer, with counts over all 240 conversations. Acoustic nuisance (ambient bed, sound effects, distant speech) is rendered in Stage 4 and carries no label.
Interaction Type
Conversation Setting
Expected Floor-Management Characteristics
Casual
Unstructured social conversation with flexible topic progression and informal exchange.
Frequent acknowledgements and backchannels, relatively short turns, topic shifts, and occasional overlap.
Task-Oriented
Goal-directed interaction in which participants jointly complete or resolve a concrete task.
Frequent clarification and confirmation, comparatively structured floor transfer, and short coordination turns.
Instructional
Asymmetric interaction in which one participant primarily explains, teaches, or guides others.
Longer floor holds by the primary speaker, question-driven transitions, and acknowledgement backchannels from listeners.
Collaborative
Shared reasoning or problem solving without a single dominant speaker.
More frequent floor exchange, cooperative overlap, short claims to the floor, and relatively balanced participation.
Argumentative
Structured disagreement or debate in which participants defend competing positions.
Longer contested turns, more competitive floor claims, interruptions, and overlapping speech.
Narrative
One participant primarily recounts an event or story while others act as listeners.
Table 14: Interaction types used in MultiPartyTurnBench. MPTB adopts the six conversation categories used in TurnBench ( Jiang and others, 2026 ) and instantiates each across N∈{2,3,4,5} speakers. The final column summarizes the intended floor-management regime used to condition dialogue generation; realized event statistics are measured from the generated corpus rather than assumed from these descriptions.
Soft targets (TurnBench dyadic statistics)
Type
Style string inserted into the prompt
Turn (s)
BC/min
INT/min
Words/min
Argumentative
a heated debate with frequent disagreement, interruptions, and people talking over each other
10.0
3.09
2.48
206
Casual
a relaxed casual chat with frequent overlaps, backchannels, and light interruptions
8.1
5.20
2.05
211
Collaborative
participants actively building on each other’s ideas, with overlaps and frequent floor contests
7.9
3.97
2.64
207
Instructional
one participant explaining/teaching others who ask clarifying questions; mostly clean turns, few interruptions
10.5
4.30
1.52
202
Narrative
storytelling with personal anecdotes; longer turns, listeners backchanneling, rare interruptions
10.2
4.76
1.50
198
Appendix
Table 15: Generator conditioning per interaction type. For a conversation of target duration D (min) with N speakers, the prompt requests about max(2N,round(60D/turn)) floor-holding turns, round(INT/min⋅D⋅sN) floor claims and round(BC/min⋅D⋅sN) backchannels/laughter, with sN=1+0.1(N−2) . Targets are guardrails; realized rates are reported in Tables 20 and 8 .
Figure 6: One generated excerpt per interaction type (test split, verbatim script text; tags are the generator’s event labels, highlighted lines are scored anchors). Argumentative and collaborative claims cut into unfinished clauses; casual and task-oriented claims are short coordination moves; instructional claims are listener questions; narrative regions are dominated by stacked listener backchannels that must not be scored as takeovers.
Domains (20). tech support call; study group / exam prep; family logistics planning; product design standup; podcast panel / current-events discussion; board game night; job interview; travel itinerary planning; classroom discussion; relationship/interpersonal conflict; restaurant/group dining decision; sports commentary; book club; home renovation planning; medical appointment scheduling (logistics only, non-clinical); roommate household planning; startup pitch practice; book/movie recommendation swap; volunteer event coordination; group trivia night.
Tones (5; realized count). professional and business-like (57); curious and inquisitive (54); enthusiastic and energetic (53); casual and friendly (42); thoughtful and contemplative (34).
Appendix
Table 16: Scenario conditioning vocabulary. All 20 domains occur in the corpus (most frequent: volunteer event coordination, 19 conversations).
Figure 7: Stage-1 system prompt , shared by all 240 conversations. The LLM proposes symbolic acts only; it never assigns timestamps or final floor outcomes.
Figure 8: Stage-1 user prompt. The template is filled from the conversation specification and Table 15 ; the middle box shows the instantiated prompt of the narrative excerpt in Figure 6 . On validation failure the repair message is appended and the model is queried again.
Parameter
Value
Lead-in silence
0.3 s
Yield gap, after_pause
U(0.20,0.60) s
Yield gap, latched
U(−0.12,0.06) s
Yield gap, long_pause
U(0.80,2.00) s
Same-holder continuation gap
U(0.02,0.18) s
Mid-turn pause: probability / min. segment
0.45 / 4.0 s
Appendix
Table 17: Stage-2 orchestrator parameters. Realized timing distributions are given in Table 21 .
ID
Kokoro
G
Style
ID
Kokoro
G
Style
v01
af_alloy
F
clipped/efficient
v13
am_onyx
M
upbeat/expressive
v02
af_aoede
F
energetic/upbeat
v14
af_nicole
F
dry-humor/deadpan
v03
am_adam
M
calm/measured
v15
am_puck
M
laid-back/casual
v04
am_echo
M
forceful/assertive
v16
am_santa
M
deadpan/flat
v05
am_eric
M
warm/friendly
v17
af_nova
F
skeptical/probing
v06
am_fenrir
M
hesitant/soft-spoken
v18
bm_daniel
M
nervous/tentative
Appendix
Table 18: Voice pool and split. Shaded ids form the development pool (4F/4M); the remaining 16 form the test pool. Prefix a / b = American/British English. The style descriptor is passed to the script generator.
One random MUSAN sound-bible clip ( ≤ 4 s) per sampled event slot, 12 dB below active speech, random time; k∼{0,1,1,2} slots
0 / 1 / 2 slots: 49 / 124 / 67 conv.
Distant speech
2–4 faded MUSAN speech snippets of 1.5–3 s at 26 dB below active speech; p=0.2
51 conversations
Appendix
Table 19: Stage-4 noise model. SNR is measured against the active-speech RMS pooled over channels. Event-slot labels: keyboard clatter, notification chime, dishes, off-mic laughter, phone ring, door knock, chairs, school bell, alarm beep, paper shuffle (23–30 occurrences each). Scenes: outdoor park 33, living room TV 32, school hallway 27, office HVAC 25, studio room tone 24, cafe 22, office hum 22, dorm 20, quiet office 19, quiet apartment 16.
Corpus Size
Interaction Dynamics
Benchmark Events
Condition
Dialogues
Duration (h)
Mean Turn (s)
Events/min
Backchannels/min
Interruptions/min
Overlap/dialogue (s)
EOT Pos./Neg.
INT Pos./Neg.
By speaker count
N=2
60
1.90
7.07
13.8
3.12
0.70
10.2
853/74
80/356
N=3
60
1.66
6.38
16.3
3.61
0.96
10.8
838/64
96/359
N=4
60
1.67
6.10
16.3
3.60
0.94
10.9
875/55
94/361
N=5
60
1.66
6.26
16.6
4.06
0.97
11.0
856/48
97/405
Appendix
Table 20: Corpus composition and turn-taking dynamics across speaker counts and interaction types. MPTB maintains balanced coverage across N=2 – 5 while exhibiting relatively stable conversational activity as speaker count increases. Interaction types induce more distinct floor-management patterns: Casual conversations exhibit faster exchanges and shorter turns, Collaborative conversations have the highest interruption rate, and Narrative conversations sustain the longest turns with the fewest interruptions. EOT and INT +/− indicate positive anchors and hard-negative spans, respectively.
Quantity (s)
n
p10
p50
p90
Floor-holding segment
4,030
2.19
5.18
10.65
Yield gap (next onset − offset)
3,182
− 0.09
0.01
0.48
Hold pause span ( EOT − )
241
0.49
0.83
1.12
Interruption overlap (onset → cut-off)
367
0.43
0.74
1.09
Failed claim duration
605
0.80
1.63
3.72
Backchannel duration ( INT − )
1,176
0.44
0.64
1.04
Appendix
Table 21: Realized timing distributions over all 240 conversations. Conversations last 48.2–208.2 s (median 99.9 s). Negative yield gaps are latched hand-overs that begin before the previous holder finishes.
System
Checkpoint / implementation
Input and context
EOT score
INT score
RMS Energy
frame RMS (dBFS)
per channel, 20 ms frames
speech → silence offset after 0.3 s silent hangover
speech onset while another channel is voiced, confirmed after 0.1 s
WebRTC VAD
py-webrtcvad
per channel, 16 kHz, 20 ms frames; θ = aggressiveness
per channel, 16 kHz, ≤ 8 s window ending at each energy-VAD offset ( −40 dBFS, 0.3 s hangover)
P(complete)
1−P(complete) in TurnBench’s Smart Turn pipeline (Silero VAD, 512-sample chunks)
Kyutai SVAD
kyutai/stt-1b-en_fr-candle
per channel, streaming Mimi, 12.5 Hz
P(end of turn) (VAD head 2)
1−P(end of turn)
Mimi Endpointer
viks66/mimi-endpointer ( oto_d1f ), two-stream LSTM over Mimi
s vs. mean of rest, 24 kHz, 25 Hz
1−P(user)
P(user)
Appendix
Table 22: Offline system configurations. Classes c / na / i / bc / t = continuation / no-activity / interruption / backchannel / turn-shift. Every score is converted to decisions by rising edges above θ with a 2 s refractory period. Mappings follow TurnBench’s reference adapters wherever one exists.
Rule
Threshold
Same-speaker segments merged (contiguous)
gap <0.15 s
Mid-turn pause → turn –hold ( EOT − )
gap ≥0.20 s
Overlap counted as “during” the holder
≥0.10 s
Backchannel ( INT − ): duration / words
≤1.20 s / ≤3
Interruption ( INT + ): interrupter speaks at least
1.00 s
and holder falls silent within
2.50 s of onset
Appendix
Table 23: AMI floor-state mapping from manual segments to MPTB events.
MPTB (ours)
TurnBench official
TurnBench dev slice
AMI meetings
Audio
synthetic TTS + noise
real, studio
real, studio
real, headset
Speakers
N=2 – 5
N=2
N=2
N=4
Labels
generator script (exact)
3-annotator consensus
3-annotator consensus
derived from manual segments
Threshold
MPTB dev → test
TB dev → test
tuned in-slice
tuned in-slice
Conversations
192 test
116 test
12
15
System
EOT
INT
EOT
INT
EOT
INT
EOT
INT
Appendix
Table 24: Interruption detection remains challenging in natural multi-party speech. Recall / FPR (%) for systems evaluated on synthetic MPTB, the official TurnBench leaderboard (real dyadic speech; TurnBench’s own runs), our TurnBench dev slice, and real four-party AMI meetings. Gray : within that source’s FPR budget. Sources differ in audio, labels and operating-point selection (header rows), and both real slices tune thresholds on the evaluated data. Absolute numbers are therefore not comparable across columns but only in directions. Every learned system has lower EOT recall on real audio than on MPTB. Interruptions, however, collapse in multi-party speech on both real and synthetic audio. VAP, the one learned system evaluated on every source, detects 94.5% of interruptions in real dyadic TurnBench, but only 35.9% in real four-party AMI, even with in-slice tuning, and 11.7% on MPTB. VAP, Mimi Endpointer and WavLM Anchor are trained on TurnBench data and are in-domain there.
Figure 10: Interaction type matters for interruptions, not for turn ends. Recall per conversation type for the five systems within the FPR budget on both tracks (test split, 32 conversations per type). Gray bars span each system’s worst-to-best type and black ticks mark pooled recall. EOT recall varies by at most 5.5 points across types. INT recall swings by up to 30.5 points (Kyutai: 47.0% collaborative vs. 77.5% instructional). The hardest INT type is model-specific: argumentative for VAP and Mimi EP, collaborative for Kyutai, narrative for MuVAP, and task-oriented for DualTurn.
N
EOT recall/FPR
INT recall/FPR
2
.93/.04
.07/.03
3
.93/.04
.10/.04
4
.92/.16
.11/.03
5
.92/.08
.18/.02
Appendix
Table 25: VAP test performance by party count. The N=4 EOT FPR exceeds the nominal budget after the dev-selected threshold is frozen.
EOT recall / FPR (%)
INT recall / FPR (%)
System
N=2
N=3
N=4
N=5
N=2
N=3
N=4
N=5
Acoustic VAD
RMS Energy
48.5 / 38.2
61.1 / 62.5
41.9 / 25.0
48.1 / 35.9
55.0 / 22.8
74.0 / 34.4
66.7 / 18.8
67.1 / 23.9
WebRTC VAD
92.1 / 85.5
93.8 / 95.8
90.4 / 86.4
90.9 / 94.9
70.0 / 83.0
82.2 / 83.0
88.9 / 82.7
93.4 / 86.1
Silero VAD
97.8 / 94.5
98.0 / 93.8
96.0 / 95.5
96.9 / 89.7
73.3 / 89.1
82.2 / 89.2
91.7 / 88.4
92.1 / 90.0
Semantic endpointers
Appendix
Table 26: Recall / FPR by party count for every system (test split, pooled operating point). The ESPnet per-channel variant runs the same checkpoint on each isolated channel instead of the mono mix.
EOT
INT
System
θ
Dev R / F
Test R / F
θ
Dev R / F
Test R / F
RMS Energy
−50 dBFS
44.9 / 30.9
49.8 / 40.9
−50 dBFS
51.2 / 19.7
66.2 / 25.0
WebRTC VAD
aggr. 3
91.6 / 87.3
91.8 / 90.3
aggr. 3
83.7 / 84.5
84.3 / 83.8
Silero VAD
0.30
97.6 / 90.9
97.2 / 93.5
0.70
83.7 / 86.4
85.4 / 89.2
Smart Turn v2
0.50
78.3 / 14.5
82.5 / 11.8
0.0442
73.3 / 58.9
67.6 / 64.3
Kyutai SVAD
0.95
93.2 / 3.6
92.5 / 4.3
0.9991
65.1 / 6.1
56.2 / 6.6
Appendix
Table 27: Dev-selected operating points. R/F = recall/FPR (%). θ is the score threshold (dBFS for RMS, WebRTC aggressiveness level, probability otherwise). Gray rows are in budget on both tracks. Candidate grids: RMS {−50,−45,…,−30} dBFS; WebRTC {0,1,2,3} ; Silero {0.3,…,0.7} ; Kyutai and Smart Turn EOT {0.3,…,0.9,0.95} ; VAP and MuVAP {0.5,…,0.9,0.95} ; other learned systems {0.05,0.10,…,0.95,0.97,0.98,0.99,0.995} plus dev-score quantiles.
EOT (ms)
INT (ms)
System
p10
p50
p90
p10
p50
p90
RMS Energy
267
292
1653
−38
859
2572
WebRTC VAD
224
332
380
114
128
146
Silero VAD
359
397
435
92
119
198
Smart Turn v2
−32
−18
614
55
87
446
Kyutai SVAD
666
1829
1960
432
689
1121
Appendix
Table 28: Signed latency tpred−tgold of matched detections on the test split. ESPnet INT percentiles rest on 3 matched detections.
System − VAP
Δ Recall
Δ FPR
EOT
Kyutai SVAD
−0.1 [ −1.5 , 1.3]
−3.2 [ −7.7 , 1.1]
Mimi EP
−2.1 [ −2.9 , −1.3 ]
−3.8 [ −8.1 , 0.5]
DualTurn
−3.2 [ −4.0 , −2.3 ]
+3.2 [ −1.6 , 8.2]
MuVAP
−4.5 [ −5.6 , −3.4 ]
+2.7 [ −1.8 , 7.2]
WavLM
−2.2 [ −3.0 , −1.4 ]
+15.1 [7.0, 23.1]
Turn-1-Mini
−5.7 [ −6.9 , −4.5 ]
+17.7 [10.2, 25.5]
Appendix
Table 29: Paired differences to VAP (points, 95% CI; B=10,000 conversation-cluster resamples of the 192 test conversations).
EOT recall by SNR (dB)
INT recall by SNR (dB)
EOT FPR, distant speech
INT FPR, distant speech
System
20–25
25–30
30–35
35–40
20–25
25–30
30–35
35–40
absent
present
absent
present
Kyutai SVAD
91.9
94.5
92.8
89.6
52.4
56.7
59.3
55.3
3.9
6.1
6.8
5.6
Mimi Endpointer
91.2
91.5
89.2
89.8
12.7
13.3
16.0
12.8
4.6
0.0
3.6
4.8
VAP
92.7
92.5
92.4
92.9
12.7
10.0
13.6
10.6
7.8
6.1
2.8
2.8
MuVAP
88.8
88.7
87.2
87.6
25.4
33.3
40.7
38.3
10.5
9.1
9.3
6.0
DualTurn
89.7
89.6
89.5
88.8
34.9
34.4
39.5
40.4
10.5
12.1
8.8
6.0
Appendix
Table 30: Robustness slices (test split, %, pooled operating points). Signal to Noise Ratio (SNR) bins refer to the ambient bed.
EOT FPR on holds (%)
Share of
System
contested ( n=28 )
uncontested ( n=158 )
false alarms
Kyutai SVAD
7.1
3.8
2 / 8
Mimi Endpointer
25.0
0.0
7 / 7
VAP
39.3
1.9
11 / 14
MuVAP
57.1
1.9
16 / 19
DualTurn
42.9
5.1
12 / 20
Appendix
Table 31: Contested holds. A hold is contested when another speaker’s non-floor-taking claim overlaps the pause span. Last column: contested false alarms / all EOT false alarms on the test split.
EOT (pre-pause / pre-yield clause ending)
INT FPR on negatives
INT recall by claim length
FPR on holds
Recall on yields
by word count
by type
words
System
“.”
“?”
“—”
“.”
“?”
1
2–3
4+
BC
Laugh
1–3
4–8
9+
Kyutai SVAD
1.8
9.1
7.7
92.2
93.6
0.2
5.7
72.3
2.4
21.7
37.2
55.6
62.6
Mimi Endpointer
2.7
9.1
2.6
88.9
95.0
1.5
4.5
20.0
2.6
8.3
23.3
12.1
12.2
VAP
9.9
9.1
0.0
90.8
96.9
1.2
3.7
10.8
2.6
3.6
9.3
14.1
10.8
MuVAP
9.9
15.2
7.7
86.2
92.9
6.0
9.4
26.2
7.3
13.4
27.9
37.4
34.5
Appendix
Table 32: Error anatomy (test split, %). Clause ending is the final character of the script utterance before the hold pause or yield (“—” marks a self-interrupted clause). Word counts are taken from the script text of the listener vocalization or the interrupting claim. † Smart Turn EOT from the re-run on released audio at the frozen threshold.
Constant
Value
Scenario families × size
4 × 30 (balanced N=2,3,4 )
Max. scenarios per source conversation
2
Selection seed
20260929
Repeats per scenario and agent
3
Stream audio
mono mix of all participants’ noisy channels, 24 kHz; the agent joins as an additional party
START context before target turn onset
20 s
Appendix
Table 33: Full-duplex protocol constants. All constants are identical across agents; no protocol parameter was adapted to any model.
Figure 11: Agent-facing text. The instruction never mentions turn-taking evaluation, interruptions or measurement. Questions elicit a 4–8 s answer that requires no reasoning, so the response content cannot confound the stop decision; only speech onset and cessation are scored.
true EOT (START + )
hold pause (START − )
true INT (STOP + )
backchannel (STOP − )
Model
N=2
3
4
Cons.
N=2
3
4
Cons.
N=2
3
4
Cons.
N=2
3
4
Cons.
Gemini-3.8-Live
.93
.80
.93
.77
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
.07
.00
.00
.93
GPT-Live-1
.95
.90
.95
.93
.95
1.00
1.00
.97
.35
.50
.65
.40
.48
.60
.75
.60
MiniCPM-o-4.5
.97
.77
.87
.77
.87
1.00
.97
.87
.04
.00
.00
1.00
1.00
1.00
1.00
1.00
Realtime-Venus
.70
.87
.87
.60
.93
.90
.90
.77
.13
.04
.07
.77
.85
.90
.88
.67
Step-Audio-3
.00
.00
.00
1.00
1.00
1.00
1.00
1.00
.00
.20
.00
1.00
1.00
.80
.80
.50
Appendix
Table 34: Behavioral accuracy by party count and run-to-run consistency (Cons.). Accuracy is over valid trials in each N (10 scenarios × up to 3 repeats). Step-Audio-3 STOP cells rest on 2–5 valid trials each.
Model
Family
p10
p50
p90
n
Gemini
true EOT
1141
2036
2258
80
Gemini
true INT
200
230
263
90
GPT-Live-1
true EOT
980
1313
1796
56
GPT-Live-1
true INT
1094
1491
2204
60
MiniCPM-o
true EOT
442
870
1223
78
MiniCPM-o
true INT
2000
8600
11000
74
Appendix
Table 35: Selected response latencies (ms) for valid trials with measurable responses.
Model
INT barge-in
BC barge-in
Resume <2 s
Gemini
1.000
.955
.022
GPT-Live-1
.000
.000
.617
MiniCPM-o
.000
.000
.122
Venus
.000
.000
.243
Step-Audio-3
.000
.000
.091
Appendix
Table 36: Fraction of STOP events associated with provider barge-in/cancel events and short resumption rate after true interruptions.
Detector correct
Detector wrong
Probe
beh. ok
beh. wrong
beh. ok
beh. wrong
START
487
122
91
20
STOP
160
114
98
134
Appendix
Table 37: Event-level contingency between offline detector correctness and behavioral correctness on the same scenario (valid trials of the four agents with an offline pass).
Figure 12: Interruption track: qualitative cases. (a)–(c) positives, (d) a hard negative. Decisions use each system’s frozen test operating point.
Figure 13: Hard negatives: qualitative cases. (a) INT negative; (b)–(c) EOT holds. Learned endpointers reject incomplete clauses but confuse a contested pause with a yield.
Figure 14: Full-duplex agents: qualitative cases. Each mark is one repeat of the same scenario. Grey numbers are the time from injection to the end of the agent’s ongoing utterance (ms unless stated).
Speakers in natural conversation take turns speaking and listening, deciding in real time when to take, hold, or yield the floor. However, turn-taking evaluation remains limited due to the lack of a consistent, linguistically grounded evaluation protocol and hand-annotated data covering diverse conversation types. To address this, we present TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection. We set conversation type as a controllable experimental variable, covering six distinct interaction styles, and triple-annotate each conversation. Benchmarking 14 heterogeneous turn-taking systems, we find end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in backchannel-dense interaction styles. Although in smooth floor transfers human listeners begin speaking a median 151 ms before the current turn ends, no current system performs equivalently without incurring excessive false positives. We release our corpus, a 104-hour training set, and a public leaderboard with an interactive dataset viewer at https://turnbench.sesame.com.
Real-time spoken dialogue systems must distinguish interruptions that require yielding the floor from backchannels that permit continued speaking. Existing benchmarks typically score events independently and may therefore assign high scores to systems with fixed action preferences rather than context-sensitive decision policies. We introduce ECHO, a paired diagnostic benchmark for Chinese turn-taking evaluation. ECHO pairs examples with the same overlap transcript but contrasting preceding multi-turn dialogue contexts, with one requiring Yield and the other Keep. It additionally includes off-talk examples for diagnosing unnecessary yielding. We introduce pair accuracy, which requires correct decisions on both members of a pair and assigns no credit to constant-action policies. Experiments on four speech systems show that three exhibit a severe over-yielding bias: they correctly keep the floor on fewer than 13% of backchannels, resulting in near-zero pairwise success rates equal or less than 4%. While the remaining system remains comparatively balanced across contexts, these findings broadly demonstrate that interruption-only evaluation can severely overestimate practical turn-taking reliability.
Real-time full-duplex speech models can listen while speaking, enabling natural interaction without rigid turn boundaries. Existing benchmarks evaluate turn-taking, interruption handling and multi-round dialogue, but largely centre on a designated user rather than an assistant participating in a shared conversation among several people. We introduce Duplex-MPE to evaluate when such an assistant should answer, remain silent or stop speaking. The benchmark contains 2,000 scenarios with three or four human speakers and one assistant, each paired across explicit and implicit addressing of the same request. Models receive continuous conversation audio without transcripts or supplied turn boundaries. Four scores measure fresh response initiation, answer accuracy, silence preservation and stopping when a human resolves a request. We evaluate five open-weight speech systems: MiniCPM-o 4.5, Moshi, FLM-Audio, Voila and Freeze-Omni. MiniCPM-o 4.5 leads on three scored capabilities, while frequent speech from other systems can coexist with inaccurate answers or failures to remain silent. A transcript-based Gemini 3.1 Pro reference responds 64.3 percentage points more often to explicit than implicit requests; paired tests detect no significant response-rate difference for the speech systems.
Chengqian Ma, Wenhao Feng, Weixuan Jin +7
Peking University · Renmin University of China · Tsinghua University +3