DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements

Keyue Xing; Wentao Ding; Mengmeng Wang; Wenming Tu; Zilong Zheng; Yipeng Kang

cs.CLSep 30, 2026

DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements

Authors: Keyue Xing, Wentao Ding, Mengmeng Wang, Wenming Tu, Zilong Zheng, Yipeng Kang

Organizations: State Key Laboratory of General Artificial Intelligence, BIGAI, China · Peking University, China · X-LANCE Lab, Shanghai Jiao Tong University, China

Abstract

Existing full-duplex speech benchmarks cover only subsets of real-time interaction behaviors, often under limited contextual conditions. We introduce DuplexAct-Bench, a bilingual benchmark that systematically covers six complementary behaviors, from interruption and yielding to proactive initiation, active silence, and backchanneling, across Pre-session, In-session, and No-explicit conditions. Across 1,290 English and Chinese streaming trials, we evaluate 12 full-duplex speech systems on both Timing and Content. Results reveal substantial variation across behaviors, conditions, and systems, as well as frequent mismatches between semantic quality and behavioral timing. These findings show that current systems remain far from robustly managing when, whether, and how to participate as real-time interaction unfolds. Project page: https://alitaxky.icu/DuplexAct-Bench/

Figures & tables

Figure 1: Representative examples of six interaction behaviors under different contextual conditions in DuplexAct-Bench. Blue/red waveforms denote user/agent speech. Peach shading shows behavior-specific latency for Timing evaluation.

Benchmark	AI	UI	IR	AS	PI	AB
Talking Turns [ 6 ]	$\circ$	$\circ$	$\circ$	$\circ$	✗	$\circ$
FDB v1 [ 7 ]	✗	✓	✗	$\circ$	✗	$\circ$
FDB v1.5 [ 8 ]	✗	✓	$\circ$	✗	✗	✗
FD-Bench [ 9 ]	$\circ$	✓	$\circ$	✗	✗	✗
FLEXI [ 10 ]	$\circ$	✓	$\circ$	$\circ$	✗	$\circ$
HumDial-FD [ 12 ]	✗	✓	$\circ$	$\circ$	✗	✗

Table 1: Comparison of real-time interaction behavior coverage and contextual variation across full-duplex benchmarks.

Figure 2: Behavior taxonomy and interaction scenarios in DuplexAct-Bench. The inner ring shows six behavior families, while the outer ring presents their corresponding interaction scenarios. The number shown in each behavior segment denotes the number of trials in that family, totaling 1,290 trials. Colored arcs indicate the three contextual conditions: red for Pre-session , blue for In-session , and black for No-explicit .

Figure 3: Backchannel timing annotations for a No-explicit otoSpeech trial. Green denotes dataset-provided annotations; blue, orange, and purple denote additional opportunities identified by Qwen3.8-Omni-Flash-Realtime, Doubao-Seed-2.1-Lite, and MiniCPM-o-4.5, respectively.

System	Lang.	Agent			User	Interruption				Active		Proactive		Agent
		Interruption			Interruption	Resistance				Silence		Initiation		Backchannel
		P	I	N	N	P	$\mathbf{I}_{\mathrm{Int.}}$	$\mathbf{I}_{\mathrm{Sing.}}$	N	P	I	P	N	P	I	N
Locally deployed systems
Freeze-Omni	en/zh	17.5	1.3	2.5	9.9	52.5	0.0	0.0	75.0	88.8	5.0	0.0	0.0	0.0	1.3	0.0
MiniCPM-o 4.5	en/zh	28.8	8.8	2.5	24.8	67.5	20.0	10.0	95.0	56.3	10.0	1.3	0.0	28.0	8.8	0.0
Moshi	en/zh	0.0	0.0	0.0	26.1	2.5	0.0	0.0	62.5	7.5	7.5	35.0	0.0	–	12.5	13.8

Table 2: Evaluation coverage and Behavioral Correctness Rate (BCR, %) on DuplexAct-Bench. Lang. denotes the evaluated languages (en: English; zh: Chinese). P/I/N denote Pre-session/In-session/No-explicit;

I_{\mathrm{Int.}}

and

I_{\mathrm{Sing.}}

denote the In-session simultaneous-interpretation and collaborative-singing scenarios. Active Silence is scored by its silence criterion alone; User Interruption is evaluated on Valid trials only. “–” indicates no reported result, not zero. Bold marks the highest reported BCR in each column.

System	Agent		User		Interruption		Proactive		Agent
	Interruption		Interruption		Resistance		Initiation		Backchannel
	$C$ $\uparrow$	$L$ (s) $\downarrow$	$C$ $\uparrow$	$L$ (s) $\downarrow$	$C$ $\uparrow$	$L$ (s) $\downarrow$	$C$ $\uparrow$	$L$ (s) $\downarrow$	$C$ $\uparrow$	BAS $\uparrow$
Locally deployed systems
Freeze-Omni	1.66	9.67	1.56	21.22	2.69	15.98	0.70	14.03	1.09	0.005
MiniCPM-o 4.5	2.32	9.26	1.97	19.15	3.13	13.60	0.70	13.98	1.59	0.054
Moshi	1.15	10.41	1.68	17.76	1.47	16.65	2.01	11.01	1.24	0.108

Table 3: Content and timing results on DuplexAct-Bench.

C

denotes the mean Content score (0–5),

L

the mean latency in seconds, and BAS the Backchannel Alignment Score. For User Interruption, both metrics are computed on Valid trials only. Active Silence is omitted because it is evaluated solely by its silence criterion in Table 2 . Arrows indicate the preferred direction; bold marks the best reported value in each metric column.

Figure 4: Case studies of Agent Interruption (top) and Active Silence (bottom). Each pair compares systems on the same trial. The top pair illustrates differences in semantic task fulfillment despite both systems satisfying the Timing criterion; the bottom pair contrasts responding with correctly withholding speech.

Full-Duplex Spoken Dialogue Systems

Full-duplex spoken dialogue systems let voice agents listen and respond simultaneously, managing timing, overlap, interruptions, and conversational content.

See all 119 papers

DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?
Oct 13, 2026Jiaheng Dong, Shihong Tan, Gongping Huang +3Spoken Dialogue Systems Speech Language Models
The University of Melbourne · University of Birmingham · Wuhan University +1
Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue
Sep 25, 2026Chengqian Ma, Wenhao Feng, Weixuan Jin +7Voice Agent Evaluation Full-Duplex Spoken Dialogue Systems
Peking University · Renmin University of China · Tsinghua University +3
DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents
Sep 3, 2026Puneet Mathur, Dinesh ManochaAI Agent Benchmarks Instruction Following
University of Maryland College Park, USA

Oct 13, 2026J/K move · Enter open · S save

DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?

Jiaheng Dong, Shihong Tan, Gongping Huang, Jean Honorio+2

The University of Melbourne · University of Birmingham · Wuhan University+1

Spoken Dialogue Systems Speech Language Models Full-Duplex Spoken Dialogue Systems Multi-Turn Dialogue Evaluation

Open paper AskPDF arXiv

End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simutanous listening and speaking continously, requiring models to take appropriate interaction actions as conversations unfold. However, existing benchmarks lack a structured action-level formulation, often evaluate action occurrence rather than success, and provide limited coverage of diverse real-world conversations. We introduce DuplexAct, a benchmark for evaluating interaction action taking in E2E FD-SLMs. DuplexAct defines a three-level action taxonomy spanning eight interaction actions and 24 corresponding interaction conditions. We further characterize successful action-taking through immediate execution and subsequent follow-through, and develop action-specific evaluation metrics for both. DuplexAct contains 4,691 test instances spanning casual conversation, meetings, customer service, and interviews, including 2,531 instances derived from real-world conversational recordings. Together, DuplexAct provides a structured and real-world-grounded framework for systematically assessing how E2E FD-SLMs participate in ongoing spoken interaction.

Abstract

Full-Duplex Spoken Dialogue Systems

Explore similar work

DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?

Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue

DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents