Organizations: State Key Laboratory of General Artificial Intelligence, BIGAI, China · Peking University, China · X-LANCE Lab, Shanghai Jiao Tong University, China
Existing full-duplex speech benchmarks cover only subsets of real-time interaction behaviors, often under limited contextual conditions. We introduce DuplexAct-Bench, a bilingual benchmark that systematically covers six complementary behaviors, from interruption and yielding to proactive initiation, active silence, and backchanneling, across Pre-session, In-session, and No-explicit conditions. Across 1,290 English and Chinese streaming trials, we evaluate 12 full-duplex speech systems on both Timing and Content. Results reveal substantial variation across behaviors, conditions, and systems, as well as frequent mismatches between semantic quality and behavioral timing. These findings show that current systems remain far from robustly managing when, whether, and how to participate as real-time interaction unfolds. Project page: https://alitaxky.icu/DuplexAct-Bench/
Figures & tables
Figure 1: Representative examples of six interaction behaviors under different contextual conditions in DuplexAct-Bench. Blue/red waveforms denote user/agent speech. Peach shading shows behavior-specific latency for Timing evaluation.
Benchmark
AI
UI
IR
AS
PI
AB
Talking Turns [ 6 ]
∘
∘
∘
∘
✗
∘
FDB v1 [ 7 ]
✗
✓
✗
∘
✗
∘
FDB v1.5 [ 8 ]
✗
✓
∘
✗
✗
✗
FD-Bench [ 9 ]
∘
✓
∘
✗
✗
✗
FLEXI [ 10 ]
∘
✓
∘
∘
✗
∘
HumDial-FD [ 12 ]
✗
✓
∘
∘
✗
✗
Table 1: Comparison of real-time interaction behavior coverage and contextual variation across full-duplex benchmarks.
Figure 2: Behavior taxonomy and interaction scenarios in DuplexAct-Bench. The inner ring shows six behavior families, while the outer ring presents their corresponding interaction scenarios. The number shown in each behavior segment denotes the number of trials in that family, totaling 1,290 trials. Colored arcs indicate the three contextual conditions: red for Pre-session , blue for In-session , and black for No-explicit .
Figure 3: Backchannel timing annotations for a No-explicit otoSpeech trial. Green denotes dataset-provided annotations; blue, orange, and purple denote additional opportunities identified by Qwen3.8-Omni-Flash-Realtime, Doubao-Seed-2.1-Lite, and MiniCPM-o-4.5, respectively.
System
Lang.
Agent
User
Interruption
Active
Proactive
Agent
Interruption
Interruption
Resistance
Silence
Initiation
Backchannel
P
I
N
N
P
IInt.
ISing.
N
P
I
P
N
P
I
N
Locally deployed systems
Freeze-Omni
en/zh
17.5
1.3
2.5
9.9
52.5
0.0
0.0
75.0
88.8
5.0
0.0
0.0
0.0
1.3
0.0
MiniCPM-o 4.5
en/zh
28.8
8.8
2.5
24.8
67.5
20.0
10.0
95.0
56.3
10.0
1.3
0.0
28.0
8.8
0.0
Moshi
en/zh
0.0
0.0
0.0
26.1
2.5
0.0
0.0
62.5
7.5
7.5
35.0
0.0
–
12.5
13.8
Table 2: Evaluation coverage and Behavioral Correctness Rate (BCR, %) on DuplexAct-Bench. Lang. denotes the evaluated languages (en: English; zh: Chinese). P/I/N denote Pre-session/In-session/No-explicit; IInt. and ISing. denote the In-session simultaneous-interpretation and collaborative-singing scenarios. Active Silence is scored by its silence criterion alone; User Interruption is evaluated on Valid trials only. “–” indicates no reported result, not zero. Bold marks the highest reported BCR in each column.
System
Agent
User
Interruption
Proactive
Agent
Interruption
Interruption
Resistance
Initiation
Backchannel
C↑
L (s) ↓
C↑
L (s) ↓
C↑
L (s) ↓
C↑
L (s) ↓
C↑
BAS ↑
Locally deployed systems
Freeze-Omni
1.66
9.67
1.56
21.22
2.69
15.98
0.70
14.03
1.09
0.005
MiniCPM-o 4.5
2.32
9.26
1.97
19.15
3.13
13.60
0.70
13.98
1.59
0.054
Moshi
1.15
10.41
1.68
17.76
1.47
16.65
2.01
11.01
1.24
0.108
Table 3: Content and timing results on DuplexAct-Bench. C denotes the mean Content score (0–5), L the mean latency in seconds, and BAS the Backchannel Alignment Score. For User Interruption, both metrics are computed on Valid trials only. Active Silence is omitted because it is evaluated solely by its silence criterion in Table 2 . Arrows indicate the preferred direction; bold marks the best reported value in each metric column.
Figure 4: Case studies of Agent Interruption (top) and Active Silence (bottom). Each pair compares systems on the same trial. The top pair illustrates differences in semantic task fulfillment despite both systems satisfying the Timing criterion; the bottom pair contrasts responding with correctly withholding speech.
Full-duplex spoken dialogue systems let voice agents listen and respond simultaneously, managing timing, overlap, interruptions, and conversational content.
End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simutanous listening and speaking continously, requiring models to take appropriate interaction actions as conversations unfold. However, existing benchmarks lack a structured action-level formulation, often evaluate action occurrence rather than success, and provide limited coverage of diverse real-world conversations. We introduce DuplexAct, a benchmark for evaluating interaction action taking in E2E FD-SLMs. DuplexAct defines a three-level action taxonomy spanning eight interaction actions and 24 corresponding interaction conditions. We further characterize successful action-taking through immediate execution and subsequent follow-through, and develop action-specific evaluation metrics for both. DuplexAct contains 4,691 test instances spanning casual conversation, meetings, customer service, and interviews, including 2,531 instances derived from real-world conversational recordings. Together, DuplexAct provides a structured and real-world-grounded framework for systematically assessing how E2E FD-SLMs participate in ongoing spoken interaction.
Jiaheng Dong, Shihong Tan, Gongping Huang +3
The University of Melbourne · University of Birmingham · Wuhan University +1
Real-time full-duplex speech models can listen while speaking, enabling natural interaction without rigid turn boundaries. Existing benchmarks evaluate turn-taking, interruption handling and multi-round dialogue, but largely centre on a designated user rather than an assistant participating in a shared conversation among several people. We introduce Duplex-MPE to evaluate when such an assistant should answer, remain silent or stop speaking. The benchmark contains 2,000 scenarios with three or four human speakers and one assistant, each paired across explicit and implicit addressing of the same request. Models receive continuous conversation audio without transcripts or supplied turn boundaries. Four scores measure fresh response initiation, answer accuracy, silence preservation and stopping when a human resolves a request. We evaluate five open-weight speech systems: MiniCPM-o 4.5, Moshi, FLM-Audio, Voila and Freeze-Omni. MiniCPM-o 4.5 leads on three scored capabilities, while frequent speech from other systems can coexist with inaccurate answers or failures to remain silent. A transcript-based Gemini 3.1 Pro reference responds 64.3 percentage points more often to explicit than implicit requests; paired tests detect no significant response-rate difference for the speech systems.
Chengqian Ma, Wenhao Feng, Weixuan Jin +7
Peking University · Renmin University of China · Tsinghua University +3
Full-duplex voice agents must continuously decide when to speak, listen, backchannel, interrupt, overlap, and yield the conversational floor. Existing benchmarks evaluate these behaviors through explicit turn-management instructions, whereas voice agents are often configured through roles or personas from which appropriate conversational behavior must be inferred. We introduce DuplexSpeechBench-IFEval (DSB-IFEval), a benchmark for evaluating implicit instruction following in real-time spoken interaction. DSB-IFEval comprises 1,038 test cases derived from 240 controlled conversations spanning eight behaviorally contrastive assistant roles and five conditioning protocols: default behavior, explicit behavioral instructions, persona-implied behavior, combined persona-rule conditioning, and instruction conflict. We measure real-time floor management using the deterministic Instruction Adherence Score (IAS) and persona-consistent response content using the LLM-judged Persona Adherence Score (PAS). Across eleven real-time speech models, we find that executing an explicit floor-management policy does not reliably imply the ability to infer the same policy from a persona. Moreover, even frontier models such as GPT-Live-1 and Gemini-3.8-Live adapt their dialogue language to the assigned persona without consistently translating that persona into the appropriate full-duplex floor-management behavior. Finally, models that successfully resolve benign instruction conflicts often fail when safety-relevant role behavior should override an explicit directive. These results show that inferring role-implied behavior, executing it in real time, and resolving instruction conflicts remain distinct challenges for full-duplex voice agents.