DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion

Heeseung Kim

cs.SDOct 8, 2026

DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion

Authors: Heeseung Kim

Organizations: Department of Artificial Intelligence University of Seoul Seoul, Republic of Korea

Abstract

Recent full-duplex spoken dialog models enable simultaneous listening and speaking, but fine-grained models still advance their backbone autoregressively at every interaction frame. We introduce DiffuPlex, a rolling masked diffusion framework that reduces this sequential computation by predicting multiple future user and assistant frames in a single backbone wake. DiffuPlex consumes only a confident prefix of each predicted future while interaction continues at the original frame rate. As user speech arrives, it checks the corresponding user predictions and, when the interaction diverges, preserves already played assistant content while revising only the unplayed future. We consider two inference policies over the same predictor: DiffuPlex-LISTEN consumes multiple future frames when they predict assistant silence, whereas DiffuPlex-SPEAK can also consume predicted assistant speech. Across full-duplex interaction and spoken-language evaluations, DiffuPlex substantially reduces sequential backbone computation while largely preserving interaction behavior and general capability. DiffuPlex-LISTEN and DiffuPlex-SPEAK achieve $1.46\times$ and $1.59\times$ deployment-path wall-clock speedups and $1.61\times$ and $1.80\times$ Core LM speedups, with all measured backbone invocations completing within the 80ms interaction interval. Human evaluation shows that LISTEN preserves speech naturalness and conversational quality, while SPEAK retains conversational quality with some degradation in speech naturalness.

Figures & tables

Figure 1: Overview of conventional full-duplex generation and DiffuPlex. Conventional models advance one interaction frame at a time, requiring a new backbone computation at every step (left). DiffuPlex predicts multiple future frames in a single backbone wake, consumes a retained prefix, and revises only the unplayed future when new user input diverges from the prediction (right).

Figure 2: Overview of DiffuPlex. (a) One backbone wake predicts a masked future block with an assistant draft and user predictions. The staircase indicates stream-wise masking of inputs unavailable at that wake; the exact 17-stream layout is shown in Figure 4 in Appendix C.3 . (b) A confident prefix is played at the original 80ms rate without additional backbone wakes, while incoming user frames can trigger revision of only the unplayed future.

	Efficiency	Pause		Backchannel			Turn-taking		Interruption			Human
Model	$c\uparrow$	Syn. $\downarrow$	Can. $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$	NMOS $\uparrow$	DMOS $\uparrow$
Moshi	1.00	0.993	1.000	0.582	0.184	0.679	1.000	0.000	0.765	3.359	1.229	$3.52\pm 0.15$	$2.86\pm 0.20$
PersonaPlex	1.00	0.307	0.343	0.200	0.058	0.824	0.941	0.285	0.930	4.328	0.218	$3.90\pm 0.12$	$4.10\pm 0.13$
RL-Seamless	1.00	0.299	0.301	0.218	0.163	0.740	1.000	0.124	0.995	4.286	0.281	$3.93\pm 0.12$	$4.09\pm 0.14$
DiffuPlex- Listen	1.91	0.277	0.343	0.109	0.140	0.753	1.000	0.173	0.990	4.328	0.329	$3.92\pm 0.11$	$4.08\pm 0.13$
DiffuPlex- Speak	2.23	0.263	0.347	0.145	0.154	0.738	1.000	0.156	0.995	4.357	0.347	$3.54\pm 0.12$	$4.07\pm 0.13$

Table 1: Full-duplex interaction and human evaluation on Full-Duplex-Bench v1.0. RL-Seamless denotes personaplex-rl-seamless , and

c

is the average number of interaction frames advanced per backbone wake. Syn. and Can. denote the Synthetic and Candor pause sets. TOR denotes turn-over rate; Freq., backchannel frequency; JSD, Jensen–Shannon divergence of backchannel timing; Lat., response latency; and Judge, the gpt-5.6-luna response-appropriateness score for interruption responses (reasoning effort: none). NMOS and DMOS denote naturalness and dialog mean opinion scores, reported as mean

\pm

95% confidence interval.

	Wake	Streaming wall		E2E GPU		Core LM		Wake behavior
Model	$c\uparrow$	Time (s) $\downarrow$	Spd. $\uparrow$	Time (s) $\downarrow$	Spd. $\uparrow$	Time (s) $\downarrow$	Spd. $\uparrow$	Rel. time/wake	p99 (ms) $\downarrow$	$>80$ ms $\downarrow$
Moshi	1.00	638.7	1.168 $\times$	638.3	1.168 $\times$	550.1	1.194 $\times$	0.84 $\times$	–	–
PersonaPlex	1.00	744.1	1.002 $\times$	743.7	1.002 $\times$	656.8	1.000 $\times$	1.00 $\times$	–	–
RL-Seamless	1.00	745.7	1.000 $\times$	745.3	1.000 $\times$	656.8	1.000 $\times$	1.00 $\times$	46.85	0%
DiffuPlex- Listen	1.98	512.4	1.455 $\times$	511.1	1.458 $\times$	408.4	1.608 $\times$	1.23 $\times$	61.34	0%
DiffuPlex- Speak	2.26	469.1	1.590 $\times$	467.8	1.593 $\times$	365.5	1.797 $\times$	1.26 $\times$	61.41	0%

Table 2: Measured inference efficiency on the fixed 60-clip runtime subset. Streaming wall is the elapsed wall-clock time of the deployment-path streaming loop, including CPU/Python control, synchronization, and GPU execution. E2E GPU and Core LM denote accumulated GPU time for the full inference pipeline and the language-model backbone, respectively. Spd. denotes speedup relative to RL-Seamless, and Rel. time/wake is the Core LM time per backbone wake normalized by RL-Seamless. Wake-frame p99 is the 99th-percentile wall-clock latency of frames that invoke the backbone, and

>80

ms is the fraction of such frames exceeding the 80ms interaction interval.

Panel of Figure 3 — Figure 3: Efficiency-interaction trade-offs under deferred execution. Deferred baselines vary $R\in\{2,4,6,8\}$ , and deferred frames are forced to silence. Silero and Smart Turn replay precomputed controller decisions; controller inference cost is measured separately.

	VoiceBench		OpenAudioBench
Model	$s{\rightarrow}t$	$s{\rightarrow}s$	$s{\rightarrow}t$	$s{\rightarrow}s$
personaplex-rl-seamless	31.95	31.61	35.66	34.03
DiffuPlex- Listen	31.93	31.52	36.71	33.53
DiffuPlex- Speak	31.20	31.04	35.82	32.12

Table 3: General spoken-language capability. Higher is better.

Policy	Variant	$c\uparrow$	BC Freq.	Turn Lat. $\downarrow$	Int. Judge $\uparrow$	Int. Lat. $\downarrow$
Listen	DiffuPlex	1.91	.140	.173	4.328	.329
	w/o confidence	2.11	.191	.170	4.398	.314
	w/o revision	2.49	.078	.440	4.372	.744
Speak	DiffuPlex	2.23	.154	.156	4.357	.347
	w/o confidence	2.80	.191	.209	3.550	.281
	w/o revision	3.08	.080	.480	4.251	.515

Table 4: Ablation of confidence-based retention and draft revision on Full-Duplex-Bench v1.0.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Training sequence layout for masked future prediction. A clean and masked copy of the same 17-stream sequence are concatenated to form a sequence of length

2T

. The masked dialog region is tiled by variable-length blocks, with lookahead

h

reset at every block start. Inputs unavailable at that backbone wake are replaced by learned stream-specific MASK tokens: assistant streams from

h\geq 2

, user cb0 from

h\geq 3

, and user cb1–7 from

h\geq 4

. Native stream delays are unchanged. The staircase denotes stream-wise input availability rather than an attention mask; temporal attention remains causal.

Figure 5: NMOS evaluation interface. Evaluators rate the speech quality and naturalness of the assistant response following the user interruption on a five-point scale.

Figure 6: DMOS evaluation interface. Evaluators rate the complete interaction on a five-point scale while focusing on interruption handling and conversational flow.

Policy	Metric	Test	$n$	AR	DiffuPlex	$\Delta$	Raw $p$	Holm $p$
Listen	Pause C	McNemar	216	0.3009	0.3426	+0.0417	0.2221	1.0000
	Pause S	McNemar	137	0.2993	0.2774	-0.0219	0.6776	1.0000
	BC TOR	McNemar	55	0.2182	0.1091	-0.1091	0.2379	1.0000
	BC Freq	Wilcoxon	55	0.1634	0.1402	-0.0231	0.0266	0.5320
	BC JSD	Wilcoxon	55	0.7405	0.7534	+0.0130	0.0859	1.0000
	Turn TOR	McNemar †	119	1.0000	1.0000	0.0000	1.0000	1.0000

Appendix

Table 5: Paired Full-Duplex-Bench v1.0 comparisons against the autoregressive personaplex-rl-seamless backbone. AR and DiffuPlex values are computed over the paired clips used by each test and may therefore differ from the aggregate values in the main table.

\Delta

denotes DiffuPlex minus AR. Holm correction is applied jointly over all 20 comparisons.

\dagger

denotes a binary comparison for which the two systems have identical clip-level outcomes.

Scenario	Class / Metric	AR backbone	DiffuPlex- Listen	DiffuPlex- Speak
User interruption	Respond $\uparrow$	0.94	0.95	0.96
	Resume $\downarrow$	0.04	0.04	0.04
	Uncertain $\downarrow$	0.00	0.01	0.00
	Unknown $\downarrow$	0.02	0.00	0.00
	Stop (s) $\downarrow$	0.443	0.459	0.446
	Resp. (s) $\downarrow$	0.368	0.417	0.455

Appendix

Table 6: Full-Duplex-Bench v1.5 behavioral response distributions and interaction latencies. Stop and response latencies are reported in seconds.

Model	WPM	Pitch Mean	Pitch SD	Intensity Mean	Intensity SD	UTMOSv2
AR backbone	$141.05\rightarrow 220.05$ $(+79.00,\;p<.001)$	$238.45\rightarrow 236.77$ $(-1.68,\;p=.134)$	$46.24\rightarrow 46.30$ $(+0.06,\;p=.950)$	$59.10\rightarrow 60.48$ $(+1.38,\;p<.001)$	$17.93\rightarrow 15.98$ $(-1.95,\;p<.001)$	$2.829\rightarrow 2.858$ $(+0.029,\;p=.429)$
DiffuPlex- Listen	$140.79\rightarrow 219.90$ $(+79.11,\;p<.001)$	$238.32\rightarrow 234.53$ $(-3.79,\;p=.001)$	$44.40\rightarrow 42.58$ $(-1.82,\;p=.065)$	$58.64\rightarrow 60.25$ $(+1.61,\;p<.001)$	$18.58\rightarrow 16.22$ $(-2.37,\;p<.001)$	$2.697\rightarrow 2.812$ $(+0.115,\;p<.001)$
DiffuPlex- Speak	$141.76\rightarrow 220.04$ $(+78.28,\;p<.001)$	$239.46\rightarrow 236.36$ $(-3.09,\;p=.023)$	$47.57\rightarrow 46.64$ $(-0.93,\;p=.367)$	$58.02\rightarrow 60.22$ $(+2.21,\;p<.001)$	$18.21\rightarrow 16.08$ $(-2.13,\;p<.001)$	$2.549\rightarrow 2.635$ $(+0.086,\;p=.024)$

Appendix

Table 7: Speech-feature changes before and after user interruption for trials classified as Respond , following the Full-Duplex-Bench v1.5 evaluation protocol. Each cell reports pre-overlap

\rightarrow

post-overlap, followed by the post-minus-pre change and paired

t

-test

p

-value. The number of valid paired trials varies slightly across systems and features because Respond filtering and the benchmark’s MAD-based outlier removal (

k=3.5

) are applied separately to each system.

Model	Mode	AlpacaEval	CommonEval	WildVoice	SD-QA	OpenBookQA	MMSU	BBH	IFEval	AdvBench	Overall
AR backbone	$s\!\rightarrow\!t$	51.99	43.73	36.19	20.34	25.71	24.89	48.60	12.99	23.08	31.95
DiffuPlex- Listen	$s\!\rightarrow\!t$	50.91	45.00	36.82	22.42	25.05	25.05	49.30	12.00	20.77	31.93
DiffuPlex- Speak	$s\!\rightarrow\!t$	49.88	44.03	35.92	19.89	22.64	25.57	47.10	13.82	21.92	31.20
AR backbone	$s\!\rightarrow\!s$	52.05	44.43	36.17	17.81	24.62	24.11	47.80	13.08	24.42	31.61
DiffuPlex- Listen	$s\!\rightarrow\!s$	50.37	44.23	36.58	20.16	25.71	24.43	48.40	12.25	21.54	31.52
DiffuPlex- Speak	$s\!\rightarrow\!s$	49.15	43.33	35.75	17.18	24.40	24.95	48.40	13.28	22.88	31.04

Appendix

Table 8: Complete VoiceBench results. Values follow the benchmark’s official task-specific scoring and are reported on the common percentage scale used for aggregation. Overall is the aggregate reported in the main paper.

Model	Mode	Llama Questions $(0\!-\!100)$	Web Questions $(0\!-\!10)$	TriviaQA $(0\!-\!10)$	AlpacaEval $(1\!-\!10)$	Overall (%)
AR backbone	$s\!\rightarrow\!t$	54.20	2.95	1.92	3.97	35.66
DiffuPlex- Listen	$s\!\rightarrow\!t$	57.60	3.07	1.92	3.93	36.71
DiffuPlex- Speak	$s\!\rightarrow\!t$	55.53	3.07	1.86	3.85	35.82
AR backbone	$s\!\rightarrow\!s$	49.33	2.96	1.75	3.97	34.03
DiffuPlex- Listen	$s\!\rightarrow\!s$	46.73	3.02	1.80	3.92	33.53
DiffuPlex- Speak	$s\!\rightarrow\!s$	46.07	2.84	1.62	3.78	32.12

Appendix

Table 9: OpenAudioBench results. The four task columns retain their original score ranges. Overall first normalizes each task to a percentage scale and then averages across tasks.

System	$c_{60}$	$c_{727}$	Wall (s)	Wall speedup	E2E GPU (s)	E2E speedup	Core LM (s)	Core speedup
AR backbone	1.000	1.000	745.7	1.000 $\times$	745.3	1.000 $\times$	656.8	1.000 $\times$
PAD $R=2$	1.531	1.447	527.8	1.413 $\times$	526.9	1.415 $\times$	433.0	1.517 $\times$
PAD $R=4$	2.163	1.911	405.7	1.838 $\times$	404.5	1.842 $\times$	308.2	2.132 $\times$
PAD $R=6$	2.525	2.202	363.0	2.054 $\times$	361.9	2.060 $\times$	264.7	2.482 $\times$
PAD $R=8$	2.805	2.383	337.6	2.209 $\times$	336.4	2.216 $\times$	239.1	2.747 $\times$
Silero $R=2$	1.416	1.346	561.1	1.329 $\times$	560.2	1.330 $\times$	468.0	1.403 $\times$

Appendix

Table 10: Complete deferred-execution runtime results.

c_{60}

is measured on the fixed runtime subset and

c_{727}

over the complete Full-Duplex-Bench v1.0 evaluation. All speedups are relative to the autoregressive backbone. For Silero and Smart Turn , controller decisions are precomputed and replayed during the timed loop, so controller inference is excluded from these wall-clock measurements.

	Efficiency		Pause		Backchannel			Turn		Interruption
Variant	$c$	LM $\times$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
personaplex-rl-seamless (AR)	1.00	1.00	.299	.301	.218	.163	.740	1.000	.124	.995	4.286	.281
Default / Listen	1.91	1.56	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
Default / Speak	2.23	1.77	.263	.347	.145	.154	.738	1.000	.156	.995	4.357	.347
No gate / Listen	1.86	1.67	.292	.301	.091	.176	.729	.983	.202	1.000	4.340	.296
No gate / Speak	2.11	1.89	.263	.282	.109	.162	.738	.975	.253	1.000	4.265	.265

Appendix

Table 11: Training-objective and architecture ablations on Full-Duplex-Bench v1.0. The shaded rows report the autoregressive PersonaPlex and personaplex-rl-seamless references evaluated under the same protocol. The future gate leaves the

h=1

autoregressive path unchanged; removing it (No gate) allows the learned adapters to modify this path.

	Efficiency		Pause		Backchannel			Turn		Interruption
Variant	$c$	LM $\times$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
Default / Listen	1.91	1.56	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
Default / Speak	2.23	1.77	.263	.347	.145	.154	.738	1.000	.156	.995	4.357	.347
Train $W=8$ / Listen	1.86	1.58	.248	.366	.109	.146	.751	1.000	.242	.985	4.371	.344
Train $W=8$ / Speak	2.21	1.82	.255	.356	.291	.149	.740	.992	.202	.990	4.308	.315
Train $W=32$ / Listen	1.90	1.43	.277	.356	.200	.146	.749	1.000	.159	.980	4.327	.342

Appendix

Table 12: Model and training-scale ablations. Matched-

W

rows use the same horizon during training and inference. Increasing adapter capacity or training duration does not produce a monotonic change in the usable future horizon.

	Efficiency		Pause		Backchannel			Turn		Interruption
Train $\rightarrow$ infer	$c$	LM $\times$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
$8\rightarrow 16$ / Listen	1.89	1.55	.307	.347	.127	.162	.736	1.000	.200	1.000	4.300	.372
$8\rightarrow 16$ / Speak	2.25	1.79	.292	.333	.091	.146	.745	1.000	.182	.995	4.226	.429
$8\rightarrow 32$ / Listen	1.91	1.44	.299	.389	.073	.146	.745	1.000	.157	.995	4.332	.400
$8\rightarrow 32$ / Speak	2.27	1.66	.255	.384	.218	.153	.741	.992	.160	.985	4.284	.349
$16\rightarrow 8$ / Listen	1.86	1.58	.365	.394	.145	.155	.742	1.000	.162	.990	4.293	.335

Appendix

Table 13: Train-inference horizon mismatch on Full-Duplex-Bench v1.0. The retained advancement

c

changes little across most combinations even when the maximum proposed horizon changes substantially.

	Efficiency		Pause		Backchannel			Turn		Interruption
Setting	$c$	LM $\times$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
Listen , no confidence filtering	2.11	1.64	.197	.343	.164	.191	.724	.966	.170	.980	4.398	.314
Listen , $\tau_{\mathrm{conf}}=0.70$	1.99	1.62	.270	.319	.182	.140	.750	1.000	.191	.985	4.345	.342
Listen , $\tau_{\mathrm{conf}}=0.80$	1.91	1.56	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
Listen , $\tau_{\mathrm{conf}}=0.90$	1.80	1.48	.263	.366	.145	.148	.743	1.000	.089	.990	4.384	.485
Speak , no confidence filtering	2.80	2.07	.321	.347	.236	.191	.729	1.000	.209	1.000	3.550	.281

Appendix

Table 14: Confidence-threshold ablation. Lower thresholds retain longer predicted futures, whereas stricter thresholds wake the backbone more frequently. The no-confidence intervention disables only the confidence threshold for Listen (retaining the PAD and

m=6

constraints), whereas Speak retains the full proposed future when confidence filtering is disabled, since its original

m=2

minimum is non-binding.

	Efficiency		Pause		Backchannel			Turn		Interruption
Setting	$c$	LM $\times$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
Listen , $\tau_{\mathrm{rev}}=-1.0$	1.83	1.50	.321	.384	.164	.161	.735	1.000	.136	1.000	4.350	.437
Listen , $\tau_{\mathrm{rev}}=-0.5$	1.91	1.56	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
Listen , $\tau_{\mathrm{rev}}=0$	1.95	1.60	.321	.352	.036	.157	.739	.992	.125	.995	4.236	.446
Listen , no revision	2.49	2.07	.146	.176	.109	.078	.796	.966	.440	.995	4.372	.744
Listen , User-only (uncalibrated)	1.00	.79	.431	.417	.145	.145	.743	1.000	.089	1.000	4.335	.307

Appendix

Table 15: Revision-rule ablation. A larger

\tau_{\mathrm{rev}}

requires stronger evidence before revising, while the no-revision variant always continues through the retained draft unless it is exhausted. The uncalibrated User-only rows remove the assistant-side contribution while retaining the default revision threshold, and should not be interpreted as the calibrated User-only ablation in Appendix H.7 .

	Efficiency	Pause		Backchannel			Turn		Interruption
Setting	$c$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
Causal / Listen	1.910	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
Bidirectional / Listen	1.884	.299	.343	.109	.148	.752	1.000	.156	.990	4.374	.432
Causal / Speak	2.230	.263	.347	.145	.154	.738	1.000	.156	.995	4.357	.347
Bidirectional / Speak	2.223	.307	.324	.127	.143	.748	.983	.169	.985	4.284	.327

Appendix

Table 16: Causal and bidirectional future attention on Full-Duplex-Bench v1.0. The two attention variants advance a similar number of frames per wake. Bidirectional attention changes individual interaction metrics in both directions rather than providing a consistent advantage.

	Revision		Pause		Backchannel			Turn		Interruption
Setting	$\tau_{\rm rev}$	$c$	S $\downarrow$	C $\downarrow$	TOR $\downarrow$	Freq.	JSD $\downarrow$	TOR $\uparrow$	Lat. $\downarrow$	TOR $\uparrow$	Judge $\uparrow$	Lat. $\downarrow$
Full / Listen	-0.5	1.910	.277	.343	.109	.140	.753	1.000	.173	.990	4.328	.329
User-only / Listen	5.5	1.920	.182	.264	.109	.117	.769	.992	.251	.990	4.354	.315
Assistant-only / Listen	-3.5	2.122	.270	.296	.127	.162	.729	1.000	.125	.980	4.270	.457
Full / Speak	-0.5	2.230	.263	.347	.145	.154	.738	1.000	.156	.995	4.357	.347
User-only / Speak	4.5	2.096	.197	.255	.018	.106	.781	.983	.301	.985	4.355	.312

Appendix

Table 17: Revision-signal decomposition on the complete Full-Duplex-Bench v1.0 test set. Full uses both incoming user surprisal and assistant uncertainty. User-only retains only

-\log q_{h}(x_{h})

, while Assistant-only retains only the assistant-side alternative-odds term. The realized test-set advancement

c

is reported because the held-out calibration does not remain exactly matched on the final evaluation set.

Panel of Figure 7 — Figure 7: Rolling-draft diagnostics. Revision-triggered wakes typically occur after several frames have already been consumed, whereas exhaustion is dominated by one-frame advances. State-level no-op revisions compare the next $K=4$ fully revisable frames. Positive NLL reduction indicates improved future acoustic prediction.

Figure 8: Long draft consumption during a stable silent interval. After several short backbone wakes, DiffuPlex- Speak consumes two consecutive 16-frame drafts without revision. The number inside each backbone pass indicates the number of frames realized before the next wake.

Figure 9: Revision after partial draft consumption. For DiffuPlex- Speak , the highlighted backbone pass drafts 12 frames of assistant silence. Revision fires after three frames have been consumed, discarding the remaining nine unplayed frames before a new assistant continuation is generated. Hatched regions denote drafted frames that are revoked before playback.

Figure 10: Different retention policies on the same interaction. Because the prediction at

h=2

is lexical, DiffuPlex- Listen retains only the autoregressive frame (

r=a=1

), whereas DiffuPlex- Speak retains

r=16

and consumes

a=9

before revision.

Figure 11: Repeated one-frame retention during a stable user context. Although the user remains silent, DiffuPlex- Speak performs 47 consecutive backbone passes that each advance only one frame. This example shows that limited acceleration can arise because the retention rule repeatedly admits only the immediate next frame, rather than because long drafts are repeatedly consumed and then revoked.

Artifact	License	URL
Backbone and Model Lineage
personaplex-rl-seamless	CC BY-NC 4.0 + NVIDIA Open Model License	https://huggingface.co/kyutai/personaplex-rl-seamless
PersonaPlex	NVIDIA Open Model License	https://huggingface.co/nvidia/personaplex-7b-v1
Moshi / Mimi	CC BY 4.0	https://huggingface.co/kyutai/moshiko-pytorch-bf16
Training Data
Seamless Interaction	CC BY-NC 4.0	https://huggingface.co/datasets/facebook/seamless-interaction

Appendix

Table 18: All external artifacts used in this work with their licenses and sources.

Explore similar work

AdaptDuplex: from static to adaptive full-duplex spoken dialogue
Sep 24, 2026Zhiyang Zhou, Yingxin Shang, Zhou Wang +9Spoken Dialogue Systems Speech Language Models
Zuoyebang Education Technology
HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models
Oct 6, 2026Kyudan Jung, Hyunsin Park, Yoonhyung Lee +5RL for Language Models Speech Language Models
Qualcomm AI Research · KAIST AI
BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
Jun 12, 2026Qingkai Fang, Shoutao Guo, Yang FengSpoken Dialogue Systems Speech Language Models
Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · Key Laboratory of AI Safety, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

Sep 24, 2026, cs.SDJ/K move · Enter open · S save

AdaptDuplex: from static to adaptive full-duplex spoken dialogue

Zhiyang Zhou, Yingxin Shang, Zhou Wang, Hongwei Cai+8

Zuoyebang Education Technology

Spoken Dialogue Systems Speech Language Models Full-Duplex Spoken Dialogue Systems Turn-Taking in Dialogue

Open paper AskPDF arXiv

Full-duplex spoken dialogue requires simultaneous listening and speaking at sub-second latency, under conversational timing and cognitive demands that change moment to moment. Yet current models mostly impose static operating points, lacking a systematic mechanism for adaptive decisions. We present AdaptDuplex, which extends Qwen3-Omni with such a mechanism, co-designed across three layers. A compact token-level protocol represents every window as a canonical sequence, trains dual-stream alignment through a bounded text lead over speech, and exposes every behavioral decision as an explicit token for training-free runtime control via logits bias. Adaptive mechanisms dynamically predict among discrete window durations and augment direct response as needed with non-blocking cognitive consolidation and multi-flight external reasoning. A progressive pipeline introduces these behaviors through a three-stage Thinker curriculum, then Talker-only and joint SFT, with GRPO as a preliminary increment. On Full-Duplex-Bench v1 and v1.5, AdaptDuplex outperforms DuplexOmni on 18 of 21 comparable turn-taking, overlap-behavior, and timing metrics and MiniCPM-o 4.5 on 19 of 22, with gains in both interaction decisions and response timing. On human-recorded HumDial-FDBench, it attains the top Final score (69.6) of the compared duplex models.

Abstract

Explore similar work

AdaptDuplex: from static to adaptive full-duplex spoken dialogue

HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models

BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM