Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends

Seonghyeon Go; Yongwoo Kim; Hyeonjin Cha; Jaeho Shin

cs.CLSep 27, 2026

Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends

Authors: Seonghyeon Go, Yongwoo Kim, Hyeonjin Cha, Jaeho Shin

Organizations: Mindlogic

Abstract

Full-duplex spoken dialogue models can listen and speak simultaneously like the real-time dynamics of human conversation. For natural dialogue, the ability to search for external information in real-time is also an important capability. Many models remain trapped in parametric knowledge, leaving them unable to access real-time information and tool execution. Furthermore, even when Large Language Models (LLM) retrieve information, many duplex speech models process it within a compressed latent space rather than in its raw text form, which can lead to information loss from compression. To address this issue, we propose Context Spanning, a framework for information injection between a full-duplex speech model and an external LLM backend via real-time chunked prefill. The injected frame is encoded in a single forward pass inside the real-time frame budget. It feeds the retrieved information to the speech model as-is, enabling it to reason over the information independently and generate responses. With this approach, our model achieves high performance on Full-Duplex benchmarks and strong results on Question Answering tasks, demonstrating its conversation potential. Context Spanning shows that external information can be injected directly into a duplex speech model, introducing a new simple and powerful mechanism for duplex systems.

Figures & tables

Figure 1: Overall architecture of proposed method. Backend works asynchronously, and results are injected as Context Span.

Figure 2: Context Spanning on the Moshi architecture with an acoustic delay of $\tau=1$ . $u^{1}_{i}$ and $u^{q}_{i}$ denote user semantic and user acoustic tokens, $a^{1}_{i}$ and $a^{q}_{i}$ denote agent semantic and agent acoustic tokens, and $t_{i}$ denotes agent text tokens at timestep $i$ , where $q\in\{2,\dots,8\}$ .

$n$ (frames)	0	16	64	256	600
Latency ( ms )	$34.7\pm 0.4$	$56.1\pm 0.2$	$57.5\pm 0.2$	$62.0\pm 0.3$	$86.1\pm 0.3$
$P_{99}$ ( ms )	$35.2$	$56.3$	$57.7$	$62.3$	$86.3$

Table 1: Context Span processing latency.

P_{99}

denotes 99th percentile.

n=0

means no span, same as vanilla Moshi.

	Spoken QA								Mathematical Reasoning
	LlamaQ		WebQ		TriviaQA		HaluEval		AddSub		MultiArith		SinglEq		SVAMP		GSM8K
Model	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.	ref.	resp.
GLM-4-Voice		64.7		32.2		39.1		21.2		59.4		62.0		71.0		4.0		29.0
STITCH-S		73.3		50.2		50.0		–		81.7		87.9		91.7		72.2		56.7
MoshiRAG ${}_{\text{Gemma3}}$	83.0	80.3	71.5	67.2	73.7	69.6	42.0	36.3	76.6	61.7	87.1	69.0	83.2	68.2	74.1	55.0	66.2	33.9
MoshiRAG ${}_{\text{GPT-4.1}}$	87.8	80.6	77.7	68.9	86.8	78.2	61.2	51.3	87.9	64.8	87.1	76.0	89.6	72.9	80.5	61.1	70.8	43.2

Table 2: QA Benchmarks. ‘ref.’ denotes the accuracy % that correct reference document is provided; ‘resp.’ denotes the accuracy % that model response correctly. Underlined model metrics are reprinted from MoshiRAG, ‘-’ denotes values inaccessible.

Task	Metric	Ours	PPlex	Moshi	Gemini
Turn-Taking	TOR ( $\uparrow$ )	0.899	0.992	0.941	0.655
Turn-Taking	Latency, s ( $\downarrow$ )	0.064	0.070	0.265	1.301
Pause Handling	TOR, Candor ( $\downarrow$ )	0.824	0.662	0.980	0.310
Pause Handling	TOR, synthetic ( $\downarrow$ )	0.861	0.584	0.985	0.255
Backchannel	TOR ( $\downarrow$ )	0.782	0.327	1.000	0.091
	Frequency, per s ( $\uparrow$ )	0.150	0.025	0.001	0.012

Table 3: FullDuplexBench v1 results. Underlined models are reprinted from the PersonaPlex paper. PPlex denotes PersonaPlex, Gemini denotes Gemini Live 2.5.

Task	Metric	Ours	MoshiRAG	GPT	Gemini
Tool Use	Tool selection ( $\uparrow$ )	0.855	0.738	0.876	0.817
	Argument accuracy ( $\uparrow$ )	0.567	0.440	0.680	0.588
	Response quality ( $\uparrow$ )	0.411	0.255	0.792	0.718
	Pass rate ( $\uparrow$ )	0.470	0.280	0.600	0.540
Turn-Taking Dynamics	Take-turn rate, % ( $\uparrow$ )	95.0	94.0	96.0	78.0
	Latency, s ( $\downarrow$ )	5.83	7.69	6.89	4.25

Table 4: FullDuplexBench v3 results. Underlined systems are reprinted from the original benchmark paper. GPT denotes GPT-Realtime, Gemini denotes Gemini Live 3.1.

Explore similar work

How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue
May 11, 2026Hui Lu, Xueyuan Chen, Huimeng Wang +4Full-Duplex Speech Models Dialogue
The Chinese University of Hong Kong · Tsinghua University · SenseTime Research
BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
Jun 12, 2026Qingkai Fang, Shoutao Guo, Yang FengFull-Duplex Speech Models Speech Language Models
Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · Key Laboratory of AI Safety, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
A frontend-backend architecture for tool calls in full-duplex speech models
Sep 16, 2026Ke Hu, Slyne Deng, Chen Chen +10Full-Duplex Speech Models Call
NVIDIA, USA

May 11, 2026, cs.CLJ/K move · Enter open · S save

How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue

Hui Lu, Xueyuan Chen, Huimeng Wang, Shuhai Peng+3

The Chinese University of Hong Kong · Tsinghua University · SenseTime Research

Full-Duplex Speech Models Dialogue Question-Answer Pairs

Open paper AskPDF arXiv

Full-duplex spoken dialogue requires a model to keep listening while generating its own spoken response. This is challenging for large language models (LLMs), which are designed to extend a single coherent sequence and do not naturally support user input arriving during generation. We argue that how the user stream is routed into the LLM is therefore a key architectural question for full-duplex modeling. To study this question, we extend a text-only LLM into a unified full-duplex spoken dialogue system and compare two routing strategies under a shared training pipeline: (i) channel fusion, which injects the user stream directly into the LLM input, and (ii) cross-attention routing, which keeps the user stream as external memory accessed through cross-attention adapters. Experiments on spoken question answering and full-duplex interaction benchmarks reveal a clear tradeoff. Channel fusion yields stronger semantic grounding and consistently better question-answering performance. However, under semantically overlapping conditions such as user interruptions, it is more vulnerable to context corruption: if the model fails to stop in time, the overlapping user stream can interfere with ongoing generation and lead to semantically incoherent continuations. Cross-attention routing underperforms on question answering, but better preserves the LLM generation context and is more robust to this failure mode. These results establish user-stream routing as a central design axis in full-duplex spoken dialogue and offer practical guidance on the tradeoff between semantic integration and context robustness. We provide a demo page for qualitative inspection.

Abstract

Explore similar work

How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue

BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM

A frontend-backend architecture for tool calls in full-duplex speech models