Chunk-interleaved full-duplex speech language models listen and speak in fixed time chunks, and each chunk must be processed before the next one arrives. Their workload is asymmetric across chunks: listen chunks leave compute idle while awaiting audio, and speak and tool-call chunks require serial generation, which delays responses and tool calls. Using the idle compute for the next chunk requires decoding it before its audio arrives, under an assumed input. Taking silence as the assumed input and comparing the output with the output under real audio on four models, we observe two properties. First, the next audio chunk does not change the output in 95.3% of chunks. Second, 88% of changes begin at the first output token. Building on these properties, we propose Speculative Listening, a training-free, lossless framework for input-time self-speculative decoding. During idle time, the model drafts the next chunk under silence and, for changes at the first token, a second branch from the runner-up first token. It keeps all drafts in a cross-chunk hypothesis pool and verifies them when the audio arrives. On our eight-task benchmark TickTock-Bench, Speculative Listening speeds up speak chunks by 2.6-2.8x and tool-call chunks by up to 23.4x. On average, it reduces the decoding delay before a response starts by 73%, to 74-241 ms, close to the 200 ms gap between human turns, and the maximum lag behind real time by 94.4%.
Figures & tables
Figure 1: Decoding in a chunk-interleaved full-duplex speech language model. (a) Autoregressive decoding leaves compute idle while it waits for audio, and a long output then misses the chunk deadline. (b) Speculative Listening drafts the next chunk under silence during the idle window and verifies the whole draft in one forward pass when the audio arrives.
Figure 2: Overview of Speculative Listening. Observation 1 motivates drafting under silence and verifying the draft in the audio prefill pass. Observation 2 motivates a runner-up branch at the first token and a hypothesis pool for later forward passes to continue from.
Figure 3: Chunk profile and oracle probe. (a) Mean post-audio latency by chunk type under AR; dashed line: 1 s deadline. (b) Chunk-level acceptance rate of the silence-hypothesis draft over all chunks, speak chunks, tool-call chunks, and chunks with user speech. (c) Share of mismatched chunks that diverge at the first output token. Tool-call chunks and the bars of Streaming SLM, Gander, and Realtime-Venus in (a) use FDB v3; all other bars use Llama Questions. Realtime-Venus and LWS produce no tool-call chunks.
Category
Task
Focus
#Sess.
#Chunks
Speech reasoning
Web Q.
factual QA
100
1,345
Llama Q.
factual QA
100
1,423
TriviaQA
factual QA
100
1,671
AlpacaEval
open-ended answers
100
1,598
BBA
multi-step reasoning
60
1,825
Conv. dynamics
FDB v1
turn-taking behaviors
100
1,885
Table 1: Composition of TickTock-Bench. #Sess.: number of sessions. #Chunks: number of 1 s chunks, fixed by the input audio.
Model
Method
Speech Reasoning
Conv. Dynamics
Agentic
#MAT
Avg.
Web Q.
Llama Q.
TriviaQA
AlpacaEval
BBA
FDB v1
FDB v1.5
FDB v3 †
Streaming SLM
PLD
1.01
1.01
1.02
1.01
1.03
1.03
1.02
1.01
1.01
1.02
Token Recycling
1.01
1.00
1.02
1.01
1.03
1.03
1.02
1.02
1.01
1.02
Lookahead
0.98
0.98
0.99
0.97
0.99
0.98
1.00
1.07
1.03
0.99
Speculative Listening
2.80
2.87
3.04
2.75
2.67
2.62
2.57
23.41
2.63
2.76
Gander
PLD
1.04
1.02
1.01
1.07
1.03
1.07
1.06
1.07
1.21
1.04
Table 2: Main results on TickTock-Bench. Speedup ( × ) over autoregressive decoding (AR) in mean post-audio latency of speak chunks across four chunk-interleaved full-duplex models (1 s chunk duration, greedy decoding, batch size 1). † FDB v3 reports tool-call chunks for Streaming SLM and Gander, and speak chunks otherwise. #MAT denotes the number of output tokens per target forward pass over all chunks relative to AR ( =1.00 ), averaged over tasks (per-task values in Section B.3 ); Avg. is the mean speedup over the seven speech-reasoning and conversational-dynamics tasks.
Model
AR
Speculative Listening
Mean
Max
Lagging
Mean
Max
Lagging
Streaming SLM
568
2619
35
0
0
0
Gander
444
1227
9
42
675
0
Realtime-Venus
1
108
0
0
0
0
LWS
430
3966
20
55
2168
1
Table 3: Maximum lag behind the incoming audio per session . Mean and Max: mean and maximum over sessions (ms). Lagging: share of sessions whose maximum lag exceeds 1 s (%). FDB v3.
Figure 4: In-depth analysis on Streaming SLM (FDB v3). (a) Post-audio forward passes per chunk by output length; the shaded bin (9–32 tokens) contains no chunks. (b) Speedup over AR under speculative sampling at temperature T . (c) Speedup over AR with N concurrent sessions on one A100 GPU, in wall-clock time.
Streaming SLM
Gander
Method
Speak
Tool
Speak
Tool
AR
1.00
1.00
1.00
1.00
+ silence-hypothesis draft
2.88
1.42
2.64
1.21
+ hypothesis pool
2.87
22.04
2.97
2.04
+ runner-up branch
2.87
23.41
3.00
3.45
Table 4: Ablation study. Speedup ( × ) over AR as the components of Speculative Listening are added. Speak: Llama Questions; tool: FDB v3. Best value per column in bold.
Figure 5: Case study of user-perceived latency. SL: Speculative Listening. (a) Response onset. Under AR, speech starts after the last token of the chunk is decoded. With Speculative Listening, the draft made under silence is verified when the audio arrives, and speech starts earlier. (b) Tool call. Under AR, a long tool call runs past the deadline and delays the next chunk. With Speculative Listening, the call drafted in earlier chunks is verified from the hypothesis pool and is ready within the chunk.
Model
AR
Speculative Listening
Onset
Tool
Onset
Tool
Streaming SLM
417
2623
74
112
Gander
763
1827
228
513
Realtime-Venus
565
–
241
–
LWS
1089
–
204
–
Table 5: User-perceived latency (ms). Onset: response onset latency, averaged over the eight tasks. Tool: tool-call latency, on FDB v3. Best value per model in bold.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Model
AR
Speculative Listening
Δ
Streaming SLM
3.10
3.13
+ 0.03
Gander
3.56
3.63
+ 0.07
Realtime-Venus
2.98
3.00
+ 0.02
LWS
3.02
3.07
+ 0.05
Appendix
Table 6: Response quality on AlpacaEval. Judge score (1–5) of AR and Speculative Listening, and their difference Δ .
Model
All
Listen
Speak
Tool call
User speech
First token
Streaming SLM
98.3
97.6
99.8
80.0
94.2
95.8
Gander
99.5
99.6
98.6
34.0
99.5
85.7
Realtime-Venus
83.3
81.4
90.0
–
37.9
86.9
LWS
99.9
99.8
100.0
–
100.0
100.0
Appendix
Table 7: Chunk-level acceptance rate (%) of the silence-hypothesis draft by chunk type. First token: share (%) of mismatched chunks that diverge at the first output token. Llama Questions, 1,423 chunks per model; tool-call chunks on FDB v3. LWS has a single mismatched chunk.
Model
Web Q.
Llama Q.
TriviaQA
AlpacaEval
BBA
FDB v1
FDB v1.5
FDB v3
Avg.
Streaming SLM
3.15
2.87
2.77
3.28
2.03
2.11
3.06
1.78
2.63
Gander
2.33
2.22
2.18
2.40
2.01
2.10
2.44
1.96
2.20
Realtime-Venus
2.40
2.31
2.21
2.31
1.73
1.95
2.45
1.82
2.15
LWS
2.63
3.16
2.83
2.40
2.82
2.16
2.62
1.58
2.52
Appendix
Table 8: #MAT of Speculative Listening across benchmarks (output tokens per target forward pass over all chunks of a task, normalized to 1.00 for AR).
Figure 6: Case study on two illustrative sessions. SL: Speculative Listening. Span labels give typical measured latencies ( Table 5 ), and green spans mark the time saved by Speculative Listening. Check marks denote drafted tokens accepted by verification. (a) Response onset. The first speak chunk is drafted under silence in chunk 2 and verified when its audio arrives. (b) Tool call. R: tool result. Drafts made while the user speaks contain the tool call and enter the hypothesis pool. Under AR, the call misses the deadline, and the dashed arrow links the audio of chunk 5 to the time AR starts processing it.