Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models
Organizations: The Chinese University of Hong Kong, Shenzhen, China · Amphion Technology Co., Ltd.
Abstract
Chunk-interleaved full-duplex speech language models listen and speak in fixed time chunks, and each chunk must be processed before the next one arrives. Their workload is asymmetric across chunks: listen chunks leave compute idle while awaiting audio, and speak and tool-call chunks require serial generation, which delays responses and tool calls. Using the idle compute for the next chunk requires decoding it before its audio arrives, under an assumed input. Taking silence as the assumed input and comparing the output with the output under real audio on four models, we observe two properties. First, the next audio chunk does not change the output in 95.3% of chunks. Second, 88% of changes begin at the first output token. Building on these properties, we propose Speculative Listening, a training-free, lossless framework for input-time self-speculative decoding. During idle time, the model drafts the next chunk under silence and, for changes at the first token, a second branch from the runner-up first token. It keeps all drafts in a cross-chunk hypothesis pool and verifies them when the audio arrives. On our eight-task benchmark TickTock-Bench, Speculative Listening speeds up speak chunks by 2.6-2.8x and tool-call chunks by up to 23.4x. On average, it reduces the decoding delay before a response starts by 73%, to 74-241 ms, close to the 200 ms gap between human turns, and the maximum lag behind real time by 94.4%.
Figures & tables
| Category | Task | Focus | #Sess. | #Chunks |
| Speech reasoning | Web Q. | factual QA | 100 | 1,345 |
| Llama Q. | factual QA | 100 | 1,423 | |
| TriviaQA | factual QA | 100 | 1,671 | |
| AlpacaEval | open-ended answers | 100 | 1,598 | |
| BBA | multi-step reasoning | 60 | 1,825 | |
| Conv. dynamics | FDB v1 | turn-taking behaviors | 100 | 1,885 |
| Model | Method | Speech Reasoning | Conv. Dynamics | Agentic | #MAT | Avg. | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Web Q. | Llama Q. | TriviaQA | AlpacaEval | BBA | FDB v1 | FDB v1.5 | FDB v3 † | ||||
| Streaming SLM | PLD | 1.01 | 1.01 | 1.02 | 1.01 | 1.03 | 1.03 | 1.02 | 1.01 | 1.01 | 1.02 |
| Token Recycling | 1.01 | 1.00 | 1.02 | 1.01 | 1.03 | 1.03 | 1.02 | 1.02 | 1.01 | 1.02 | |
| Lookahead | 0.98 | 0.98 | 0.99 | 0.97 | 0.99 | 0.98 | 1.00 | 1.07 | 1.03 | 0.99 | |
| Speculative Listening | 2.80 | 2.87 | 3.04 | 2.75 | 2.67 | 2.62 | 2.57 | 23.41 | 2.63 | 2.76 | |
| Gander | PLD | 1.04 | 1.02 | 1.01 | 1.07 | 1.03 | 1.07 | 1.06 | 1.07 | 1.21 | 1.04 |
| Model | AR | Speculative Listening | ||||
|---|---|---|---|---|---|---|
| Mean | Max | Lagging | Mean | Max | Lagging | |
| Streaming SLM | 568 | 2619 | 35 | 0 | 0 | 0 |
| Gander | 444 | 1227 | 9 | 42 | 675 | 0 |
| Realtime-Venus | 1 | 108 | 0 | 0 | 0 | 0 |
| LWS | 430 | 3966 | 20 | 55 | 2168 | 1 |
| Streaming SLM | Gander | |||
|---|---|---|---|---|
| Method | Speak | Tool | Speak | Tool |
| AR | 1.00 | 1.00 | 1.00 | 1.00 |
| + silence-hypothesis draft | 2.88 | 1.42 | 2.64 | 1.21 |
| + hypothesis pool | 2.87 | 22.04 | 2.97 | 2.04 |
| + runner-up branch | 2.87 | 23.41 | 3.00 | 3.45 |
| Model | AR | Speculative Listening | ||
|---|---|---|---|---|
| Onset | Tool | Onset | Tool | |
| Streaming SLM | 417 | 2623 | 74 | 112 |
| Gander | 763 | 1827 | 228 | 513 |
| Realtime-Venus | 565 | – | 241 | – |
| LWS | 1089 | – | 204 | – |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | AR | Speculative Listening | |
|---|---|---|---|
| Streaming SLM | 3.10 | 3.13 | 0.03 |
| Gander | 3.56 | 3.63 | 0.07 |
| Realtime-Venus | 2.98 | 3.00 | 0.02 |
| LWS | 3.02 | 3.07 | 0.05 |
| Model | All | Listen | Speak | Tool call | User speech | First token |
|---|---|---|---|---|---|---|
| Streaming SLM | 98.3 | 97.6 | 99.8 | 80.0 | 94.2 | 95.8 |
| Gander | 99.5 | 99.6 | 98.6 | 34.0 | 99.5 | 85.7 |
| Realtime-Venus | 83.3 | 81.4 | 90.0 | – | 37.9 | 86.9 |
| LWS | 99.9 | 99.8 | 100.0 | – | 100.0 | 100.0 |
| Model | Web Q. | Llama Q. | TriviaQA | AlpacaEval | BBA | FDB v1 | FDB v1.5 | FDB v3 | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Streaming SLM | 3.15 | 2.87 | 2.77 | 3.28 | 2.03 | 2.11 | 3.06 | 1.78 | 2.63 |
| Gander | 2.33 | 2.22 | 2.18 | 2.40 | 2.01 | 2.10 | 2.44 | 1.96 | 2.20 |
| Realtime-Venus | 2.40 | 2.31 | 2.21 | 2.31 | 1.73 | 1.95 | 2.45 | 1.82 | 2.15 |
| LWS | 2.63 | 3.16 | 2.83 | 2.40 | 2.82 | 2.16 | 2.62 | 1.58 | 2.52 |
Speech Language Models
Speech language models connect spoken audio with large language models for understanding, transcription, translation, and real-time dialogue.
See all 327 papers