Organizations: College of Computing and Data Science, Nanyang Technological University · Mila, Quebec Artificial Intelligence Institute, Université de Montréal
During conversational interactions, humans subconsciously engage in concurrent thinking while listening to a speaker. Although this internal cognitive processing may not always manifest as explicit linguistic structures, it is instrumental in formulating high-quality responses. Inspired by this cognitive phenomenon, we propose a novel Full-duplex LAtent and Internal Reasoning method named FLAIR that conducts latent thinking simultaneously with speech perception. Unlike conventional "thinking" mechanisms in NLP, which require post-hoc generation, our approach aligns seamlessly with spoken dialogue systems: during the user's speaking phase, it recursively feeds the latent embedding output from the previous step into the next step, enabling continuous reasoning that strictly adheres to causality without introducing additional latency. To enable this latent reasoning, we design an Evidence Lower Bound-based objective that supports efficient supervised finetuning via teacher forcing, circumventing the need for explicit reasoning annotations. Experiments demonstrate the effectiveness of this think-while-listening design, which achieves competitive results on a range of speech benchmarks. Furthermore, FLAIR robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.
Figures & tables
Figure 1: (a) Turn-based interaction. The agent remains idle and starts to respond after the end of the user turn, but cannot be interrupted by user, as shown in the red box. (b) Full-duplex SDLM continuously listens to streaming speech input, supports user barge-in, and automatically switches between thinking and speaking like a human speaker.
Figure 2: The overview of proposed FLAIR. During the user’s speech phase, the LLM performs latent reasoning, using the LLM’s output latent embeddings as the input for the next step. Once the user finishes speaking, the assistant autonomously decides when to respond; the LLM then executes an explicit forward pass, using text tokens as the input for the next step. When the user barges in, the LLM autonomously decides when to stop speaking and reverts to a state of latent reasoning.
Method
FD
LlamaQ
WebQ
TriQA
SDQA
AlpacaE
ComE
OBQA
MMSU
Moshi ( Défossez et al., 2024 )
✓
54.5
22.1
16.7
15.6
2.01
1.60
25.9
24.0
Freeze-Omni ( Wang et al., 2024b )
✓
56.2
27.9
28.5
53.5
4.03
3.46
31.0
28.1
SALMONN-omni ( Yu et al., 2025 )
✓
73.6
43.7
56.0
-
3.22
-
-
30.0
SALM-Duplex ( Hu et al., 2025 )
✓
51.3
25.0
16.9
26.0
2.99
2.50
39.6
26.3
GLM-4-Voice ( Zeng et al., 2024 )
✗
65.7
37.0
47.5
37.0
3.97
3.42
53.4
39.8
Qwen2-Audio ( Chu et al., 2024 )
✗
69.7
45.2
40.3
35.7
3.74
3.43
49.5
35.7
Table 1: Accuracy (%) or GPT-Score of different methods on QA benchmarks. Results of baseline systems are taken from ( Yu et al., 2025 ; Chen et al., 2024 ) . We use “ thk ” to denote the proposed latent reasoning. “FD”, ”LlamaQ”, “WebQ”, “TriQA”, “OBQA”, “AlpacaE” and “ComE” stand for “if Full-duplex”, Llama Questions, WebQuestions, TriviaQA, OpenbookQA and AlpacaEval, CommonEval respectively.
Method
E2E
Turn Taking
Barge-in
MOS
Latency ( ↓ )
Latency ( ↓ )
Success rate ( ↑ )
Freeze-Omni ( Wang et al., 2024b )
✗
1.17
1.20
79.5
4.3
dGSLM ( Nguyen et al., 2023 )
✓
0.57
0.86
85.0
2.2
Moshi ( Défossez et al., 2024 )
✓
-
0.81
55.1
3.9
ORISE ( Chen et al., 2025 )
✓
0.43
0.61
96.8
4.2
FLAIR w/o thk
✓
0.33
0.49
100
4.3
Table 2: Evaluation of conversational behaviors of different methods on the Impatient dataset proposed in ( Chen et al., 2025 ) , in terms of turn-taking latency (s), barge-in latency (s), and barge-in success rate (%). We use “ thk ” to denote the proposed latent reasoning.
Method
Turn-taking
Barge-in
TOR ( ↑ )
Latency ( ↓ )
TOR ( ↑ )
GPT-4o ( ↑ )
Latency ( ↓ )
Freeze-Omni ( Wang et al., 2024b )
33.6
0.95
86.7
3.62
1.41
dGSLM ( Nguyen et al., 2023 )
97.5
0.35
91.7
0.20
2.53
Moshi ( Défossez et al., 2024 )
94.1
0.27
100
0.77
0.26
Gemini Live 1
65.5
1.30
89.1
3.38
1.18
FLAIR w/o thk
94.1
0.37
89.0
4.08
0.35
Table 3: Turn-taking and barge-in performance on Full-Duplex-Bench. Results of baseline systems are taken from ( Lin et al., 2025 ) . We use “thk” to denote the proposed latent reasoning.
Figure 3: The distribution of the input audio, target text, and latent reasoning embeddings. Specifically, the latent reasoning embeddings act as a bridge that connects the input audio with the target text.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
LlamaQ
WebQ
OBQA
MMSU
PT Model
63.6
40.4
61.3
49.5
Appendix
Table 4: Accuracy (%) of Pretraining model on QA benchmarks. “LlamaQ”, “WebQ”, and “OBQA” stand for Llama Questions, WebQuestions, and OpenbookQA respectively.
User query
Can you tell me a very easy way to clean a shower head?
Response by PT model
A easy and effective method you can try at home is using common household items.
Response by FLAIR
Sure! A simple way to clean a shower head is to remove it and soak it in white vinegar for about an hour. This helps dissolve any mineral deposits or buildup. After soaking, rinse it thoroughly with water.
Response by LLM
Certainly! Cleaning a shower head is a straightforward process that can help improve water flow and remove mineral buildup. Here’s a simple method:\n\nMaterials Needed:\nWhite vinegar\nPlastic bag\nRubber bands or clips\nTowel\n\nSteps:\nRemove the Shower Head:\n\nTurn off the water supply to the shower.\nUnscrew the shower head from the pipe. You may need a wrench or pliers for this step.\n\nSoak the Shower Head:\n\nPlace the shower head in a plastic bag.\nPour enough white vinegar into the bag to cover the shower head.\nSecure the bag around the shower head with rubber bands or clips to keep the vinegar in contact with the shower head.\n\nLet It Soak:\n\nLeave the shower head in the vinegar for at least 8 hours, or overnight for best results. The longer it soaks, the more effectively it will remove mineral buildup.\n\nRinse and Reattach:\n\nAfter the soaking period, remove the shower head from the bag.\nRinse the shower head thoroughly with water to remove any remaining vinegar.\nReattach the shower head to the pipe and turn the water supply back on.\n\nTest the Flow:\n\nTurn on the shower and check the water flow. You should notice an improvement in the water pressure and clarity.\n\nThis method is effective and easy to do, making it a great way to keep your shower head clean and functioning well.
Appendix
Table 5: Case study for PT model, FLAIR, and LLM (Qwen2.5-7B-Ins)
While Chain-of-Thought (CoT) reasoning has improved the capability of language models, directly applying it to Spoken Language Models (SLMs) may introduce long silent intervals under the serial "think-then-speak" paradigm, disrupting real-time spoken interaction. To address this issue, we propose an asynchronous think-aloud framework for reasoning-based SLMs within the Thinker-Talker architecture. The framework maintains a primary reasoning stream for logical deduction and a lightweight think-aloud stream that generates short, task-grounded progress utterances conditioned on the user input and the evolving reasoning state. A dynamic balance strategy coordinates the two streams at runtime, triggering additional think-aloud speech to avoid silent gaps and canceling pending utterances when the final response becomes ready. Experiments on spoken reasoning and question-answering benchmarks show that our approach substantially reduces user-audible silence during reasoning while maintaining answer accuracy comparable to that of a serial "think-then-speak" baseline, demonstrating the potential of asynchronous think-aloud for responsive interaction in SLMs.
Junyi Ao, Kainan Peng, Mingbo Ma +10
Meta Superintelligence Labs · The Chinese University of Hong Kong, Shenzhen · Work done at Meta
The thinking-while-speaking paradigm aims to make AI communication more human. A key challenge is maintaining fluent speech while performing deep reasoning. Our method, InterRS, tackles this by inserting reasoning steps only during natural speech generation. This requires high-quality data where reasoning and speech are precisely aligned, and the length ratio are under controlled. We introduce a novel pipeline to generate such seamlessly interleaved audio data. To train our model, we combine interleaved SFT with refined data and reinforcement learning with two new rewards: a TA-Balance Reward to manage timing and thinking-answer ratio, and a Linguistic Quality Reward to refine expression. Experiments show our approach achieves 13% better performance on mathmatical and logic benchmarks while generating instant response like a spoken-language instruct model which outputs fast CoT response. Furthermore, our method generates more natural and fluent answers than prior methods.
Empathetic spoken dialogue requires models to use both what is said and how it is said to decide how to respond. Explicit CoT can improve paralinguistic perception and make acoustic cues more explicit in replies, yet does not ensure their effective use in response planning. We call this mismatch the perception-reasoning gap. In addition, CoT may not fully capture acoustic cues in words, and generating it adds inference latency. To address these limitations, we introduce LoopSLM, which builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass. Its two-stage training further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT. On EchoMind, LoopSLM improves paralinguistic understanding, reasoning, and reply quality over Qwen2.5-Omni-7B. Against the CoT-SFT baseline, LoopSLM gains over 20 points in reasoning accuracy while generating 64.5% fewer tokens at half the latency. It also outperforms Qwen3-Omni-Thinking on most empathetic reply metrics with 34x lower latency. Despite training only on dialogue data, LoopSLM improves accuracy on general audio benchmarks.
Shengbo Cai, Yuxiang Wang, Jingran Xie +5
Tsinghua University · Tencent Hunyuan · The Chinese University of Hong Kong, Shenzhen