AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models

Yuxiang Wang; Kunyu Feng; Yuancheng Wang; Zihang Liu; Shengbo Cai; Qinke Ni; Wan Lin; Tao Feng

cs.CLOct 1, 2026

AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models

Authors: Yuxiang Wang, Kunyu Feng, Yuancheng Wang, Zihang Liu, Shengbo Cai, Qinke Ni, Wan Lin, Tao Feng, +7 more

Organizations: The Chinese University of Hong Kong, Shenzhen · Tencent Hunyuan · Tsinghua University · The Hong Kong University of Science and Technology · Amphion Technology Co., Ltd.

Abstract

Model intelligence and fast response jointly shape the quality of interaction with speech language models, yet remain difficult to achieve together. Explicit chain-of-thought (CoT) improves reasoning and audio understanding, but generating intermediate reasoning tokens delays responses. Describing fine-grained acoustic cues further lengthens CoT and increases latency. Latent reasoning can reduce this overhead, yet existing methods often trail CoT and remain limited by single-path supervision and reasoning budgets that do not adapt to problem difficulty. We introduce AURAL, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency. To provide initial supervision for latent reasoning, we construct AuralReason-683K: 683K bilingual speech utterances (about 1,000 hours) with concise CoT for emotion recognition, empathetic dialogue, and general reasoning. AURAL-RL then explores beyond these traces, rewarding concise reasoning that yields high-quality answers and adapting reasoning effort to each problem. Across two backbones, AURAL-RL achieves performance comparable to CoT-RL, with larger gains over the respective supervised checkpoints on most metrics. Analysis further shows that harder questions elicit more latent reasoning steps. On Qwen2.5-Omni, it reduces time to the first answer token by 11.8x, from 1.22 to 0.10 s, versus 0.05 s for direct answering.

Figures & tables

Figure 1: Overview of AURAL. Top left: AURAL-SFT pools CoT embeddings into latent targets under compression factor $c$ , then predicts a Gaussian mixture over a joint chunk of $m$ future states that reenters the backbone in one pass, while the language head keeps each state readable and emits EOL . Bottom left: AURAL-RL scores 8 rollouts per prompt with a quality-gated conciseness reward, making latent depth adaptive. Right: AURAL-RL is comparable to CoT-RL over eleven metrics while cutting time to first token from $1.22$ to $0.10$ s ( $11.8\times$ ).

Qwen2.5-Omni based methods
	EchoMind		Emotion		General Ability			VoiceBench
Method	MCQ	OpenQ	IEMOCAP	MELD	MMSU	MMAU-Pro	GPQA	OBQA	MMSU	BBH	IFEval
Base	58.09	3.492	66.24	52.87	61.60	55.66	23.44	79.78	51.53	66.70	54.64
CoT	67.94	3.631	71.39	59.12	63.72	62.33	35.35	85.27	61.97	68.30	55.30
$\mathrm{Latent}_{\mathrm{Coconut}}$	60.83	3.548	67.12	54.29	62.08	57.41	25.82	80.66	53.87	64.50	53.47
$\mathrm{Latent}_{\mathrm{CODI}}$	62.37	3.532	67.85	54.06	62.40	57.82	28.21	81.32	55.30	64.20	53.92

Table 1: Main results across the full evaluation suite.

\mathrm{Latent}_{\mathrm{MSE}}

and

\mathrm{Latent}_{\mathrm{GMM}}

respectively regress one next state and model alternative continuations, both without chunking. AURAL-SFT adds joint chunk prediction and AURAL-RL adds adaptive RL. CoT uses two epochs, while latent methods match this budget with one CoT epoch and one latent SFT epoch. OpenQ uses the official 1 to 5 scale. Other entries are percentages, and higher is better.

Figure 2: Effects of compression factor $c$ , chunk size $m$ , and RL on EchoMind MCQ and response latency. Panel a reports AURAL-SFT accuracy. Panel b reports the gain from AURAL-RL over the matching SFT checkpoint. Panel c reports measured AURAL-RL speedup in time to first token over CoT-RL, as a ratio of means under the protocol of Appendix D . Color and the value marked with $\times$ encode this speedup, and $F\downarrow$ gives the reduction in serial reasoning forward passes, where each CoT reasoning token and each latent chunk require one pass.

Variant	EchoMind MCQ	MMSU Acc.	Latent states	EOL hit rate
AURAL-RL	68.63	64.76	12.36	100.00
w/o CSA	64.74	63.16	12.58	96.62
w/o LSS	66.82	64.30	11.92	98.80
w/o conciseness bonus	68.81	63.96	14.77	100.00
$K=1$	65.99	63.68	13.15	100.00
$K=2$	67.02	63.84	11.94	100.00

Table 2: Ablations of AURAL-RL.

Generation scheme	Two valid paths $\uparrow$	Individual acc.	GPQA	OBQA	BBH
Standard AURAL-RL, one path	–	63.57%	39.38%	87.25%	66.00%
One path per GMM component	38.88%	62.19%	42.49%	89.23%	68.30%
Four samples from one component	12.84%	62.81%	40.29%	87.69%	66.50%
Four components merged	16.09%	62.48%	40.84%	88.13%	67.00%

Table 3: Testing whether GMM components provide useful alternatives. Each branching row generates four paths. Two valid paths counts prompts with two meaningfully different correct paths. Individual accuracy is the average accuracy of the four paths before voting. GPQA, OBQA, and BBH report majority-vote accuracy. The first row is the standard one-path reference.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	CoT initialization	AURAL-SFT	AURAL-RL	CoT-RL
Epochs	1	1	2	2
Updates	$\sim$ 10,700	$\sim$ 10,700	484	484
Nodes	2	2	2	2
GPUs	16	16	16	16
Batch per GPU	4	4	1	1
Gradient accumulation	1	1	1	1

Appendix

Table 4: Training schedule for the Qwen backbone. Batch sizes count examples or prompts. Each node contains eight GPUs, and the head learning rate applies to the latent policy.

Setting	Value
Latent structure	Compression $c=4$ , chunk size $m=10$ , components $K=4$
Head width	3,584
Covariance	Low-rank plus diagonal, rank $r=16$
Log standard deviation	Clipped to $[-4,2]$
Loss weights	Token CE 1, chunk NLL 1, second-pass LSS 1
CSA	Soft targets over the tokens in each pooled CoT group

Appendix

Table 5: Default AURAL-SFT configuration.

Setting	Value
Clipping coefficient	0.2
Advantage constant	$10^{-6}$
Action loss weights	Latent chunk 1, EOL decision 1, answer token 1
Sampling temperatures	Latent 1.0, EOL 1.0, answer 1.0
Answer sampling	Top $p=0.95$ , maximum 128 tokens
Latent budget	Maximum 64 states

Appendix

Table 6: AURAL-RL optimization, rollout, and reward settings.

Benchmark	Subset	Examples	Metric
EchoMind	MCQ	13,401	Accuracy
EchoMind	OpenQ	4,715	Mean judge score
MMSU	Full benchmark	5,000	Accuracy
MMAU-Pro	Objectively scored subset	4,680	Accuracy
GPQA	Main, Diamond, and Extended	546	Accuracy
VoiceBench	OpenBookQA	455	Accuracy

Appendix

Table 7: Evaluation subsets and metrics. EchoMind OpenQ averages scores from 1 to 5 across four dimensions.

System	Decoding
AURAL	Latent and EOL temperatures 1.0, maximum 100 latent states, greedy visible answer
Qwen base	Temperature 0, top $p=1$
Kimi base	Text and audio temperatures 0, top $k=5$ , repetition penalty 1.0, repetition window 16

Appendix

Table 8: Inference settings.

Figure 3: Effects of the compression factor

c

and the chunk size

m

on the ten evaluation metrics beyond EchoMind MCQ for Qwen2.5-Omni. For each metric, the upper panel reports AURAL-SFT and the lower panel reports the gain of AURAL-RL over the matching SFT checkpoint. EchoMind OpenQ uses the 1–5 scale and all other metrics are percentages, so each panel uses its own vertical scale.

Figure 4: Response latency on 256 paired examples. Panel a shows mean stage times, with triangles marking TTFT and bar endpoints marking the full response. Panel b gives speedup over CoT-RL as successive stages enter the measurement. Panel c gives benchmark speedups for AURAL-RL at

c=4,m=10

, ordered by mean visible response length. All speedups use ratios of means.

System	Mean	p50	p95	Speedup
Time to first visible answer token
Base	0.050	0.049	0.069	$24.4\times$
CoT-RL	1.218	1.090	2.056	Reference
AURAL-SFT ( $c=4,m=10$ )	0.107	0.106	0.131	$11.4\times$
AURAL-RL ( $c=4,m=10$ )	0.103	0.103	0.121	$11.8\times$
AURAL-RL ( $c=8,m=10$ )	0.087	0.086	0.105	$13.9\times$

Appendix

Table 9: Latency distributions in seconds over 256 paired examples. Speedup divides the CoT-RL mean by the system mean. The two panels share the same systems and decoding settings.

Figure 5: AURAL-RL speedup over CoT-RL across compression factors and chunk sizes. Panels report reasoning time, TTFT, and full response time as ratios of means over the same 256 examples. Each panel uses its own color scale.

Operator	$D\uparrow$	$J\uparrow$	$O\downarrow$	$\Delta L$	Answer change	Branch accuracy	DCR
Mode intervention	0.312	0.283	0.518	3.41	38.68	62.19	38.88
Within-component	0.087	0.064	0.791	1.28	14.24	62.81	12.84
Collapsed mixture	0.108	0.089	0.742	1.54	18.49	62.48	16.09
Single Gaussian	0.071	0.047	0.816	0.93	11.34	61.77	9.45

Appendix

Table 10: Continuation diversity and correctness over 2,001 prompts.

D

and

J

measure latent and language projection distance,

O

is token overlap, and

\Delta L

is depth difference. Answer change, branch accuracy, and DCR are percentages.

	GPQA		OpenBookQA		BBH
Operator	Branch	Vote	Branch	Vote	Branch	Vote
AURAL-RL, single rollout	39.38	n/a	87.25	n/a	66.00	n/a
Mode intervention	38.19	42.49	85.82	89.23	64.55	68.30
Within-component	38.83	40.29	86.37	87.69	65.18	66.50
Collapsed mixture	38.46	40.84	86.04	88.13	64.88	67.00
Single Gaussian	36.72	38.28	84.89	86.37	64.93	65.90

Appendix

Table 11: Branch and majority vote accuracy by benchmark, in percent. The reference is the default AURAL-RL single rollout from Table 1 . Each branching operator generates four continuations.

Entropy tercile	$H(\pi_{i^{\star}})$ range	$D\uparrow$	$J\uparrow$	Answer change	DCR
Bottom	$[0.08,0.51]$	0.189	0.158	21.44	22.79
Middle	$[0.51,0.96]$	0.314	0.284	38.53	39.13
Top	$[0.96,1.38]$	0.434	0.407	56.07	54.72

Appendix

Table 12: Mode intervention by mixture entropy at the branching state. Terciles split all 2,001 prompts into three equal groups by entropy. Answer change and DCR are percentages.

Branch	Chinese	English	Total	Hours
LIME-440K	197K	123K	320K	367
EmotionCoT-35K	0	18K	18K	22
HumanSpeech-1M	47K	28K	75K	83
GeneralSpeech	133K	137K	270K	526
Total	377K	306K	683K	$\sim$ 1,000

Appendix

Table 13: AuralReason-683K after filtering and CoT construction.

LIME-440K
Source	Retained	Acceptance
LIME Core Chinese	191,405	89.9%
LIME Core English	89,785	93.5%
ECD-TSE extension	24,991	29.8%
Emotion Speech Dataset extension	15,169	59.4%
Total	321,350	76.8%

Appendix

Table 14: Retained sources in LIME-440K and EmotionCoT-35K. Acceptance is the share of input clips that reach the retained set; the Total row pools inputs across sources rather than averaging rates.

Language	Source	Clips	Hours
English	Affective media collection	259,152	337.10
English	GigaSpeech Audiobook	254,871	303.79
English	GigaSpeech YT	130,254	187.28
English	GigaSpeech POD	108,657	156.90
English	VoxMovies	3,752	3.43
Chinese	WenetSpeech L	511,155	542.48

Appendix

Table 15: HumanSpeech-1M after speech screening and segmentation.

Language	Source	Retained
Chinese	WenetSpeech L	26,854
Chinese	Affective media collection	19,784
English	GigaSpeech Audiobook	9,464
English	GigaSpeech YT	6,809
English	Affective media collection	6,308
English	GigaSpeech POD	4,681

Appendix

Table 16: HumanSpeech-1M after dialogue filtering.

Task family	Share
Explanation and comprehension	35.8%
Numeric and mathematical	20.2%
Limited generation	20.0%
Multiple choice	13.6%
Factual entity	8.1%
Boolean judgement	2.3%

Appendix

Table 17: GeneralSpeech task distribution before final balancing.

Dialogue
State	Content
Cue	Audible properties that affect the reply, such as a shaky voice and long pauses
Need	The conversational need suggested by those cues, such as reassurance before advice
Risk	Ambiguity or safety considerations that affect the reply
Plan	The response act, order, and tone, such as brief validation followed by one next step
Emotion classification

Appendix

Table 18: CoT states for dialogue and emotion classification. The dialogue Risk state is included when ambiguity or safety changes the response plan.

Branch	p10	p50	p90	Mean states
LIME-440K	50	71	95	3.67
EmotionCoT-35K	41	46	52	3.00
HumanSpeech-1M	49	65	89	3.31
GeneralSpeech	40	63	88	2.10

Appendix

Table 19: CoT token lengths and mean state counts after rewriting. A state is one reasoning step; the dialogue and emotion branches use the named states in Table 18 . All traces contain fewer than 120 tokens.

Pairing	Fully executed	Partly executed	Contradictory
True pair	91.3%	8.0%	0.7%
Mismatched control	6.0%	31.0%	63.0%

Appendix

Table 20: Plan execution on 300 dialogue examples. The judge rates whether the response carries out the plan stated in the CoT. Controls pair each CoT with a randomly chosen response from another example.

Constructed CoT	Response
D1. Good news, a heavy voice LIME-440K
Transcript: “They promoted me to lead the sustainability project…”
Cue: low pitch, slow tempo, heavy sighing breath at the start, downward intonation $\to$ heavy reluctance, overwhelm Need: validation that a promotion can feel like a burden rather than a reward Risk: congratulating them forces a positive reaction they do not feel Plan: acknowledge the heavy weight of the news, then ask what makes this feel daunting	That sounds like a massive undertaking, and from your tone, it feels more like a heavy weight than a celebration. What is the part of this new role that is weighing on you the most right now?
D2. Letting a success be a success LIME-440K
Transcript: “The professor approved my thesis draft without any changes!”
Cue: bright, rapid tempo, rising pitch, audible smile $\to$ relief, pride Need: validation of massive effort behind clean pass Risk: focusing on next steps ruins rare moment of absolute victory Plan: match triumphant energy, highlight zero-change approval, invite celebration	Without any changes? That is practically unheard of! You must be walking on air right now after all that work. How are you going to celebrate tonight?

Appendix

Table 21: Empathetic dialogue examples. The response plan changes with the voice and conversational need: make room for reluctance, share a success, offer a manageable boundary, or respect a request for space.

Constructed CoT	Response
S1. Amusement in a factual utterance Expresso
Transcript: “There’s an alarm set for today at two thirty AM, another for tomorrow at two fifteen PM, and for Wednesday at seven PM.”
Cue: breathy giggling, wide pitch variation, and bright, smiling vocal resonance. Rule out: confused — presence of bubbling laughter and rising, playful pitch contours. State: high-arousal positive valence with spontaneous amusement.	happy
S2. Strained restraint versus overt anger IEMOCAP
Transcript: “Okay, no, look, I- I- I- I’m really – I’m trying to be patient. I know this isn’t your fault. I know that you didn’t do it”
Cue: rapid and uneven tempo, audible sighs, tense vocal quality, frequent self-interruptions Rule out: angry — lacks the high-energy acoustic projection and sharp volume spikes State: moderate-to-high arousal negative valence, showing signs of strained self-control	frustrated

Appendix

Table 22: Speech emotion recognition (SER) examples from three human speech sources. The task is to identify emotion from the audio. Each CoT connects audible evidence to a competing-class exclusion and an affective state; the selected label appears only in the response.

G1. Minimum feasible dividend GeneralSpeech
Question: When two numbers are divided, the quotient is 12 and the remainder is 3. What is the smallest possible dividend?
Original CoT: Qwen3-Omni-Thinking	Our rewritten CoT
Okay, let’s see. The problem says when two numbers are divided, the quotient is 12 and the remainder is 3. We need to find the smallest possible dividend. Hmm, right. So first, I remember that when you divide numbers, there’s a relationship between the dividend, divisor, quotient, and remainder. Let me recall the formula: Dividend = Divisor $\times$ Quotient + Remainder. Yeah, that’s right. So if we let the divisor be $d$ , the quotient is 12, remainder is 3, then the dividend $N$ is $N=12d+3$ . But wait, the remainder has to be less than the divisor, right? Because if the remainder was equal to or larger than the divisor, you could divide again. So the remainder $r$ must satisfy $0\leq r<d$ . In this case, the remainder is 3, so $3<d$ . That means the divisor $d$ has to be greater than 3. So the smallest possible divisor would be 4, since it has to be an integer greater than 3. Let me check that. If $d=4$ , then the dividend would be $12\times 4+3=48+3=51$ . Let’s verify: 51 divided by 4 is 12 with a remainder of 3, because 4 times 12 is 48, and 51 minus 48 is 3. Yep, that works. Is there a smaller divisor? Well, if $d=3$ , then the remainder would have to be less than 3, but the remainder here is 3, which is not less than 3, so that’s not allowed. If $d=2$ , same problem, remainder 3 is bigger than 2, so you could divide more. So the smallest possible divisor is 4, which gives the smallest dividend of 51. Let me just make sure. Let’s try $d=4$ : $51\div 4=12.75$ , but in integer division, it’s 12 with remainder 3. Correct. If we take $d=5$ , then dividend is $12\times 5+3=63$ , which is bigger, so 51 is smaller. So yeah, 51 is the smallest possible dividend.	Given: Dividend = Divisor $\times 12+3$ , and remainder 3 must be less than divisor. Step: The smallest integer divisor satisfying $3<d$ is 4. Step: Dividend $=12\times 4+3=48+3$ . Response The smallest possible dividend is 51, because the divisor must be at least 4, and 12 times 4 plus 3 equals 51.

Appendix

Table 23: General reasoning: the division identity and remainder bound determine the smallest valid divisor, making repeated checks unnecessary. CoT:

529\to 62

tokens (

8.5\times

shorter).

G2. Selecting the obtuse angle GeneralSpeech
Question: The ratio of two interior angles of a parallelogram is $4:5$ . What is the degree measure of one of the two obtuse angles of the parallelogram?
Original CoT: Qwen3-Omni-Thinking	Our rewritten CoT
Okay, let’s try to figure out this problem. So, we have a parallelogram, and the ratio of two interior angles is $4:5$ . We need to find the measure of one of the obtuse angles. Hmm, first, I remember that in a parallelogram, opposite angles are equal, and consecutive angles are supplementary. That means if one angle is, say, $x$ , the next one is $180-x$ , right? Because they add up to 180 degrees. So, the problem says the ratio of two interior angles is $4:5$ . Let’s assume these are consecutive angles because if they were opposite angles, they would be equal, so their ratio would be $1:1$ , which isn’t $4:5$ . So, they must be consecutive angles. Let me confirm that. In a parallelogram, adjacent angles are supplementary, so they add up to 180 degrees. So, if the ratio is $4:5$ , let’s let the measures be $4k$ and $5k$ . Then, $4k+5k=180$ degrees. Because they are consecutive angles, they should add up to 180. Let me check that. Yes, in a parallelogram, consecutive angles are supplementary. So, $4k+5k=180$ . That would be $9k=180$ , so $k=20$ . Then, the angles would be $4\times 20=80$ degrees and $5\times 20=100$ degrees. Now, which one is obtuse? Obtuse angles are greater than 90 degrees, so 100 degrees is obtuse. So, the measure of one of the obtuse angles is 100 degrees. Let me just make sure. In a parallelogram, opposite angles are equal. So, if two consecutive angles are 80 and 100, then the other two angles are also 80 and 100. So, there are two acute angles (80 degrees) and two obtuse angles (100 degrees). So, the obtuse angles are 100 degrees each. That makes sense. Let me verify the ratio: $80:100$ simplifies to $4:5$ , which matches the problem statement. Yep, that works. So, the answer should be 100 degrees.	Given: parallelogram interior angles ratio $4:5$ , opposite angles equal, consecutive supplementary. Step: The ratio must be for consecutive angles. Let them be $4k$ and $5k$ . Then $4k+5k=180\Rightarrow 9k=180\Rightarrow k=20$ . Step: The two angles are $4k$ and $5k$ . The larger is obtuse. Response The obtuse angle is 100 degrees.

Appendix

Table 24: General reasoning: establish consecutive angles, derive their common scale, and select the requested angle. CoT:

517\to 86

tokens (

6.0\times

shorter).

G3. Resolving two stars in blue light GeneralSpeech
Question: Why might some stars appear double in blue light, though they could not be resolved in red light with the same telescope?
Original CoT: Qwen3-Omni-Thinking	Our rewritten CoT
We are considering the phenomenon where a star appears double in blue light but not in red light with the same telescope. The key point is the diffraction limit of the telescope. The angular resolution ( $\theta$ ) of a telescope is given by the Rayleigh criterion: $\theta=1.22\times(\lambda/D)$ where: $\lambda$ = wavelength of light $D$ = diameter of the telescope’s aperture Since blue light has a shorter wavelength ( $\lambda_{\mathrm{blue}}$ ) than red light ( $\lambda_{\mathrm{red}}$ ), the angular resolution for blue light is better (smaller $\theta_{\mathrm{blue}}$ ) than for red light ( $\theta_{\mathrm{red}}$ ). Therefore, if two stars are separated by an angular distance that is greater than $\theta_{\mathrm{red}}$ (so they are not resolved in red light) but less than $\theta_{\mathrm{blue}}$ (so they are resolved in blue light), then we would see them as double in blue light but not in red. However, note that the Rayleigh criterion is an approximation and the actual resolution also depends on the point spread function and other factors. But the main idea is that the resolving power of a telescope improves as the wavelength decreases. Example: Let $\lambda_{\mathrm{red}}\approx 700$ nm, $\lambda_{\mathrm{blue}}\approx 400$ nm. Then $\theta_{\mathrm{blue}}/\theta_{\mathrm{red}}=\lambda_{\mathrm{blue}}/\lambda_{\mathrm{red}}\approx 400/700\approx 0.57$ . So the resolving power in blue is about 1.75 times better than in red. Therefore, if two stars are separated by an angle that is between $\theta_{\mathrm{blue}}$ and $\theta_{\mathrm{red}}$ , they will be resolved in blue but not in red. Hence, the answer is: because blue light has a shorter wavelength, which allows the telescope to resolve finer details (smaller angular separations) than red light. So the phenomenon occurs due to the wavelength dependence of the diffraction limit.	Given: telescope angular resolution $\theta\approx 1.22\lambda/D$ Step: blue light has shorter $\lambda$ than red light $\to$ smaller $\theta$ for blue Step: if two stars have angular separation $\Delta\theta$ such that $\theta_{\mathrm{blue}}<\Delta\theta<\theta_{\mathrm{red}}$ , they are unresolved in red but resolved in blue Response Stars appear double in blue light because the telescope’s diffraction limit depends on wavelength. Blue light, with its shorter wavelength, gives better angular resolution than red light. If the stars’ separation falls between the red and blue resolution limits, they blend into one in red but appear as two in blue.

Appendix

Table 25: General reasoning: explaining an observation from a physical law. The rewrite keeps the diffraction limit and the required separation interval. It also resolves the original CoT’s reversed inequalities in its first “Therefore” paragraph; the original later gives the correct interval verbally. CoT length:

385\to 66

tokens (

5.8\times

shorter).

Explore similar work

Sep 29, 2026cs.CL

Thinking in Depth, Speaking Directly: Recurrent Latent Reasoning for Paralinguistically Grounded Spoken Dialogue

Empathetic spoken dialogue requires models to use both what is said and how it is said to decide how to respond. Explicit CoT can improve paralinguistic perception and make acoustic cues more explicit in replies, yet does not ensure their effective use in response planning. We call this mismatch the perception-reasoning gap. In addition, CoT may not fully capture acoustic cues in words, and generating it adds inference latency. To address these limitations, we introduce LoopSLM, which builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass. Its two-stage training further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT. On EchoMind, LoopSLM improves paralinguistic understanding, reasoning, and reply quality over Qwen2.5-Omni-7B. Against the CoT-SFT baseline, LoopSLM gains over 20 points in reasoning accuracy while generating 64.5% fewer tokens at half the latency. It also outperforms Qwen3-Omni-Thinking on most empathetic reply metrics with 34x lower latency. Despite training only on dialogue data, LoopSLM improves accuracy on general audio benchmarks.

Shengbo Cai, Yuxiang Wang, Jingran Xie +5

Tsinghua University · Tencent Hunyuan · The Chinese University of Hong Kong, Shenzhen

Speech Language Models Paralinguistic Cues Efficient Latent Reasoning Empathy Perception Gap

Ask PDF

Apr 20, 2026cs.SD

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.

Xiang He, Chenxing Li, Jinting Wang +5

Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Audio Understanding Text-To-Audio Chain-of-Thought Reasoning

Ask PDF

May 26, 2026cs.CL

Learning When to Think While Listening in Large Audio-Language Models

Recent advances in Large Audio-Language Models (LALMs) have made real-time, streaming spoken interaction increasingly practical. In this setting, reasoning quality and responsiveness are tightly coupled: delaying reasoning until the speech endpoint can improve answer quality but moves deliberation into user-visible response delay, while answering too early risks committing before decisive evidence arrives. We introduce a learnable wait-think-answer control formulation for LALMs. Motivated by the incremental nature of human conversation, the controller decides under partial audio evidence when to wait, when to externalize a compact reasoning update, and when to answer. Using Qwen2.5-Omni-7B as the base model, we construct aligned wait-think-answer traces from spoken reasoning data, train the controller with supervised fine-tuning (SFT), and then apply Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO). The reward combines answer correctness, action validity, update timing, latency synchronization, reasoning quality, and chain consistency, optimizing the complete wait-think-answer trajectory and not the final answer alone. On a six-task synthetic spoken reasoning question answering (SRQA) benchmark, the six-reward DAPO controller improves the row-weighted accuracy from 67.6% to 70.3% while reducing post-endpoint final-think length by 14% under the same Qwen deployment harness. On a 186-item human-recorded Real Audio Bench, a transfer check beyond text-to-speech (TTS)-rendered speech, the controller family remains functional: SFT achieves the strongest accuracy, while the six-reward DAPO controller is the only learned variant whose final-think length falls below the base. These results suggest that a streaming model should learn when to make intermediate reasoning explicit during the audio stream.

Zhiyuan Song, Weici Zhao, Yang Xiao +3

University of Pennsylvania

Large Audio Language Models

Ask PDF