SeLaR: Selective Latent Reasoning in Large Language Models

Renyu Fu; Guibo Luo

cs.CLApr 9, 2026

SeLaR: Selective Latent Reasoning in Large Language Models

Authors: Renyu Fu, Guibo Luo

Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology Shenzhen Graduate School, Peking University

Abstract

Chain-of-Thought (CoT) has become a cornerstone of reasoning in large language models, yet its effectiveness is constrained by the limited expressiveness of discrete token sampling. Recent latent reasoning approaches attempt to alleviate this limitation by replacing discrete tokens with soft embeddings (probability-weighted mixtures of token embeddings) or hidden states, but they commonly suffer from two issues: (1) global activation injects perturbations into high-confidence steps, impairing reasoning stability; and (2) soft embeddings quickly collapse toward the highest-probability token, limiting exploration of alternative trajectories. To address these challenges, we propose SeLaR (Selective Latent Reasoning), a lightweight and training-free framework. SeLaR introduces an entropy-gated mechanism that activates soft embeddings only at low-confidence steps, while preserving discrete decoding at high-confidence steps. Additionally, we propose an entropy-aware contrastive regularization that pushes soft embeddings away from the dominant (highest-probability) token's direction, encouraging sustained exploration of multiple latent reasoning paths. Experiments on five reasoning benchmarks demonstrate that SeLaR consistently outperforms standard CoT and state-of-the-art training-free methods.

Figures & tables

Figure 1: Normalized entropy distributions of decoding steps across five reasoning benchmarks using Qwen3-8B. Each subplot shows the density of step-wise entropy values, revealing a clear long-tail structure: the majority of steps concentrate in a low-entropy region (deterministic steps), while a smaller tail extends toward higher entropy values (exploratory steps).

Figure 2: Overview of SeLaR. At each decoding step, we compute the normalized entropy over top- $k$ tokens. If entropy falls below threshold $\tau$ (deterministic step), standard discrete decoding is applied. Otherwise (exploratory step), we construct a soft embedding from top- $k$ candidates and apply contrastive regularization to push it away from the dominant token, encouraging exploration of alternative reasoning paths.

Method	GSM8K	MATH500	GPQA	AIME24	AIME25	Avg
	Qwen3-1.7B Yang et al. (2025a)
CoT (Sampling)	90.07	92.00	39.39	50.00	33.33	60.96
CoT (Greedy)	88.32	90.60	31.31	40.00	30.00	56.05
Soft Thinking	89.46	91.00	33.83	36.67	36.67	57.53
SwiR	89.84	92.00	37.88	46.67	23.33	57.94
SeLaR	90.60	92.60	35.35	53.33	36.67	61.71

Table 1: Detailed results on reasoning benchmarks. Results highlighted in green indicate performance comparable to or better than CoT (Sampling). Results highlighted in red indicate a performance drop relative to CoT (Sampling).

Figure 3: Computational overhead of SeLaR and SwiR relative to CoT (Sampling) on Qwen3-8B. (a) Tokens per correct answer: SeLaR beats SwiR on 4 of 5 benchmarks, most notably on AIME 2024/2025. (b) Average tokens on correctly-answered samples. (c) Average tokens on wrongly-answered samples.

Figure 4: Layer-wise top-

k

overlap (

k{=}10

) between the soft-embedding forward pass and single-token reference passes, aggregated over

N{=}200

branching steps from AIME 2025 on Qwen3-8B. Shaded bands denote standard error. Left: without contrastive regularization,

O_{\text{top1}}

dominates

O_{\text{top2}}

in deep layers, reproducing the collapse behavior of Wu et al. (2025) . Right: with contrastive regularization, both overlaps remain non-zero and comparable, indicating multiple reasoning trajectories coexist rather than a single one being swapped.

Component Ablation
Method	GSM8K	MATH500	GPQA	AIME24	AIME25	Avg
CoT (Sampling)	95.45	98.00	61.62	76.67	66.67	79.68
SeLaR (Full)	95.83	97.00	61.62	83.33	80.00	83.56
w/o Selective Activation	95.14	95.80	57.58	76.67	66.67	78.37
w/o Contrastive Reg.	94.92	96.20	57.58	70.00	60.00	75.74

Table 2: Ablation study on Qwen3-8B. We evaluate the contribution of each component in SeLaR.

Figure 5: Activation frequency analysis on Qwen3-8B. Exploratory steps (high entropy) account for 6.2%–13.8% of total reasoning tokens.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

$k$	$\tau$	GSM8K	MATH500	GPQA	AIME24	AIME25	Avg
Varying $\tau$ (fixed $k=3$ )
3	0.3	95.00	96.40	53.03	76.67	80.00	80.22
3	0.4	95.22	96.60	54.55	83.33	70.00	79.94
3	0.5	95.60	97.00	55.05	76.67	80.00	80.86
3	0.6	95.83	96.00	60.10	76.67	70.00	79.72
3	0.7	95.53	96.40	60.10	76.67	56.67	77.07

Appendix

Table 3: Sensitivity analysis on Qwen3-8B. We vary the entropy threshold

\tau

and top-

k

value while keeping other settings fixed.

Figure 6: Case study on an AIME 2025 geometry problem. Standard CoT computes Arc HJ = 23° at the critical exploratory step, leading to an incorrect final answer of 334 . SeLaR activates selective latent reasoning at this exploratory step, correctly computing Arc HJ = 24° and yielding the correct answer 336 .

Figure 7: Activation frequency analysis on DeepSeek-R1-Distill-Llama-8B. Exploratory steps account for 8.8%–15.3% of total reasoning tokens, higher than Qwen3-8B (6.2%–13.8%).

Method	GSM8K	MATH500	GPQA	AIME24	AIME25	Avg
	DeepSeek-R1-Distill-Llama-8B DeepSeek-AI et al. (2025)
CoT (Sampling)	89.01	90.20	42.93	36.67	30.00	57.76
CoT (Greedy)	85.29	83.60	30.30	30.00	26.67	51.17
Soft Thinking	85.67	82.20	32.83	30.00	23.33	50.81
SwiR	89.31	87.80	45.96	50.00	23.33	59.28
SeLaR	90.22	88.20	40.91	46.67	36.67	60.53

Appendix

Table 4: Results of DeepSeek-R1-Distill-Llama-8B. Results highlighted in green indicate performance comparable to or better than CoT (Sampling). Results highlighted in red indicate a performance drop relative to CoT (Sampling).

CoT Reasoning

Chain-of-thought (CoT) reasoning studies how language models use intermediate steps, visible or latent, to solve multi-step tasks.

See all 228 papers

Latent Thought Flow: Efficient Latent Reasoning in Large Language Models
Jun 15, 2026Xiandong Zou, Jing Huang, Jianshu Li +1RL for Language Model Reasoning Efficient Language Model Reasoning
Singapore Management University · Ant Group
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
May 25, 2026Hui Xie, Jie Liu, Ziyue Qiao +1Efficient Language Model Reasoning LLM Inference Acceleration
Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China
Sparse RL Relaxation for LLM Reasoning
Oct 19, 2026Jiabin Fan, Matthias Lindemann, Zita Marinho +2CoT Reasoning RL for Language Model Reasoning
University of Alberta · Alberta Machine Intelligence Institute (Amii) · Instituto de Telecomunicações +3

Jun 15, 2026J/K move · Enter open · S save

Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

Xiandong Zou, Jing Huang, Jianshu Li, Pan Zhou

Singapore Management University · Ant Group

RL for Language Model Reasoning Efficient Language Model Reasoning Generative Flow Networks LLM Inference Acceleration Continuous Latent Reasoning Latent Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) increasingly rely on intermediate reasoning, yet explicit Chain-of-Thought (CoT) suffers from a linguistic space bottleneck: each thought must be decoded into tokens, causing high inference overhead. Latent reasoning moves deliberation into continuous space, but existing methods mostly learn deterministic or reward-maximizing paths, lacking a principled way to allocate probability across trajectories with different correctness and costs. We propose Latent Thought Flow (LTF), which models reasoning as variable-length continuous trajectories and trains a sampler to match a reward-induced posterior over answer quality and computation cost. We instantiate this with a continuous GFlowNet using stochastic latent transitions. To handle sparse answer supervision, we introduce an Entropy-Weighted Subtrajectory Balance objective for intermediate rewards and a reference-prior regularizer to anchor exploration. Experiments under finetuning and transfer learning settings show that LTF outperforms explicit CoT and latent reasoning baselines, improving accuracy by 9.5% while reducing reasoning length by 27.2% on average compared with strong latent reasoning baselines.

Abstract

CoT Reasoning

Explore similar work

Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains

Sparse RL Relaxation for LLM Reasoning