Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Jiecong Wang; Hao Peng; Zhanyi Wang; Chunyang Liu; Guanlin Wu

cs.AIJan 29, 2026

Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Authors: Jiecong Wang, Hao Peng, Zhanyi Wang, Chunyang Liu, Guanlin Wu

Organizations: Beihang University · Didi Chuxing

Abstract

Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and early token commitments in discrete reasoning traces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, many such methods optimize latent states end to end without a trained interface for intermediate textual readout, and several representative configurations use a pre-defined number of latent steps during inference. In this work, we introduce \textbf{PLaT} (\textbf{P}lanning with \textbf{La}tent \textbf{T}houghts), a framework that decouples latent planning from verbalization. The Planner deterministically evolves latent planning states, while an independent Decoder provides textual readouts when needed. Answer-aware textual stopping allows the latent rollout to use a problem-dependent number of groups rather than a pre-specified chain length. PLaT achieves competitive coverage at larger $k$ in several mathematical settings, with lower Pass@1: on Llama-1B GSM8K, it reaches 80.59% Pass@128 versus CODI's 72.37%. These results support PLaT as a candidate-generation interface supplying multiple textual readouts for downstream verification or reranking.

Figures & tables

Figure 1: Comparison of PLaT and other reasoning strategies. Orange blocks denote token embeddings and blue blocks denote continuous latent states. CoT generates explicit reasoning tokens; Coconut progressively replaces textual reasoning with latent states during training; CODI distills explicit reasoning into a latent chain. PLaT separates latent-state evolution from textual readout: the vertical chain represents planning, and each horizontal branch represents a decoded reasoning segment or answer. Decoded text is not fed back into the Planner.

Figure 2: Framework of the proposed PLaT paradigm. The Planner autoregressively generates latent states conditioned on the question, and an independent Decoder uses each latent group as a soft prefix to generate a text segment. The illustration shows a general

L=2

example, with two latent vectors per decoded segment; the main experimental configuration uses

L=1

Model	Method	GSM8K				GSM-Hard				SVAMP				MultiArith
Model	Method	@1	@32	@64	@128	@1	@32	@64	@128	@1	@32	@64	@128	@1	@32	@64	@128
GPT-2	CoT-SFT	35.30	74.20	80.41	86.33	7.10	18.35	20.65	22.52	36.63	65.37	70.50	75.47	85.00	99.07	99.44	99.63
	Coconut	32.94	55.69	61.45	66.19	7.62	13.65	15.01	16.15	35.25	50.15	53.05	56.45	81.67	90.83	91.94	93.33
	CODI	40.16	63.00	67.12	69.90	8.74	14.58	15.52	16.53	38.60	53.67	55.80	58.13	92.04	96.67	96.67	96.67
	PLaT	18.35	59.51	67.74	76.08	3.75	14.22	17.06	18.88	21.80	53.70	60.75	66.45	33.61	79.44	87.50	91.94
Llama-1B	CoT-SFT	49.96	86.62	90.45	93.18	11.45	23.68	25.50	27.72	59.50	87.97	91.07	93.37	94.07	100.00	100.00	100.00

Table 1: Main results on the GSM8K-trained models (%). Pass@

k

is computed from the first

k

of 128 samples within each run, then averaged over the three runs. Standard deviations are given in Appendix B.5 . Gray CoT-SFT rows are excluded from the latent-method ranking. Bold and underlined entries mark the highest and second-highest displayed latent-method scores per column, including ties. PLaT is evaluated with

K=8

(PLaT-8).

	CoT	Coconut	CODI	PLaT
Fwd.	25.55	6.00	6.00	12.83
Time (ms)	$349.6_{\pm 8.9}$	$100.6_{\pm 3.2}$	$240.0_{\pm 17.2}$	$237.5_{\pm 59.2}$

Table 2: Efficiency comparison. The PLaT column reports lazy, greedy inference with the GPT-2 backbone on 200 GSM8K test questions. Forward counts include all backbone calls during planning and readout; times are per-question means

\pm

across-question standard deviations. All methods are measured on the same hardware.

Panel of Figure 3 — Figure 3: Textual readout branching from latent states. Results use PLaT and CoT-SFT on 1,319 questions with 10 sampled paths per question. For PLaT, the Planner trajectory is fixed for a question; the reported branches are distinct textual continuations sampled from intermediate latent states. In (a), curves show distinct and valid readout counts after normalizing each question’s state sequence to 11 positions. In (b), samples without intermediate steps are excluded; the 16 zones annotate counts and between-method differences.

	Pass@1	Pass@32	Pass@64	Pass@128
PLaT-8	$36.20\pm 0.38$	$68.73\pm 0.05$	$75.28\pm 0.00$	$80.59\pm 0.32$
PLaT semantic-step	$37.86\pm 1.10$	$69.45\pm 0.46$	$75.16\pm 0.62$	$80.82\pm 0.62$
CoT initialization	$29.95\pm 0.43$	$67.48\pm 0.86$	$74.15\pm 0.43$	$78.01\pm 0.11$
Decoder latent history	$38.51\pm 0.11$	$67.29\pm 0.70$	$73.62\pm 0.21$	$78.85\pm 0.32$
Decoder question + latent history	$31.61\pm 0.95$	$67.73\pm 0.99$	$74.37\pm 0.23$	$80.29\pm 0.82$

Table 3: PLaT architectural and segmentation variants on Llama-1B GSM8K (%). Entries report mean

\pm

sample standard deviation over multiple verified sampling runs. Decoder-context variants use PLaT-8 with

L=1

; fixed-

K

and latent-count sweeps are reported separately.

Method	MATH-500		AIME 2024		AIME 2025
	Pass@1	Pass@128	Pass@1	Pass@128	Pass@1	Pass@128
CoT-SFT	$16.20\pm 0.28$	$77.70\pm 0.99$	$1.67\pm 2.36$	$15.00\pm 2.36$	$1.67\pm 2.36$	$20.00\pm 4.71$
Coconut	$\underline{5.20}\pm 0.85$	$27.60\pm 0.57$	$\underline{0.00}\pm 0.00$	$3.33\pm 0.00$	$\underline{0.00}\pm 0.00$	$\underline{10.00}\pm 0.00$
CODI	$\textbf{7.70}\pm 0.71$	$\underline{38.70}\pm 0.71$	$\textbf{1.67}\pm 2.36$	$\textbf{18.33}\pm 2.36$	$\underline{0.00}\pm 0.00$	$3.33\pm 4.71$
PLaT-16	$2.50\pm 0.14$	$\textbf{44.80}\pm 2.55$	$\underline{0.00}\pm 0.00$	$\underline{11.67}\pm 7.07$	$\textbf{1.67}\pm 2.36$	$\textbf{16.67}\pm 0.00$
PLaT-32	$0.80\pm 0.00$	$35.60\pm 1.13$	$\underline{0.00}\pm 0.00$	$6.67\pm 0.00$	$\underline{0.00}\pm 0.00$	$8.33\pm 7.07$

Table 4: OpenR1-Math-trained Llama-1B results on longer-form benchmarks (Pass@1 and Pass@128, %). Each dataset reports the two prefix-coverage metrics separately. All entries report means

\pm

sample standard deviations over three complete sampling runs. Gray CoT-SFT is excluded from ranking; bold and underlined scores denote the best and second-best latent-method results in each column, with ties sharing a mark. This table is a separate evaluation setting and is not directly comparable with the GSM8K-trained main table.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition	Note
$x$	Input question sequence	Token sequence
$y$	Complete reasoning chain	$N$ segments, $T$ tokens
$y_{k}$	The $k$ -th textual segment	At most $K$ tokens
$k$	Index of textual segments	$k\in\{1,\dots,N\}$
$i$	Index within a latent group	$i\in\{1,\dots,L\}$
$\mathcal{M}$	Planner backbone	LoRA-adapted

Appendix

Table 1: Summary of notations used in PLaT.

Method	Dataset	Pass@1	Pass@32	Pass@64	Pass@128
CoT-SFT	GSM8K	$35.30\pm 0.54$	$74.20\pm 0.23$	$80.41\pm 0.29$	$86.33\pm 0.56$
CoT-SFT	GSM-Hard	$7.10\pm 0.44$	$18.35\pm 0.23$	$20.65\pm 0.49$	$22.52\pm 0.60$
CoT-SFT	SVAMP	$36.63\pm 0.93$	$65.37\pm 0.84$	$70.50\pm 0.82$	$75.47\pm 0.55$
CoT-SFT	MultiArith	$85.00\pm 0.00$	$99.07\pm 0.32$	$99.44\pm 0.00$	$99.63\pm 0.32$
Coconut	GSM8K	$32.94\pm 0.59$	$55.69\pm 0.16$	$61.45\pm 0.27$	$66.19\pm 0.64$
Coconut	GSM-Hard	$7.62\pm 0.05$	$13.65\pm 0.32$	$15.01\pm 0.11$	$16.15\pm 0.11$

Appendix

Table 2: GPT-2 prefix results (%): mean

\pm

sample standard deviation over repeated sampling runs.

Method	Dataset	Pass@1	Pass@32	Pass@64	Pass@128
CoT-SFT	GSM8K	$49.96\pm 0.32$	$86.62\pm 0.48$	$90.45\pm 0.54$	$93.18\pm 0.21$
CoT-SFT	GSM-Hard	$11.45\pm 0.20$	$23.68\pm 0.64$	$25.50\pm 0.44$	$27.72\pm 0.19$
CoT-SFT	SVAMP	$59.50\pm 0.95$	$87.97\pm 0.31$	$91.07\pm 0.25$	$93.37\pm 0.40$
CoT-SFT	MultiArith	$94.07\pm 1.70$	$100.00\pm 0.00$	$100.00\pm 0.00$	$100.00\pm 0.00$
Coconut	GSM8K	$27.52\pm 0.57$	$49.28\pm 0.50$	$53.68\pm 0.47$	$58.07\pm 0.67$
Coconut	GSM-Hard	$6.47\pm 0.29$	$11.95\pm 0.38$	$13.17\pm 0.36$	$14.18\pm 0.55$

Appendix

Table 3: Llama-1B prefix results (%): mean

\pm

sample standard deviation over repeated sampling runs.

Method	Dataset	Pass@1	Pass@32	Pass@64	Pass@128
CoT-SFT	GSM8K	$58.88\pm 0.93$	$92.90\pm 0.53$	$95.27\pm 0.69$	$96.71\pm 0.32$
CoT-SFT	GSM-Hard	$18.95\pm 0.15$	$38.34\pm 0.27$	$41.37\pm 0.76$	$44.12\pm 1.28$
CoT-SFT	SVAMP	$74.67\pm 1.05$	$95.50\pm 0.17$	$96.90\pm 0.20$	$97.83\pm 0.21$
CoT-SFT	MultiArith	$97.78\pm 1.67$	$100.00\pm 0.00$	$100.00\pm 0.00$	$100.00\pm 0.00$
Coconut	GSM8K	$28.38\pm 0.83$	$62.32\pm 0.13$	$70.18\pm 0.69$	$75.87\pm 0.34$
Coconut	GSM-Hard	$6.09\pm 0.37$	$15.62\pm 0.20$	$17.29\pm 0.13$	$19.13\pm 0.27$

Appendix

Table 4: Llama-3B prefix results (%): mean

\pm

sample standard deviation over repeated sampling runs.

	Pass@1	Pass@32	Pass@64	Pass@128
CoT initialization	$29.95\pm 0.43$	$67.48\pm 0.86$	$74.15\pm 0.43$	$78.01\pm 0.11$
semantic-step	$37.86\pm 1.10$	$69.45\pm 0.46$	$75.16\pm 0.62$	$80.82\pm 0.62$
PLaT-2	$13.98\pm 0.95$	$49.43\pm 1.32$	$58.78\pm 0.86$	$68.54\pm 1.19$
PLaT-5	$34.72\pm 0.96$	$69.56\pm 0.27$	$75.93\pm 0.48$	$81.80\pm 0.21$
PLaT-8	$36.20\pm 0.38$	$68.73\pm 0.05$	$75.28\pm 0.00$	$80.59\pm 0.32$
PLaT-8, $L=2$	$37.04\pm 0.27$	$68.69\pm 0.11$	$74.53\pm 0.21$	$79.49\pm 0.27$

Appendix

Table 5: Llama-1B GSM8K ablations (%).

	Pass@1	Pass@32	Pass@64	Pass@128
CoT initialization	$7.25\pm 0.31$	$16.50\pm 0.12$	$18.52\pm 0.55$	$20.04\pm 0.42$
semantic-step	$9.00\pm 0.31$	$17.11\pm 0.32$	$18.65\pm 0.35$	$20.50\pm 0.37$
PLaT-2	$2.77\pm 0.27$	$10.80\pm 0.27$	$13.68\pm 0.16$	$16.72\pm 0.48$
PLaT-5	$8.04\pm 0.21$	$17.02\pm 0.27$	$18.95\pm 0.21$	$20.58\pm 0.38$
PLaT-8	$8.26\pm 0.00$	$16.30\pm 0.54$	$18.23\pm 0.05$	$19.75\pm 0.16$
PLaT-8, $L=2$	$8.30\pm 0.59$	$16.91\pm 0.21$	$18.57\pm 0.32$	$19.94\pm 0.21$

Appendix

Table 6: Llama-1B GSM-Hard ablations (%).

	Pass@1	Pass@32	Pass@64	Pass@128
CoT initialization	$36.70\pm 0.14$	$65.40\pm 1.70$	$69.55\pm 0.07$	$72.35\pm 0.07$
semantic-step	$44.20\pm 0.44$	$68.77\pm 0.49$	$72.37\pm 0.65$	$75.93\pm 0.84$
PLaT-2	$19.50\pm 0.14$	$50.80\pm 0.71$	$57.90\pm 0.85$	$65.15\pm 0.64$
PLaT-5	$45.05\pm 0.21$	$68.70\pm 0.42$	$73.15\pm 0.07$	$77.45\pm 0.21$
PLaT-8	$43.00\pm 0.95$	$66.87\pm 0.42$	$70.63\pm 0.55$	$74.47\pm 0.32$
PLaT-8, $L=2$	$45.10\pm 0.28$	$67.45\pm 0.07$	$71.75\pm 0.21$	$75.50\pm 0.42$

Appendix

Table 7: Llama-1B SVAMP ablations (%).

	Pass@1	Pass@32	Pass@64	Pass@128
CoT initialization	$72.50\pm 1.18$	$93.61\pm 0.39$	$95.56\pm 0.00$	$95.83\pm 0.39$
semantic-step	$88.89\pm 1.67$	$95.74\pm 0.64$	$96.67\pm 0.56$	$96.85\pm 0.64$
PLaT-2	$27.50\pm 0.39$	$86.11\pm 0.79$	$91.94\pm 1.96$	$95.00\pm 0.00$
PLaT-5	$78.61\pm 0.39$	$95.83\pm 0.39$	$97.50\pm 0.39$	$98.33\pm 0.00$
PLaT-8	$84.63\pm 1.16$	$96.85\pm 0.85$	$97.41\pm 1.28$	$98.70\pm 0.85$
PLaT-8, $L=2$	$93.61\pm 0.40$	$98.89\pm 0.78$	$99.44\pm 0.00$	$99.44\pm 0.00$

Appendix

Table 8: Llama-1B MultiArith ablations (%).

Human label	Valid	Invalid	Ambiguous
Valid	170	9	2
Invalid	5	14	0

Appendix

Table 9: Human versus GPT labels on all 200 reviewed items. Columns are GPT labels.

Metric	Cohen’s Kappa ( $\kappa$ )	Accuracy
Value	0.5943	0.9200

Appendix

Table 10: Human-LLM Agreement Metrics.

Figure 1: Confusion matrix of Human-LLM agreement. All 200 reviewed items are included. The two ambiguous LLM judgments count as disagreements.

Figure 2: Visualization of PLaT’s reasoning process. GT , Deterministic , and Sample are the ground-truth reasoning steps in the dataset, the deterministic readouts from each group of states, and sampled readouts from each group of states, respectively. This example is drawn from the GSM8K test set, with two stored sampled trajectories. State labels are one-based. Gray boxes preserve raw intermediate fragments without individual validity labels; green and red final-answer boxes indicate correctness and incorrectness. Arrows connect consecutive readouts within each stored trajectory.

Figure 3: Evolution of Token Distribution Entropy over Normalized Reasoning Progress. Updated PLaT-8 first-token decoder entropy (1,318 of 1,319 questions with retained reasoning states, solid red) with original CoT-SFT, Coconut, and CODI measurements (1,319 questions each, dashed). Each question contributes equally after interpolation (20 progress points for PLaT; 21 for the original baselines). Entropy is measured in nats; baseline model outputs and method-specific probes are retained from the original analysis.

Figure 4: Sensitivity to token-group size

K

with one latent vector per segment. Single-run results on four datasets; no mean or standard deviation across repeated runs is reported.

Figure 5: Impact of latent vectors per segment

L

at fixed token-group size

K=8

. Single-run results on four datasets, without repeat-run error bars.

Explore similar work

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate
Jun 15, 2026Hanyu Lin, Min Cai, Jiawei Wen +1Chain-of-Thought Reasoning
1Shenzhen University · University of Alberta
Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
Apr 27, 2026Dong Liu, Yanxuan Yu, Ying Nian WuLarge Language Model Planning Large Language Model Reinforcement Learning
University of California, Los Angeles · Columbia University
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
May 25, 2026Hui Xie, Jie Liu, Ziyue Qiao +1Latent Thoughts Chain-of-Thought Reasoning
Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China

Jun 15, 2026, cs.CLJ/K move · Enter open · S save

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

Hanyu Lin, Min Cai, Jiawei Wen, Haodi Zhang

1Shenzhen University · University of Alberta

Chain-of-Thought Reasoning

Open paper AskPDF arXiv

Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose \textbf{Ty}ped \textbf{L}at\textbf{e}nt \textbf{R}easoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.

Abstract

Explore similar work

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains