SYNLAT: Syntax-Aligned Text-Latent Compression for Chain-of-Thought Reasoning

Yifeng Zhao; Hongjun Yu; Shibo Wang; Yunjiao Zhou; Zixiao Zhu; Zhipeng Ning; Kezhi Mao; Junlang Qian

cs.CLOct 2, 2026

SYNLAT: Syntax-Aligned Text-Latent Compression for Chain-of-Thought Reasoning

Authors: Yifeng Zhao, Hongjun Yu, Shibo Wang, Yunjiao Zhou, Zixiao Zhu, Zhipeng Ning, Kezhi Mao, Junlang Qian

Organizations: Nanyang Technological University · Nankai University

Abstract

Long chain-of-thought (CoT) traces impose substantial output-token costs. Under constrained budgets, compression must preserve answer-critical information, making boundary placement central. Token-level and fixed-length boundaries can fragment coherent spans such as phrases, formulas, and local derivations, whereas step-level boundaries can bind content requiring different compression actions. We introduce SynLat, a text-latent CoT framework that aligns compression boundaries with syntactic structure through non-overlapping Syntax-Aligned Units (SAUs). An answer-conditioned Teacher constructs progressive KEEP/LATENT targets for a single compression-conditioned Student, which generates mixed reasoning from only the question and requested compression level at inference. Across two Qwen3 Student scales, Standard-CoT and Long-CoT groups, and three compression levels, SynLat matches or exceeds the strongest evaluated baseline in all 12 task-group aggregates and strictly leads in 11 under the reported achieved-CR selection protocol. Overall gains reach 3.6/2.6 points at MEDIUM and 7.0/5.5 points at HIGH for Qwen3-8B/14B, with larger advantages under stronger compression, particularly on Long-CoT groups.

Figures & tables

Figure 1: Compression-decision boundaries for the same reasoning sentence. Token and fixed-window units can fragment local relations, whereas step-level units can bind distinct relations; Syntax-Aligned Units preserve coherent relations while keeping them separately assignable to text or latent form.

Figure 2: Overview of SynLat. Offline target construction converts a reference trace into mixed-CoT targets through SAU construction, KEEP/LATENT allocation, and latent-code assignment; at deployment, the Student uses only the question and compression condition.

(a) Qwen3-8B Student
		Standard-CoT			Long-CoT
Setting	Method	MATH-500 Acc./CR	StrategyQA Acc./CR	Avg. Acc./CR	OlympiadBench Acc./CR	WorldTree V2 Acc./CR	ProofWriter Acc./CR	Avg. Acc./CR	Overall Acc./CR
Reference	No CoT	64.1/100.0	64.8/100.0	64.5/100.0	12.6/100.0	68.9/100.0	75.4/100.0	52.3/100.0	57.2/100.0
	Raw Full CoT	87.2/–	75.0/–	81.1/–	49.4/–	83.3/–	92.7/–	75.1/–	77.5/–
	SynLat (NONE)	87.8/0.0	75.4/0.0	81.6/0.0	49.8/0.0	83.6/0.0	93.1/0.0	75.5/0.0	77.9/0.0
LOW	TokenSkip	84.9/23.5	72.8/21.5	78.9/22.5	44.5/25.4	80.3/22.8	89.4/23.0	71.4/23.7	74.4/23.2

Table 1: Main results. Each cell reports answer accuracy and achieved compression rate (Acc./CR); Avg. and Overall are macro averages. Bold accuracy is best within each compression setting, and bold method names indicate SynLat. Raw Full CoT has no normalized CR;

\dagger

marks the nearest point outside the strict budget-matching window.

Figure 3: Syntactic constituency parsing and SAU construction. A reasoning sentence is parsed into nested contiguous spans, then partitioned into non-overlapping Syntax-Aligned Units for text–latent allocation.

	Unit characteristics			End-to-end result
Compression scheme	Avg. len.	Const. split $\downarrow$	Formula split $\downarrow$	Acc. ret. $\uparrow$	CR
Token Deletion	1.0	76.8	69.5	89.1	44.1
5-Token	4.7	35.4	41.6	91.4	44.6
5-Token & Guard	4.6	28.9	15.7	92.2	44.5
Step	18.6	5.7	7.1	89.6	43.9
SAU (Ours)	3.6	8.1	5.9	94.0	44.3

Table 2: Compression-unit comparison at comparable achieved CRs. Only Token Deletion also changes representation. Acc. Ret. (%) is relative to each row’s NONE accuracy; Avg. Len. counts tokens per unit. Const./Formula Split (%) measures constituent/formula spans split by unit boundaries; CR (%) is achieved compression rate.

CoT Compression

Chain-of-thought compression reduces the length or computation cost of model reasoning while aiming to preserve answer quality.

See all 57 papers

SCA: Segment-Wise CoT Compression with Answer Alignment
Mar 8, 2026Ye Tian, Hongyu LinCoT Compression Efficient Language Model Reasoning
HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
Jun 2, 2026Xin Liu, Runsong Zhao, Xinyu Liu +8Memory-Augmented Language Models Efficient Language Model Reasoning
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · China Unicom Cloud-Link, China · NiuTrans Research, Shenyang, China
A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM
Sep 7, 2026Xiaoang Xu, Siyuan Liu, Shuo Wang +13Efficient Language Model Reasoning Continuous Latent Reasoning
Beijing University of Posts and Telecommunications · OpenBMB · The Hong Kong Polytechnic University +2

Mar 8, 2026J/K move · Enter open · S save

SCA: Segment-Wise CoT Compression with Answer Alignment

Ye Tian, Hongyu Lin

CoT Compression Efficient Language Model Reasoning LLM Compression

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning improves problem solving, but long think traces increase inference cost. Existing CoT compression methods usually optimize completion-level length. For structured thinking models, however, a completion contains both a think segment and an answer segment, so completion-level compression can save tokens by compressing not only the CoT but also the answer. We call this failure mode answer drift. We propose Segment-wise CoT Compression with Answer Alignment (SCA), an answer-preserving think-compression method. SCA parses completions into functional segments, routes compression rewards only to successful think tokens, and protects answer tokens through length and distribution alignment to a frozen base model. Experiments show that, across datasets from multiple domains, SCA achieves state-of-the-art-level chain-of-thought compression while preserving the base model's performance and answer alignment. Training data and code are included in the supplementary code and data package.

Abstract

CoT Compression

Explore similar work

SCA: Segment-Wise CoT Compression with Answer Alignment

HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps

A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM