LightThinker++: From Reasoning Compression to Memory Management

Yuqi Zhu; Jintian Zhang; Zhenjie Wan; Yujie Luo; Shuofei Qiao; Zhengke Gui; Da Zheng; Lei Liang

cs.CLApr 4, 2026

LightThinker++: From Reasoning Compression to Memory Management

Authors: Yuqi Zhu, Jintian Zhang, Zhenjie Wan, Yujie Luo, Shuofei Qiao, Zhengke Gui, Da Zheng, Lei Liang, +2 more

Organizations: Zhejiang University · Ant Group

Abstract

Large language models (LLMs) excel at complex reasoning, yet their efficiency is limited by the surging cognitive overhead of long thought traces. In this paper, we propose LightThinker, a method that enables LLMs to dynamically compress intermediate thoughts into compact semantic representations. However, static compression often struggles with complex reasoning where the irreversible loss of intermediate details can lead to logical bottlenecks. To address this, we evolve the framework into LightThinker++, introducing Explicit Adaptive Memory Management. This paradigm shifts to behavioral-level management by incorporating explicit memory primitives, supported by a specialized trajectory synthesis pipeline to train purposeful memory scheduling. Extensive experiments demonstrate the framework's versatility across three dimensions. (1) LightThinker reduces peak token usage by 70% and inference time by 26% with minimal accuracy loss. (2) In standard reasoning, LightThinker++ slashes peak token usage by 69.9% while yielding a +2.42% accuracy gain under the same context budget for maximum performance. (3) Most notably, in long-horizon agentic tasks, it maintains a stable footprint beyond 80 rounds (a 60%-70% reduction), achieving an average performance gain of 14.8% across different complex scenarios. Overall, our work provides a scalable direction for sustaining deep LLM reasoning over extended horizons with minimal overhead.

Figures & tables

Figure 1: An illustration of the compressed reasoning paradigms. (a) A CoT example. Tokens highlighted in yellow represent critical reasoning tokens, while the remaining tokens primarily ensure fluency. (b) The Vanilla approach generates full thought tokens. (c) LightThinker compresses each thought into a concise representation ( $C\ T_{i}$ ). (d) LightThinker++ further incorporates explicit memory management to handle summaries of thoughts, enhancing reasoning efficiency and coherence.

Figure 4: Overview of LightThinker++. a) Memory Action Space : Reasoning steps are instantiated as dual-form entities

\mathcal{I}_{i}=(R_{i},Z_{i})

. The visibility state of each step is explicitly managed via commit , expand , and fold primitives. b) Inference Overview : An illustration of the step-wise inference process.

\tilde{\mathcal{H}}_{t}

denotes the stateful managed context where historical steps are dynamically projected as either summaries (

Z_{i}

, marked with

<

) or raw derivations (

R_{i}

, marked with

\vee

) based on the model’s self-directed memory policy.

Qwen2.5-7B Series
Method	GSM8K				MMLU				GPQA				BBH				AVG.
Method	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$
CoT	86.12	1.66	513	0.1M	66.50	1.77	649	0.2M	26.76	0.60	968	0.5M	65.45	0.68	570	0.1M	61.21	1.18	675	0.2M
Distill-R1	81.88	5.60	844	1.1M	51.70	14.31	2483	7.5M	24.75	8.01	6718	31M	57.78	5.53	1967	6.0M	54.03	8.36	3003	11.3M
Vanilla	90.90	11.83	2086	3.9M	59.98	20.61	3417	10M	30.81	10.76	8055	39M	69.90	11.50	3786	13M	62.90	13.68	4336	16.6M
+ H2O	89.92	22.19	640	1.2M	59.69	29.02	1024	3.2M	24.75	15.61	1200	9.8M	70.10	15.61	1024	3.5M	61.12	20.61	972	4.4M

Table 1: Main results of LThinker. The CoT is based on the instruction model, while Vanilla, AnLLM, and LightThinker are based on Distill-R1. The light blue background indicates acceleration methods, with bold representing the best and underline the second best among them. The Acc of Vanilla serves as the upper bound for Acc of acceleration methods. Dep is measured in million, Time in hours, and Peak in counts. The compression ratio can be roughly estimated by the ratio of Dep between acceleration methods and Vanilla. See Appendix A for more details. Note that the results here are based on the same batch size. The results under the same memory budget are shown in Table 3 .

Figure 5: Efficiency Analysis and Ablation Results. (a) shows the average number of generated tokens for each model on each dataset. (b) shows the distribution of token lengths across ranges, while the cumulative curve indicates the overall proportion up to each range. (c) illustrates the relationship between output length and inference time, with each subplot reporting inference time and peak token count. (d) reports the average compression ratios, with error bars showing 95% confidence intervals. (e–f) examine how cache size $|C|$ affects accuracy, Dep, inference time, peak tokens, generated tokens, and compression frequency.

Table 6

	GSM8K	MMLU	GPQA	BBH	AVG
AnLLM	78.39	54.63	19.70	54.95	51.92
Ours (\|C\|=1, T)	78.32	58.23	20.71	55.35	53.15
Ours (\|C\|=1, F)	80.21	58.23	22.22	62.02	55.67

Table 4: Ablation results on Qwen for accuracy across four datasets. “T” refers to AnLLM’s attention-mask mechanism, whereas “F” refers to LThinker’s attention-mask mechanism.

Figure 6: Case study. The figure shows a partial inference trace for one GSM8K example. The full example is provided in App. C.1.6 . Pink and light blue backgrounds indicate alternating compression steps, with each color corresponding to one compression.

Qwen2.5-7B Series
Method	GSM8K				MMLU				GPQA				BBH				AVG.
Method	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Time $\downarrow$	Peak $\downarrow$	Dep $\downarrow$
CoT	86.12	99.6	513	0.1M	66.50	106.2	649	0.2M	26.76	36.0	968	0.5M	65.45	40.8	570	0.1M	61.21	70.8	675	0.2M
Distill-R1	81.88	336.0	844	1.1M	51.70	858.6	2483	7.5M	24.75	480.6	6718	31M	57.78	331.8	1967	6.0M	54.03	501.6	3003	11.3M
Vanilla	87.34	12.7	812	1.1M	55.27	33.1	2682	7.7M	34.34	15.3	5781	21.9M	61.55	24.7	3205	10.2M	59.62	21.5	3120	10.3M
TokenSkip	87.92	47.8	775	1.0M	54.40	72.3	2831	9.1M	38.22	17.5	5611	21.2M	59.87	33.2	2850	9.3M	60.10	42.7	3017	10.2M

Table 5: Main results of LThinker++ ( Throughput setting ). Time reports the end-to-end wall-clock time in minutes (see Appx. C.2.5 for more). The CoT is based on the instruction model, while the other methods are compared within each corresponding model series. In this table, LThinker++ denotes our full method, and LThinker* denotes our variant with only the commit action enabled.

Qwen2.5-7B Series
Method	GSM8K			MMLU			GPQA			BBH			AVG.
Method	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$
CoT	86.12	513	0.1M	66.50	649	0.2M	26.76	968	0.5M	65.45	570	0.1M	61.21	675	0.2M
Distill-R1	81.88	844	1.1M	51.70	2483	7.5M	24.75	6718	31M	57.78	1967	6.0M	54.03	3003	11.3M
Vanilla	87.62	782	1.1M	57.58	2432	6.5M	32.49	5493	20.0M	62.76	2714	8.2M	60.11	2855	8.9M
TokenSkip	87.92	775	1.0M	54.40	2831	9.1M	38.22	5611	21.2M	59.87	2850	9.3M	60.10	3017	10.2M

Table 6: Main results of LThinker++ (Budget setting). Note that the results here are based on the same batch size. Other settings and metric definitions are the same as Table 5 .

Figure 7: Efficiency Analysis and Ablation Results under the Throughput setting. (a) illustrates the average number of generated tokens retained in the current context. (b) presents the distribution ratio of memory actions ( Commit vs. Expand+Fold ) for Qwen-2.5-7B. (c) shows the percentage of pre-commit token lengths falling within specified ranges on GPQA(Qwen-2.5-7B). (d) examines the scaling of peak memory tokens as a function of the generation budget. (e) summarizes the average compression factors and saved token ratios across different model series. (f) provides an ablation study comparing accuracy and average peak tokens between the full LThinker++ and its degraded variants.

Figure 8: Case Study. The figure illustrates partial inference results of a case of LThinker++ from GSM8K.

Qwen2.5-7B Series
Method	GSM8K			MMLU			GPQA			BBH			AVG.
Method	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$	Acc $\uparrow$	Peak $\downarrow$	Dep $\downarrow$
CoT	86.12	513	0.1M	66.50	649	0.2M	26.76	968	0.5M	65.45	570	0.1M	61.21	675	0.2M
Distill-R1	81.88	844	1.1M	51.70	2483	7.5M	24.75	6718	31M	57.78	1967	6.0M	54.03	3003	11.3M
$\mathrm{LThinker}_{\text{tho1}}$	84.61	1043	2.1M	52.96	2792	8.4M	18.69	6781	29.7M	53.54	3687	13.8M	52.45	3576	13.5M
$\mathrm{LThinker}_{\text{tho2}}$	85.75	1003	1.8M	52.00	2822	8.23M	24.24	7042	30.8M	51.31	4130	16.3M	53.33	3749	14.3M

Table 7: Comparison between LThinker and LThinker++ on four benchmarks. For a fair comparison,

\mathrm{LThinker}_{\text{tho1}}

and

\mathrm{LThinker}_{\text{tho2}}

are trained on the same training data as in Table 5 .

Figure 9: Thought segment length distribution. Kernel density estimates of per-thought segment lengths (in characters) for LThinker_tho , LThinker_tho1 , and LThinker_tho2 . The x-axis is the segment length (characters), and the y-axis is probability density. Dashed vertical lines indicate the peak locations.

Figure 10: Overview of LightThinker++ for long-horizon agentic reasoning. LightThinker++ follows the Thought–Action–Observation loop while explicitly managing a stateful memory. Each interaction step can be committed into an archived summary, expanded back into the active context to recover raw evidence when needed, and folded again to maintain context hygiene.

Proprietary Agents
Method	xBench-DeepSearch	BrowseComp-ZH	BrowseComp-EN
GPT-5	66.0	61.3	61.5
Claude-4-Sonnet	35.0	29.1	12.2
DeepSeek-V3.2	51.0	53.6	35.0
DeepSeek-V3.1	44.0	49.5	23.6
GLM-4.6	47.0	42.2	34.9

Table 8: Overall results on BrowseComp benchmarks across three datasets. We report Pass@1 for all methods, and additionally report Pass@3 (in gray parentheses) for our agents to reflect multi-attempt robustness.

Our Agents
Method	xBench-DeepSearch ( $hard_{01}$ )		BrowseComp-ZH ( $hard_{01}$ )		BrowseComp-EN ( $hard_{01}$ )
Method	Pass@1 (%) $\uparrow$	Pass@3 (%) $\uparrow$	Pass@1 (%) $\uparrow$	Pass@3 (%) $\uparrow$	Pass@1 (%) $\uparrow$	Pass@3 (%) $\uparrow$
Vanilla SFT	6.8	20.3	8.6	25.9	5.1	15.4
LightThinker++	20.9	33.9	20.6	40.3	10.5	22.4

Table 9: Performance comparison on the

hard_{01}

subset across three benchmarks (in %).

Figure 11: Quantitative Analysis of Context Management Efficiency. Fig.(a) illustrates the active context trajectories (

C_{t}

) across interaction rounds, highlighting the stable footprint of LThinker++ compared to the rapid inflation of the Vanilla baseline. Fig.(b) provides a statistical comparison of Mean and P95 (peak) token usage across tasks, demonstrating significant reductions in memory overhead.

Figure 12: Quantitative Analysis of Context Management Efficiency. Fig.(a) illustrates the active context trajectories ( $C_{t}$ ) across interaction rounds, highlighting the stable footprint of LThinker++ compared to the rapid inflation of the Vanilla baseline. Fig.(b) provides a statistical comparison of Mean and P95 (peak) token usage across tasks, demonstrating significant reductions in memory overhead.

Figure 13: Ablation results of memory orchestration.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 14: An illustration of the Dependency metric.

Figure 15: An illustration of the attention mask used in Table 4 .

Figure 16: Average number of generated tokens.

Query Source	Count
HotpotQA + MuSiQue (Filtered by Qwen3)	5,322
WebDancer	200
WebShaper	500
WebWalkerQA-Silver (EN + Hard ZH)	2,932
Total Base Queries	8,954

Appendix

Table 10: Breakdown of the Base Query Pool for Agentic Research Synthesis.

Panel of Figure 17 — Figure 17: Synthesis prompt used for generating reasoning data.

Panel of Figure 18 — Figure 18: Synthesis prompt used for agentic research trajectory generation and filtering.

Panel of Figure 19 — Figure 19: Task prompt for Vanilla , LightThinker++ in general reasoning tasks for both Qwen-based model and Llama-based model.

Figure 20: Efficiency Analysis and Ablation Results under the Budget setting. Fig.(a) illustrates the average number of generated tokens retained in the current context. Fig.(b) presents the distribution ratio of memory actions ( Commit vs. Expand+Fold ) for Qwen-2.5-7B. Fig.(c) shows the percentage of pre-commit token lengths falling within specified ranges on GPQA(Qwen-2.5-7B). Fig.(d) examines the scaling of peak memory tokens as a function of the generation budget. Fig.(e) summarizes the average compression factors and saved token ratios across different model series. Fig.(f) provides an ablation study comparing accuracy and average peak tokens between the full LThinker++ and its degraded variants.

Figure 21: Comparison between AnLLM and LightThinker. The two main differences are highlighted with a red box and with blue/pink arrows.

Figure 22: Token compression frequency distribution for LightThinker.

Panel of Figure 23 — Figure 23: System prompt for Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct .

Panel of Figure 24 — Figure 24: System prompt for Vanilla , H2O , SepLLM , AnLLM , and LightThinker for both Qwen-based model and Llama-based model.

Panel of Figure 25 — Figure 25: Task prompt for Distill-R1 , Vanilla , H2O , SepLLM , AnLLM , and LightThinker for both Qwen-based model and Llama-based model.

Panel of Figure 26 — Figure 26: Task prompt for Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct .

Panel of Figure 27 — Figure 27: A training case. <w> is a compression trigger.

Panel of Figure 28 — Figure 28: Bad Case. <|splitter|> is equal to <w> in Figure 27 .

Method	Memory Optimization	Training Cost	Interpretability	Generalization
System-1.5 [ 76 ]	Significant	High	Weak	Weak
SoftThinking [ 77 ]	Limited	None	Weak	Good
LightThinker	Significant	Low	Good	Good

Appendix

Table 11: Comparison of different methods.

Figure 29: Case Study: Memory Orchestration for Long-Horizon Reasoning.

Explore similar work

Training Large Reasoning Models Efficiently via Progressive Thought Encoding
Feb 18, 2026Zeliang Zhang, Xiaodong Liu, Hao Cheng +3Efficient Language Model Reasoning LLM Reasoning
University of Rochester · Microsoft Research
ActiveMem: Distributed Active Memory for Long-Horizon LLM Reasoning
Jun 9, 2026Yunhan Jiang, Wenbin Duan, Shasha Guo +3Long-Horizon LLM Agents Agent Memory Management
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
May 25, 2026Hui Xie, Jie Liu, Ziyue Qiao +1Efficient Language Model Reasoning LLM Inference Acceleration
Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China

Feb 18, 2026J/K move · Enter open · S save

Training Large Reasoning Models Efficiently via Progressive Thought Encoding

Zeliang Zhang, Xiaodong Liu, Hao Cheng, Hao Sun+2

University of Rochester · Microsoft Research

Efficient Language Model Reasoning LLM Reasoning Large Reasoning Models Parameter-Efficient Fine-Tuning RL for Language Models

Open paper AskPDF arXiv

Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rewards, where autoregressive decoding dominates time and memory usage. While sliding-window cache strategies can bound memory, they disrupt long-context reasoning and degrade performance. We introduce Progressive Thought Encoding, a parameter-efficient fine-tuning method that enables LRMs to reason effectively under fixed-size caches. By progressively encoding intermediate reasoning into fixed-size vector representations, our approach eliminates the need to backpropagate through full-cache rollouts, thereby reducing memory usage, while maintaining constant memory during inference. Experiments on three models, including Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Llama-8B, on six widely used challenging mathematical benchmarks show consistent gains: our method achieves +19.3% improvement over LoRA-based fine-tuning and +29.9% over LRMs without fine-tuning on average, with up to +23.4 accuracy improvement on AIME2024/2025 under the same tight cache budgets. These results demonstrate that Progressive Thought Encoding not only improves reasoning accuracy but also makes RL training of LRMs substantially more efficient and scalable under real-world memory constraints.

Abstract

Explore similar work

Training Large Reasoning Models Efficiently via Progressive Thought Encoding

ActiveMem: Distributed Active Memory for Long-Horizon LLM Reasoning

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains