Why Pay for Long Reasoning When Rewrite Do Trick? Causal Effects of Reasoning History Compaction on Future Generation and Agent Cost
Organizations: Technological School Electronic Systems
Abstract
Reasoning substantially improves language models' ability to solve difficult problems, but long reasoning traces make these gains expensive. Efficient reasoning should express useful analysis in fewer tokens by reducing repetition and text that adds little value to solving the problem. Progress in proprietary models motivates this goal, yet their hidden scratchpads limit direct study of how it is achieved. We investigate whether open-weight models can generate less verbose reasoning by changing the form of their own earlier thoughts, without retraining. Our intervention rewrites prior reasoning into compact state records before pending tool results are revealed, then compares continuations from original and compact histories. Across the tested models, accepted compact histories usually shorten future reasoning; a coding case study also shows less repeated verification after a shared candidate solution. In software-agent deployments, compaction can reduce total spending, although task quality and rewrite-inclusive time depend on the target and compressor. These findings identify reasoning history as one way to reduce future reasoning-token use and motivate model-specific compaction policies evaluated for both efficiency and retained capability.
Figures & tables
| Route | Sources retained | Compact shorter | Median [95% CI] | Success clean compact |
|---|---|---|---|---|
| DeepSeek V4 Flash | 12/12 | 30/36 | 36/36 35/36 | |
| Laguna S 2.1 | 12/12 | 32/36 | 36/36 35/36 | |
| MiniMax M3 | 10/12 | 25/30 | 30/30 30/30 | |
| GLM 4.7 Flash | 5/12 | 13/13 | 11/15 14/15 | |
| GLM 5.1 | 10/12 | 28/30 | 25/30 25/30 | |
| DeepSeek V4 Pro | 9/12 | 23/27 | 27/27 27/27 |
| Batch | Rater | Pass | Problem | Unsure |
|---|---|---|---|---|
| Easy | A | 6 | 0 | 0 |
| Easy | B | 4 | 1 | 1 |
| Longer | A | 6 | 0 | 0 |
| Longer | B | 5 | 1 | 0 |
| Unique reasoning-history blocks | Change from Normal (%) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Target | Compressor | Score | Raw words | Compact words | Saved words | Reduction | Spend | Tool calls | Target time ∗ |
| DeepSeek V4 Flash | External | 32/50 | 168,611 | 83,851 | 84,760 | 50.3% | |||
| Self | 29/50 | 158,476 | 68,423 | 90,053 | 56.8% | ||||
| MiniMax M3 | External | 31/50 | 146,126 | 55,901 | 90,225 | 61.7% | |||
| Self | 30/50 | 131,026 | 49,662 | 81,364 | 62.1% | ||||
| Nemotron 3 Ultra | External | 32/50 | 129,834 | 74,037 | 55,797 | 43.0% | |||
| Compressor effort | Time (s) | Sol Pass / 20 |
|---|---|---|
| Historical xhigh | 90.1 | 12; 13 |
| None | 1.7 | 11 |
| Low | 9.1 | 10 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Original reasoning (152 words) | Compact reasoning (41 words) |
| Let me check the hard constraints: 1. binding_quote_usd <= 1,000,000 - all options have estimated_usd below 1M, but these are estimates, not binding quotes. The hard constraint says "binding_quote_usd_lte" - so we need binding quotes to check this. But we don’t have binding quotes yet. Let me proceed. 2. minimum_sites >= 2 - V1 has 1 site, so V1 violates this constraint. V2 (3), V3 (2), V4 (3), V5 (2) all satisfy. 3. reliability_gte >= 0.94 - we don’t have reliability data yet. 4. risk_lte <= 0.05 - we don’t have risk data yet. So we need to simulate to get more data. Let me proceed to simulate. But first, I need to know which options survive static checks. V1 is out due to sites=1 < 2. So the remaining options are V2, V3, V4, V5. But I need exactly 4 option_ids for simulate_vaccine_options. Let me try with V2, V3, V4, V5. | Binding quote \geq\geq\leq$ 0.05 lack data. Simulate the four surviving options. Requested tool: simulate_vaccine_options({"case_id":"VAX-28","option_ids":["V2","V3","V4","V5"]}) |
| First-attempt captures | Retry-augmented captures | |||
| Assigned-cohort quantity | Clean | Compact | Clean | Compact |
| Reasoning tokens | 152,475 | 94,357 | 135,778 | 97,790 |
| Accumulated input tokens | 1,399,681 | 1,259,755 | 1,429,111 | 1,277,229 |
| Completion tokens | 234,817 | 159,887 | 220,049 | 163,876 |
| Continuation turns | 515 | 518 | 521 | 520 |
| Exact failures | 12 | 9 | 9 | 8 |
| Omitted route | Shorter/comparable | Median | Median token | Success C K | Quality (pp) |
|---|---|---|---|---|---|
| DeepSeek V4 Flash | 121/136 | 129 131 | |||
| Laguna S 2.1 | 119/136 | 129 131 | |||
| MiniMax M3 | 126/142 | 135 136 | |||
| GLM 4.7 Flash | 138/159 | 154 152 | |||
| GLM 5.1 | 123/142 | 140 141 | |||
| DeepSeek V4 Pro | 128/145 | 138 139 |
| Route | Shorter | Median | Success C K |
|---|---|---|---|
| DeepSeek | 15/18 | 18/21 18/21 | |
| Laguna | 14/15 | 15/15 15/15 | |
| Combined | 29/33 | 33/36 33/36 |
| Target | Compressor | Gain/loss | Accuracy (pp) | Paired-bootstrap 95% CI (pp) |
|---|---|---|---|---|
| DeepSeek V4 Flash | External | 1/1 | ||
| Self | 0/3 | |||
| MiniMax M3 | External | 3/2 | ||
| Self | 3/3 | |||
| Nemotron 3 Ultra | External | 3/0 | ||
| Self | 2/2 |
| Target | Arm | Spend ($) | Target (h) | Rewrite (h) | Inclusive (h) | Inclusive (%) |
|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | Normal | 2.43 | 3.37 | 0.00 | 3.37 | — |
| External | 1.59 | 2.22 | 1.23 | 3.45 | ||
| Self | 1.30 | 1.91 | 3.94 | 5.84 | ||
| MiniMax M3 | Normal | 17.72 | 23.90 | 0.00 | 23.90 | — |
| External | 11.70 | 8.86 | 0.90 | 9.76 | ||
| Self | 10.95 | 9.38 | 1.44 | 10.82 |
| Effort | Time (s) | Reasoning | Shorter | Sol Pass |
|---|---|---|---|---|
| xhigh | 90.08 | 10,536 | 18 | 12; 13 |
| none | 1.69 | 0 | 18 | 11 |
| low | 9.06 | 275.5 | 16 | 10 |
| Regime (background) | Clean (s) | External | Self |
|---|---|---|---|
| Dedicated (0) | 45.821 | ||
| Shared (20) | 73.464 | ||
| Overloaded (61) | 154.469 |
Efficient Language Model Reasoning
Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.
See all 393 papers