Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression

Chain-of-thought (CoT) reasoning allows humans to inspect how large language models reach their answers, and oversee model behaviour. This reasoning comes at an increased inference cost, motivating efficient methods that train models to solve tasks using fewer tokens. However, a common concern is that such training may cause models to skip important reasoning steps, so the CoT no longer faithfully reflects the model's decision. It is unclear whether or when this occurs in practice, since different efficiency methods apply length pressure to models' CoT in distinct ways, and faithfully explaining a model's decision takes more tokens on some tasks than others. To understand these dynamics, we fine-tune a variety of models with three methods that apply length pressure differently, namely a fixed generation budget, a per-example length target, and a group-relative length reward. We evaluate how efficient reasoning affects CoT faithfulness (i.e., how well the CoT reflects model decisions on related inputs) and monitorability (i.e., whether the CoT reveals when input interventions alter the output). We find that it affects faithfulness and monitorability differently. Faithfulness falls in most settings, primarily because the trained models are less consistent. Monitorability is more robust, as models keep acknowledging the influence on their answer even when the CoT is much shorter.

Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference cost. Existing CoT compression methods fall into two families, selective pruning and generative rewriting, yet prior studies have left key factors entangled: granularity is confounded with importance criteria in pruning, restructuring level is rarely isolated in rewriting, and compression budgets are not systematically evaluated across domains or regimes. We recast CoT compression along three dimensions: importance criterion, restructuring level, and compression budget. Sweeping these across two model families, Math and General domains, and Long-/Short-CoT regimes, we find that (i) importance criterion utility is strictly governed by granularity: step-level criteria converge on a shared reasoning backbone, while token-level pruning requires symbol-aware signals to preserve the logical core; (ii) restructuring level inverts across domains: Math degrades monotonically with structural disruption, while aggressive rewriting acts as a denoiser on General tasks; (iii) training-time compression does not necessarily translate to inference-time savings: Long-CoT students retain verbose habits despite concise supervision, making the training ratio an optimistic lower bound on deployment cost. These findings yield condition-aware guidelines for matching compression to deployment context.

Long Chain-of-Thought (CoT) traces can improve reasoning accuracy, but repeatedly generating them is costly for smaller or latency-constrained language models. This paper studies a practical alternative: produce a rich rationale once with a capable thinking model, compress it, and reuse the compressed trace as context for a cheaper answering model. We introduce CoT-X, an adaptive framework for cross-model CoT transfer. CoT-X segments reasoning traces into semantic units, scores their diagnostic and logical importance, selects budget-feasible evidence paths, and reconstructs a coherent compressed rationale for the answering model. On $7,501$ Japanese medical licensing questions spanning $10$ specialties, CoT-X improves accuracy over direct truncation by up to $40.5\%$ under the same token budget, with the largest gains at $64$--$256$ tokens. Across $64$ thinking--answering pairs from eight DeepSeek-R1 and Qwen3 models (1.5B--32B parameters), reasoning transfer is most reliable within a model family, yet remains effective across families once compression normalizes the trace. A Gaussian Process Bayesian optimization layer finds near-optimal model--budget configurations with $15$ evaluations rather than an exhaustive search over all $64$ pairs, reducing evaluation cost by $84\%$. These results show that reasoning quality, token budget, and model compatibility can be optimized jointly, making CoT-style reasoning more practical under realistic deployment constraints.