Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoning
Authors: Zhuoyuan Hao, Zhuo Li, Wu Li, Fangming Liu, Min Zhang, Jing Li
Organizations: Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · Huazhong University of Science and Technology, China
Recent work improves reasoning in large reasoning models (LRMs) by test-time scaling through self-consistency and parallel thinking, adding generic "thinking tokens", or presenting the question twice, as in re-reading (RE2). However, these approaches either inject task-agnostic tokens or mandate heuristics that do not explain, and often ignore, the spontaneous repetition that many LRMs exhibit at the head of their long chain-of-thought (CoT). In this work, we analyze and harness the model's tendency to restate the question, dubbed Echo of Prompt (EOP), as a front-loaded, compute-shaping mechanism, which can be viewed as self-generated prompt repetition inside the reasoning trace rather than in the input. Specifically, we formalize its probabilistic cost by casting echo removal as rejection-based conditioning, and define the Echo Likelihood Gap ΔL as a computable proxy linking early repetition to likelihood gains and downstream accuracy. We find that (1) in a causal intervention, reinserting an echo into failed GSM8K traces raises exact match by 7.9-10.4 points for two reasoning models but not for a non-reasoning base model; and (2) in length- and suffix-controlled likelihood analyses and layer-wise attention studies conducted to show that EOP's benefits go beyond verbosity, EOP increases answer to answer-prefix attention in middle layers, consistent with an attention refocusing mechanism. Building on these findings, we develop Echo-Distilled SFT (ED-SFT) to instill an "echo-then-reason" pattern through supervised finetuning, and training-free Echoic Prompting (EP) to re-ground the model mid-trace. We evaluate on GSM8K, MathQA, Hendrycks-MATH, AIME24, and MATH-500 under identical decoding settings and budgets, and find consistent gains over baselines, including thinking-token insertion. Our code is publicly released at https://github.com/hhh2210/echoes-as-anchors.
Figures & tables
Figure 1: An illustration of the Echo of Prompt (EOP). Left: An example of a model’s thinking process starting with an echo of the user’s query. Right: The frequency of EOP across several open-source models on the GSM8K dataset, as measured by our trained MLP echo probe (§ 3.1 ).
Echo Likelihood Gap (per-token)
Extended Echo Metrics
Group
N
Mean ΔL
Std σ(ΔL)
Neg. ratio (%)
ΔL (per token)
Δ/#removed
Suffix-only gap
Avg. removed tokens
Correct
819
2.5231
0.7786
0.12
2.4614
0.01103
1.1449
219.7
Wrong
500
2.4421
0.7657
0.00
2.3666
0.01085
1.2938
218.7
Table 1: Echo metrics on GSM8K. Averages over samples in each group, where N denotes the number of samples. The Suffix-only Likelihood Gap measures the per-token log-likelihood difference on the reasoning suffix when conditioned with versus without the echo prefix.
Predictor
Coefficient ( β )
Std. Error
P-value
Intercept ( β0 )
0.12
0.15
0.423
Echo Likelihood Gap ( β1 )
0.24
0.11
0.022
Echo Length ( β2 )
0.001
0.0005
0.089
Table 2: Logistic regression results predicting correctness on GSM8K.
Group
N
Accuracy (%)
Last-layer Ans → Ans-prefix Attn
EOP-present
985
63.8
–
EOP-absent
334
57.2
–
Conditioned on Outcome:
Correct, EOP-present
628
–
0.1374
Correct, EOP-absent
191
–
0.1328
Wrong, EOP-present
357
–
0.1046
Table 3: Comparison of EOP-present and EOP-absent traces on GSM8K (DeepSeek-R1-Distill-Llama-8B). EOP presence is associated with higher accuracy and stronger attention refocusing.
Figure 2: Echo metrics on GSM8K. Left: High-resolution histogram of removed echo-prefix lengths (10-token bins) for correct and wrong traces; most mass lies between roughly 200 and 240 tokens. Right: Echo Likelihood Gap ΔL (per-token) stratified by removed-prefix length bin; the gap remains positive across all bins.
Metric
Correct
Wrong
Diff (C − W)
Global Statistics
Last-layer: answer → question
5.77%
5.54%
+0.23%
Last-layer: answer → answer-prefix
13.69%
10.41%
+3.28%
All-layers mean: answer → question
8.45%
8.00%
+0.45%
All-layers mean: answer → answer-prefix
10.64%
8.49%
+2.15%
Peak Effect Layers (7-18)
Table 4: Average attention weights (%) on GSM8K (DeepSeek-R1-Distill-Llama-8B) with probe-estimated prefix length. Global statistics and layer-specific analysis showing attention from answer tokens to question and answer prefix.
Figure 3: Layer-wise attention weight distribution on GSM8K (DeepSeek-R1-Distill-Llama-8B) for Left: answer → answer-prefix and Right: answer → question. The blue lines represent correct reasoning traces while orange lines represent incorrect ones. The attention refocusing effect is most pronounced in layers 7-18 for answer → answer-prefix, with correct traces maintaining consistently higher attention weights.
Layer Group (layers)
AUC ↑ (Ans → Pref)
d↑ (Ans → Pref)
AUC (Ans → Q)
d (Ans → Q)
Early (0–6)
0.716
0.820
0.628
0.482
Mid (7–18)
0.719
0.832
0.585
0.303
Late (19–31)
0.723
0.828
0.563
0.184
Table 5: Layer-wise discriminability (Correct vs. Wrong) aggregated into layer groups . Metrics are computed on answer → answer-prefix and answer → question attention. The mid-layer group shows the strongest effect size ( d ) for answer → answer-prefix.
Figure 4: Robustness of the answer → answer-prefix gap. Left: raw attention difference (Correct − Wrong) and normalized effect size (Cohen’s d ) across layers; the two track each other, and d exceeds 0.75 throughout layers 7–18. Right: attention to a fixed-length prefix of the first K answer tokens, averaged over all layers; the Correct − Wrong gap stays positive for every K .
Figure 5: Fine-grained attention evidence on GSM8K (DeepSeek-R1-Distill-Llama-8B). Top: token-wise attention from the first 32 answer tokens to the answer-prefix (left) and the question (right), with standard-error bands; Correct traces (blue) attend more to the prefix, Wrong traces (orange) more to the question. Bottom: word-level attention from a correct trace to its echo prefix on the “Janet’s ducks” problem (layers 7–18; darker is higher), concentrated on the key quantities.
Figure 6: Information flow visualization for generated answer tokens across three different examples. The graphs show the primary attention routes through the model layers. In correct traces, information flows significantly through the Echo of Prompt (EOP) tokens in the middle layers, acting as a bridge between the question and the answer.
Model
Echo-free EM (%)
Echo reinsertion EM (%)
DeepSeek-R1-Distill-Llama-8B
15.85
26.22
Qwen3-8B
21.34
29.27
Qwen3-8B-Base (no CoT)
10.56
10.56
Table 6: Echo reinsertion ablation on the wrong-subset GSM8K traces for several models.
Model
GSM-8K
MathQA
Hendrycks-MATH
Strict EM
Flex EM
Qwen3-8B-Base
Base
79.4
80.5
31.0
0.76
+ ED-SFT
94.2 (+3.4)
94.2 (+3.4)
58.8 (+11.8)
10.0 (+8.2)
+ normal-SFT
90.8
90.8
47.0
1.8
Qwen3-8B (Instruct Version)
Table 7: Supervised fine-tuning with distilled CoT data improves mathematical reasoning. -ED-SFT denotes models fine-tuned on our EOP distilled dataset. -normal-SFT refers to the normal CoT distilled dataset.Base is the pretrained model; the unmarked variant is instruction-tuned. Best scores in each column are in bold .
Figure 7: Echo-then-reason on a Hendrycks-MATH counting problem. Top: the question. Left (green): Qwen3-8B-ED-SFT restates the question first and answers correctly. Right (red): Qwen3-8B-Base jumps straight to a formula without echoing and answers incorrectly.
Figure 8: Echoic Prompting (EP) vs. TTTS on AIME24 (left) and MATH-500 (right).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Accuracy
Precision
Recall
F1-Score
AUROC
Value
0.912
0.921
0.908
0.914
0.963
Appendix
Table 8: MLP probe performance on the held-out test set. The probe reliably identifies echo prefixes, justifying its use in our framework.
Competition mathematics (algebra, geometry, number theory, probability, etc. )
“A new dataset of 12,500 challenging competition mathematics problems. Each problem in MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations.” The evaluation follows the hendrycks_math group in the LM Evaluation Harness, which decomposes MATH into diverse subsets ( _algebra , _counting_and_prob , _geometry , _intermediate_algebra , _num_theory , _prealgebra , _precalc ), covering a wide range of mathematical skills beyond our training distribution.
GSM8K
8.5K problems; free-form answers; word problems
Grade-school math word problems
“A dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution.”
MathQA
37K problems; multiple choice
Multiple math domains; derived from AQuA operation programs
“A large-scale dataset of 37k English multiple-choice math word problems covering multiple math domain categories by modeling operation programs corresponding to word problems in the AQuA dataset.”
Large reasoning models (LRMs) have achieved strong performance on challenging mathematical and scientific tasks through long chain-of-thought reasoning, but they often suffer from overthinking with unnecessary reasoning steps. Existing truncation-based methods typically detect consecutive trial-answer agreement and force early exit, yet such local stopping signals do not necessarily indicate final-answer stability and can disrupt useful subsequent reflection or verification, potentially leading to performance degradation. We propose consistency-aware logit masking (CALM), a training-free decoding intervention for mitigating overthinking in LRMs. CALM first monitors trial answers at step boundaries and then applies logit masking to action transition point (ATP) tokens such as "Wait", "Alternatively", and "Hmm", suppressing redundant transitional reasoning while allowing the model to continue its thought process more naturally. Experiments across four advanced LRMs and five benchmarks show that CALM reduces token consumption by 18.6%-25.0% compared with full CoT reasoning while maintaining performance, and can even improve accuracy by 0.07%-1.48% by reducing redundant reasoning. LLM-as-a-Judge results also demonstrate that CALM provides more coherent and concise results.
Shengjing Liu, Feiyang Li, Yuxuan Yang +2
College of Computer Science and Software Engineering, Shenzhen University
Chain-of-thought reasoning unfolds in discrete token space: each step is committed as text, errors propagate, and eliciting good traces presupposes traces to imitate. Reasoning instead in a model's continuous representation space - where intermediate states are vectors rather than words - sidesteps these constraints, but leaves open how those latent states should be computed. We approach this along two axes. First, we keep a large language model (LLM) frozen and use it for what it is already good at - modeling and decoding sequences - while a small auxiliary network supplies continuous latent thoughts as input. Second, we produce those latents by recurrence: a tiny recurrent reasoner refines them over many steps, decoupling the depth of computation from the size of the model, so that the latents are a product of iterative processing rather than a single forward pass. We instantiate this as Latent Recurrent Thoughts (LRT): a task-dedicated proposer supplies base latents, a recurrent reasoner refines them through bounded residual corrections, and the frozen LLM decodes the answer. On symbolic reasoning with answer supervision but no reasoning traces (Countdown-4, Sudoku) and on natural-language reasoning (HumanEval, MBPP, StrategyQA), LRT substantially outperforms prior frozen-decoder continuous-space reasoning methods under an identical decoder, prompt, data, and training budget, and outperforms non-thinking-mode chain-of-thought prompting on the same backbone at a small fraction of its inference compute.
Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting. However, these models often exhibit "computational overthinking," generating redundant reasoning steps that increase latency and cost without improving accuracy. Recent studies suggest that CoT trajectories can be significantly pruned, yet existing methods often rely on forcing a static thinking budget, heuristic filtering, sub-optimal early exit via classification, or expensive re-training. In this paper, we introduce OS-Pruner, a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem. Given a reasoning prefix, OS-Pruner learns whether further reasoning is worth its token cost by optimizing an explicit utility that trades off final-answer accuracy against generated length. Our novel formulation enables the model to dynamically assess the sufficient point of termination for a reasoning chain. OS-Pruner is designed to be lightweight during both training and inference, and to provide users with fine-grained control over the reasoning-effort vs. accuracy trade-off. On diverse reasoning benchmarks and base models, OS-Pruner achieves 20-60% reduction in generation length with minimal accuracy sacrifice.
Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2