Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature plays. With open reasoning models, we find that looping is common at low temperature. Larger models tend to loop less, and distilled students loop significantly even when their teachers rarely do. This points to mismatches between the training distribution and the learned model, which we refer to as errors in learning, as a key cause. To understand how such errors cause loops, we introduce a synthetic graph reasoning task and demonstrate two mechanisms. First, risk aversion caused by hardness of learning: when the correct progress-making action is hard to learn but an easy cyclic action is available, the model puts relatively more probability on the cyclic action and gets stuck. Second, even when there is no hardness, Transformers show an inductive bias toward temporally correlated errors, so the same few actions keep being chosen and loops appear. Higher temperature reduces looping by promoting exploration, but it does not fix the errors in learning, so generations remain much longer than necessary at high temperature; in this sense, temperature is a stopgap rather than a holistic solution. We end with a discussion of training-time interventions aimed at directly reducing errors in learning.
Figures & tables
Figure 1: Looping and response length for OpenThinker models. Looping fraction and response length for OpenThinker3-1.5B/7B (students) and QwQ-32B (teacher), averaged over AIME 2024/2025. For each (problem, model, temperature) we sample 20 chains; a response is looping if some 30-gram appears ≥20 times. (Left) Students loop much more than the teacher at low temperature. (Right) While increasing temperature largely removes looping, the student models still produce substantially longer responses than the teacher at high temperatures.
Figure 2: Illustration of star graph. Figure from Bachmann and Nagarajan (2024) .
Figure 3: Hard-to-learn actions induce low-temperature loops. We train small Transformers on random-walk traces on star graphs G(5,5) (hard) and G(5,3) (easier). In the training distribution, the progress-making action is taken with probability 0.7 and the reset action with 0.3 ; dashed curves use an exploration variant at the root ( 0.5 correct edge, 0.2 other children, 0.3 reset). The looping count is the average number of root→start transitions per trace; a perfect learner would have looping count 0 at T=0 . On G(5,5) , looping and response length are large at low temperature and much smaller at higher temperatures; without exploration, accuracy is below chance at T=0 and near chance at the positive temperatures tested, while with exploration it is near 0 at T=0 and substantially higher at higher temperatures. On the easier G(5,3) instances, looping remains close to zero and accuracy is near-perfect across temperatures. Overall, a hard progress decision at the root, paired with an easy cyclic reset action, drives low-temperature loops, and making the problem easier or increasing temperature reduces the severe looping on the harder task.
Figure 4: Temporally correlated errors induce low-temperature loops. We train on G(5,5) star-graph random-walk traces that choose a child at the root, take the unique outgoing edge at other internal nodes, and reset to the start at non-goal leaves. The looping count is the maximum number of visits to any root child in a trace, averaged over test instances. With no margin ( m=0 ), the training walk chooses root children uniformly, but temporally correlated errors cause the model to repeatedly select the same children under greedy decoding. Greedy accuracy is 44.43% ; at positive temperatures, looping count and response length fall sharply and accuracy rises to approximately 99% . With margin m=0.1 , the training walk favors children that have not yet been visited. This eliminates greedy looping: the looping count is 1 , so no child is revisited, and accuracy is near-perfect throughout.
Figure 5: Tiny bias drives reset probability toward 1. In the risk-aversion setting on G(5,5) with exploration, we record at every root visit the model’s probability of taking the reset-to-start action, across 4000 greedy-decoded traces (thin lines; thick line is the average). When training includes a tiny fraction ( 0.001 ) of deterministic looping traces (orange), the reset probability rises toward 1 as the loop persists, making escape increasingly unlikely. Without this bias (blue), the probability stays roughly constant.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Looping with greedy decoding. Bars show the looping fractions at temperature 0 , averaged over AIME 2024 and 2025. All reasoning models exhibit looping, and within each family larger models loop less (e.g., Qwen 1.5B > 7B > 32B; OpenThinker3 1.5B > 7B > QwQ-32B). Distilled students can loop significantly more than their teacher (OpenThinker3 vs. QwQ-32B). Reasoning models can also loop heavily even when their instruction-tuned counterparts barely loop (e.g., Qwen2.5 and Phi-4 families). Finally, RL post-training has limited effect on looping in the Phi-4 family (Reasoning vs. Reasoning-Plus).
Figure 7: Openthinker metrics as a function of temperature.
Figure 8: Qwen metrics as a function of temperature.
Figure 9: Phi-4 metrics as a function of temperature
Figure 10: Llama metrics as a function of temperature.
Figure 11: Looping with greedy decoding for a hard-easy split of AIME problems. For each model, we use accuracies at temperature 0.8 to create a split of the 5 easiest problems and 5 hardest problems. Then we evaluate the looping percentage at temperature 0 separately on these two sets of problems. The bars on the left show the looping percentage on hard problems and bars on the right show the percentage on easy problems. We can see that for all model families except Phi-4, models looping significantly more in hard problems than in easy ones.
Model
(20,20)
(30,20)
(30,30)
(30,60)
(40,20)
R1-Distill-Qwen-1.5B
0.76
0.76
0.75
0.72
0.75
R1-Distill-Qwen-7B
0.51
0.49
0.48
0.41
0.49
R1-Distill-Qwen-32B
0.40
0.37
0.33
0.22
0.36
OpenThinker3-1.5B
0.33
0.30
0.28
0.27
0.29
OpenThinker3-7B
0.12
0.10
0.09
0.09
0.09
QwQ-32B
0.04
0.04
0.03
0.03
0.03
Appendix
Table 1: Looping percentages at temperature 0 for different (n, k) combinations
Model
(10,10)
(10,20)
(10,40)
(30,20) (orig.)
R1-Distill-Qwen-1.5B
0.89
0.80
0.77
0.76
R1-Distill-Qwen-7B
0.74
0.58
0.52
0.49
R1-Distill-Qwen-32B
0.64
0.47
0.41
0.37
OpenThinker3-1.5B
0.84
0.59
0.38
0.30
OpenThinker3-7B
0.72
0.37
0.16
0.10
QwQ-32B
0.49
0.16
0.06
0.04
Appendix
Table 2: Looping percentages at temperature 0 for different (n, k) combinations
Figure 12: Looping with greedy decoding on GPQA.
Figure 13: Qwen metrics on GPQA as a function of temperature.
Figure 14: Openthinker metrics on GPQA as a function of temperature.
Figure 15: Looping from combined hardness and temporally correlated errors. We work on G(10,5) in a variant of the correlated-errors setting from Section 4 . The teacher places probability 0.5 on the correct root child and spreads the remaining 0.5 over four of the nine wrong children. Despite this preference for the correct action, the learned model exhibits strong looping and long responses under greedy decoding. Increasing temperature reduces looping and response length and improves accuracy.
Figure 16: Confidence buildup during looping. We plot the maximum next-token probability over time for two looping traces from OpenThinker3-1.5B on AIME 2025 Problem I-7 (a) and Problem I-5 (b), generated with greedy decoding. Each blue dot shows the top-1 probability at a single decoding step; the apparent vertical bands are due to plotting 30k time steps at this resolution. The green curve is a running average (window size 200). The vertical dashed line marks the step where the final infinite loop begins. In both cases, once the loop starts, the top-1 probability rises toward 1 , indicating that the model becomes increasingly confident in continuing the loop.
Figure 17: Explicit looping bias drives reset probability toward 1. We work in the risk-aversion setting on G(5,5) (Section 3 ), with and without exploration. For each of 4000 greedy-decoded traces, we record at every visit to the root the model’s probability of taking the reset-to-start action (“source prob”) and plot it against the root-visit index (thin lines). The thicker lines show the average over all traces that reach that visit index (traces that have already ended are not included in the average; the average is shown only at visit indices reached by at least 5% of traces). In the biased models (orange), the training distribution is modified so that with probability 0.001 the example is a trace that deterministically loops between start and root, and with the remaining probability it is sampled as before. Under this bias, the reset probability rises toward 1 for most traces as the number of visits grows, so once a loop has persisted for a while the model becomes almost certain to continue it. In the unbiased models (blue), the reset probability stays roughly constant in the exploration variant (a) and generally drifts downward in the non-exploration variant (b). Note that panel (a) also appears in the main text in Figure 5 .
Figure 18: Confidence bias amplifies looping and degrades accuracy. In the risk-aversion setting on G(5,5) (Section 3 ), we plot accuracy, looping count, and response length versus temperature, with and without exploration, comparing models trained on the original distribution (solid) and with an added looping bias (dashed). The bias increases low-temperature looping counts and response lengths and reduces accuracy, with especially large effects at T=0.2 .
Figure 19: Explicit looping bias reinforces a single child at the root. For the correlated-errors setting on G(5,5) at margin 0 , we compare models trained with 0% (blue) and 1% (orange) deterministic looping traces. For each of 4000 greedy-decoded traces, we identify the most visited child at the root and record the probability assigned to that child at each root visit. Thin lines show individual traces; thicker lines show the average over traces that reach each visit index, shown only where at least 5% of traces contribute. With the bias, confidence in the most visited child grows over successive visits; without it, the probability stays near 0.21 .
Figure 20: Effect of 1% looping bias on accuracy and looping. In the correlated-errors setting on G(5,5) (Section 4 ), we plot accuracy, looping count, and response length versus temperature at margin 0 . Solid blue lines use the original training distribution; dashed orange lines use 1% deterministic looping traces. The effect is most visible at temperature 0.2 , where looping count and response length increase slightly while accuracy decreases slightly. Because confidence buildup requires repeated visits to the same child, the aggregate effect is limited when sampling breaks the underlying loop.
Figure 21: Model capacity affects looping on G(5,3) . We train 1-layer, 2-layer, and 12-layer models on the G(5,3) task (no exploration variant) from Section 3 , and plot accuracy, looping count, and response length against temperature. The 12-layer model achieves near-perfect accuracy across temperatures and has no greedy looping; the 2-layer model loops more at low temperature and achieves accuracy above chance but well below the 12-layer model; the 1-layer model loops even more and remains below chance, with invalid edges limiting accuracy at higher temperatures.
Figure 22: Smaller models overestimate reset on generated histories. For the capacity models on G(5,3) , we measure the mean reset probability (left) and the fraction above 0.3 (right) on 4000 held-out graphs. Solid curves pool root positions in the valid prefixes of greedy traces, excluding a final end-of-sequence decision. Dashed curves use the fixed context consisting of the prompt followed by start and root, once per graph. The dotted line marks the teacher’s reset probability, 0.3 .
Looped Language Models (LoopLMs) perform "latent reasoning" by recursively refining internal latent representations with shared weights, offering a more effective alternative to explicit verbal reasoning. Despite their effectiveness, we find that LoopLMs remain prone to loop instability: unstable refinement across iterations can produce localized uncertain "hard" tokens associated with reasoning errors. To address this, we propose LoopCD, loop-wise contrastive decoding that enhances the reasoning performance of LoopLMs by intervening on these tokens at inference time. Specifically, we exploit the internal dynamics of LoopLMs and contrast the logits from earlier iterations with logits from the last refined iteration to form the final sampling distribution. We find that this strategy is highly efficient, introducing only negligible inference overhead and requiring no additional training, while effectively improving reasoning performance by naturally refining reasoning-critical hard tokens. Extensive experiments show that our method improves the performance of recent representative LoopLMs across various reasoning tasks.
Byeongho Yu, Junhyuk So, Eunhyeok Park
Department of Computer Science and Engineering Pohang University of Science and Technology (POSTECH) · Graduate School of Artificial Intelligence Pohang University of Science and Technology (POSTECH)
Reasoning-trained language models often spend more tokens on harder problems, but longer chains of thought do not show whether a model is merely computing for more steps or following a different internal trajectory. We study this distinction through hidden-state trajectories during chain-of-thought generation across competitive programming, mathematics, and Boolean satisfiability. Raw trajectory geometry is strongly shaped by generation length: longer generations mechanically alter path statistics, so difficulty-dependent comparisons are misleading without adjustment. After residualizing trajectory statistics on length, difficulty remains systematically coupled to corrected trajectory geometry across all domains studied. The clearest reasoning-specific separation appears in the code domain, where harder problems show more direct corrected trajectories and less heterogeneous local curvature in reasoning-trained models than in matched instruction-tuned baselines. Corrected difficulty-geometry coupling is weaker, but still present, in mathematics and Boolean satisfiability. Prompt-stage linear probes do not mirror the code-domain separation, and behavioral annotations show that stronger corrected coupling co-occurs with strategy shifts and uncertainty monitoring. Together, these findings establish length correction as a prerequisite for generation-time trajectory analysis and show that reasoning training can be associated with distinct corrected trajectory geometry, with the strength of the effect depending on the domain.
Anders Gjølbye, Lars Kai Hansen, Sanmi Koyejo
Technical University of Denmark · Stanford University
Looping, reusing a block of layers across depth, and depth growing, training shallow-to-deep models by duplicating middle layers, have both been linked to stronger reasoning, but their relationship remains unclear. We provide a mechanistic unification: looped and depth-grown models exhibit convergent depth-wise signatures, including increased reliance on late layers and recurring patterns aligned with the looped or grown block. These shared signatures support the view that their gains stem from a common form of iterative computation. Building on this connection, we show that the two techniques are adaptable and composable: applying inference-time looping to the middle blocks of a depth-grown model improves accuracy on some reasoning primitives by up to 2×, despite the model never being trained to loop. Both approaches also adapt better than the baseline when given more in-context examples or additional supervised fine-tuning data. Additionally, depth-grown models achieve the largest reasoning gains when using higher-quality, math-heavy cooldown mixtures, which can be further boosted by adapting a middle block to loop. Overall, our results position depth growth and looping as complementary, practical methods for inducing and scaling iterative computation to improve reasoning.
Ferdinand Kapl, Emmanouil Angelis, Kaitlin Maile +2
Technical University of Munich · Helmholtz AI, Munich · Google, Paradigms of Intelligence Team