cs.LGDec 15, 2025
SaveWait, Wait, Wait... Why Do Reasoning Models Loop?
Organizations: MIT · Microsoft Research · University of Wisconsin–Madison
Abstract
Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature plays. With open reasoning models, we find that looping is common at low temperature. Larger models tend to loop less, and distilled students loop significantly even when their teachers rarely do. This points to mismatches between the training distribution and the learned model, which we refer to as errors in learning, as a key cause. To understand how such errors cause loops, we introduce a synthetic graph reasoning task and demonstrate two mechanisms. First, risk aversion caused by hardness of learning: when the correct progress-making action is hard to learn but an easy cyclic action is available, the model puts relatively more probability on the cyclic action and gets stuck. Second, even when there is no hardness, Transformers show an inductive bias toward temporally correlated errors, so the same few actions keep being chosen and loops appear. Higher temperature reduces looping by promoting exploration, but it does not fix the errors in learning, so generations remain much longer than necessary at high temperature; in this sense, temperature is a stopgap rather than a holistic solution. We end with a discussion of training-time interventions aimed at directly reducing errors in learning.
Figures & tables
Figure 1: Looping and response length for OpenThinker models. Looping fraction and response length for OpenThinker3-1.5B/7B (students) and QwQ-32B (teacher), averaged over AIME 2024/2025. For each (problem, model, temperature) we sample 20 chains; a response is looping if some 30-gram appears times. (Left) Students loop much more than the teacher at low temperature. (Right) While increasing temperature largely removes looping, the student models still produce substantially longer responses than the teacher at high temperatures.
Figure 2: Illustration of star graph. Figure from Bachmann and Nagarajan (2024) .
Figure 3: Hard-to-learn actions induce low-temperature loops. We train small Transformers on random-walk traces on star graphs (hard) and (easier). In the training distribution, the progress-making action is taken with probability and the reset action with ; dashed curves use an exploration variant at the root ( correct edge, other children, reset). The looping count is the average number of transitions per trace; a perfect learner would have looping count at . On , looping and response length are large at low temperature and much smaller at higher temperatures; without exploration, accuracy is below chance at and near chance at the positive temperatures tested, while with exploration it is near at and substantially higher at higher temperatures. On the easier instances, looping remains close to zero and accuracy is near-perfect across temperatures. Overall, a hard progress decision at the root, paired with an easy cyclic reset action, drives low-temperature loops, and making the problem easier or increasing temperature reduces the severe looping on the harder task.
Figure 4: Temporally correlated errors induce low-temperature loops. We train on star-graph random-walk traces that choose a child at the root, take the unique outgoing edge at other internal nodes, and reset to the start at non-goal leaves. The looping count is the maximum number of visits to any root child in a trace, averaged over test instances. With no margin ( ), the training walk chooses root children uniformly, but temporally correlated errors cause the model to repeatedly select the same children under greedy decoding. Greedy accuracy is ; at positive temperatures, looping count and response length fall sharply and accuracy rises to approximately . With margin , the training walk favors children that have not yet been visited. This eliminates greedy looping: the looping count is , so no child is revisited, and accuracy is near-perfect throughout.
Figure 5: Tiny bias drives reset probability toward 1. In the risk-aversion setting on with exploration, we record at every root visit the model’s probability of taking the reset-to-start action, across 4000 greedy-decoded traces (thin lines; thick line is the average). When training includes a tiny fraction ( ) of deterministic looping traces (orange), the reset probability rises toward 1 as the loop persists, making escape increasingly unlikely. Without this bias (blue), the probability stays roughly constant.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Looping with greedy decoding. Bars show the looping fractions at temperature , averaged over AIME 2024 and 2025. All reasoning models exhibit looping, and within each family larger models loop less (e.g., Qwen 1.5B 7B 32B; OpenThinker3 1.5B 7B QwQ-32B). Distilled students can loop significantly more than their teacher (OpenThinker3 vs. QwQ-32B). Reasoning models can also loop heavily even when their instruction-tuned counterparts barely loop (e.g., Qwen2.5 and Phi-4 families). Finally, RL post-training has limited effect on looping in the Phi-4 family (Reasoning vs. Reasoning-Plus).
Figure 7: Openthinker metrics as a function of temperature.
Figure 8: Qwen metrics as a function of temperature.
Figure 9: Phi-4 metrics as a function of temperature
Figure 10: Llama metrics as a function of temperature.
Figure 11: Looping with greedy decoding for a hard-easy split of AIME problems. For each model, we use accuracies at temperature 0.8 to create a split of the 5 easiest problems and 5 hardest problems. Then we evaluate the looping percentage at temperature 0 separately on these two sets of problems. The bars on the left show the looping percentage on hard problems and bars on the right show the percentage on easy problems. We can see that for all model families except Phi-4, models looping significantly more in hard problems than in easy ones.
| Model | (20,20) | (30,20) | (30,30) | (30,60) | (40,20) |
|---|---|---|---|---|---|
| R1-Distill-Qwen-1.5B | 0.76 | 0.76 | 0.75 | 0.72 | 0.75 |
| R1-Distill-Qwen-7B | 0.51 | 0.49 | 0.48 | 0.41 | 0.49 |
| R1-Distill-Qwen-32B | 0.40 | 0.37 | 0.33 | 0.22 | 0.36 |
| OpenThinker3-1.5B | 0.33 | 0.30 | 0.28 | 0.27 | 0.29 |
| OpenThinker3-7B | 0.12 | 0.10 | 0.09 | 0.09 | 0.09 |
| QwQ-32B | 0.04 | 0.04 | 0.03 | 0.03 | 0.03 |
Table 1: Looping percentages at temperature 0 for different (n, k) combinations
| Model | (10,10) | (10,20) | (10,40) | (30,20) (orig.) |
|---|---|---|---|---|
| R1-Distill-Qwen-1.5B | 0.89 | 0.80 | 0.77 | 0.76 |
| R1-Distill-Qwen-7B | 0.74 | 0.58 | 0.52 | 0.49 |
| R1-Distill-Qwen-32B | 0.64 | 0.47 | 0.41 | 0.37 |
| OpenThinker3-1.5B | 0.84 | 0.59 | 0.38 | 0.30 |
| OpenThinker3-7B | 0.72 | 0.37 | 0.16 | 0.10 |
| QwQ-32B | 0.49 | 0.16 | 0.06 | 0.04 |
Table 2: Looping percentages at temperature 0 for different (n, k) combinations
Figure 12: Looping with greedy decoding on GPQA.
Figure 13: Qwen metrics on GPQA as a function of temperature.
Figure 14: Openthinker metrics on GPQA as a function of temperature.
Figure 15: Looping from combined hardness and temporally correlated errors. We work on in a variant of the correlated-errors setting from Section 4 . The teacher places probability on the correct root child and spreads the remaining over four of the nine wrong children. Despite this preference for the correct action, the learned model exhibits strong looping and long responses under greedy decoding. Increasing temperature reduces looping and response length and improves accuracy.
Figure 16: Confidence buildup during looping. We plot the maximum next-token probability over time for two looping traces from OpenThinker3-1.5B on AIME 2025 Problem I-7 (a) and Problem I-5 (b), generated with greedy decoding. Each blue dot shows the top-1 probability at a single decoding step; the apparent vertical bands are due to plotting 30k time steps at this resolution. The green curve is a running average (window size 200). The vertical dashed line marks the step where the final infinite loop begins. In both cases, once the loop starts, the top-1 probability rises toward , indicating that the model becomes increasingly confident in continuing the loop.
Figure 17: Explicit looping bias drives reset probability toward 1. We work in the risk-aversion setting on (Section 3 ), with and without exploration. For each of 4000 greedy-decoded traces, we record at every visit to the root the model’s probability of taking the reset-to-start action (“source prob”) and plot it against the root-visit index (thin lines). The thicker lines show the average over all traces that reach that visit index (traces that have already ended are not included in the average; the average is shown only at visit indices reached by at least 5% of traces). In the biased models (orange), the training distribution is modified so that with probability the example is a trace that deterministically loops between start and root, and with the remaining probability it is sampled as before. Under this bias, the reset probability rises toward for most traces as the number of visits grows, so once a loop has persisted for a while the model becomes almost certain to continue it. In the unbiased models (blue), the reset probability stays roughly constant in the exploration variant (a) and generally drifts downward in the non-exploration variant (b). Note that panel (a) also appears in the main text in Figure 5 .
Figure 18: Confidence bias amplifies looping and degrades accuracy. In the risk-aversion setting on (Section 3 ), we plot accuracy, looping count, and response length versus temperature, with and without exploration, comparing models trained on the original distribution (solid) and with an added looping bias (dashed). The bias increases low-temperature looping counts and response lengths and reduces accuracy, with especially large effects at .
Figure 19: Explicit looping bias reinforces a single child at the root. For the correlated-errors setting on at margin , we compare models trained with (blue) and (orange) deterministic looping traces. For each of greedy-decoded traces, we identify the most visited child at the root and record the probability assigned to that child at each root visit. Thin lines show individual traces; thicker lines show the average over traces that reach each visit index, shown only where at least of traces contribute. With the bias, confidence in the most visited child grows over successive visits; without it, the probability stays near .
Figure 20: Effect of looping bias on accuracy and looping. In the correlated-errors setting on (Section 4 ), we plot accuracy, looping count, and response length versus temperature at margin . Solid blue lines use the original training distribution; dashed orange lines use deterministic looping traces. The effect is most visible at temperature , where looping count and response length increase slightly while accuracy decreases slightly. Because confidence buildup requires repeated visits to the same child, the aggregate effect is limited when sampling breaks the underlying loop.
Figure 21: Model capacity affects looping on . We train 1-layer, 2-layer, and 12-layer models on the task (no exploration variant) from Section 3 , and plot accuracy, looping count, and response length against temperature. The 12-layer model achieves near-perfect accuracy across temperatures and has no greedy looping; the 2-layer model loops more at low temperature and achieves accuracy above chance but well below the 12-layer model; the 1-layer model loops even more and remains below chance, with invalid edges limiting accuracy at higher temperatures.
Figure 22: Smaller models overestimate reset on generated histories. For the capacity models on , we measure the mean reset probability (left) and the fraction above (right) on held-out graphs. Solid curves pool root positions in the valid prefixes of greedy traces, excluding a final end-of-sequence decision. Dashed curves use the fixed context consisting of the prompt followed by start and root, once per graph. The dotted line marks the teacher’s reset probability, .