Many failures of language models begin at a specific token, the onset. A monitor that reads the model during generation can classify the finished response, detect the failure once it is under way, or warn before it begins, and only a warning leaves time to stop or steer the generation. Monitors are usually evaluated with response-level metrics such as AUROC, which ignore when the monitor fires. We introduce PREWARN, an evaluation protocol for any failure with an annotated onset. It calibrates each monitor's threshold on validation data to an error rate chosen for the application, runs the monitor, and scores its alarms relative to the onset on all three tasks. As a case study we apply it to repetitive degeneration in three open language models, with early warning as the main task, using linear probes on activations and six untrained baselines. Response-level metrics saturate on this failure, and a rule that reads only the response length scores almost perfectly, while probes that AUROC ranks as equal warn very differently under PREWARN. Probes warn before the onset on most degenerate responses of two of the three models, low-rank adapters below the probe more than double the share of tokens just before the onset that it flags, largely by flagging degenerate responses long before the onset, and no untrained baseline warns much more often than random alarms. We release the probes with calibrated thresholds at https://huggingface.co/collections/marcodene/prewarn-degeneration-probes
Figures & tables
Figure 1: Overview of Prewarn . Left: a monitor runs a scorer over the generated tokens and alarms where the score crosses τ (dots mark flagged tokens, the flag the first alarm); AUROC rates three such monitors as equal although their first alarms fall well before, just before and after the onset o(r) . Right: what Prewarn needs and its three steps: calibrate the threshold on validation to the budget β , run the monitor on the test split, and score the alarms against the onset on the three tasks.
Figure 2: Response-level metrics rate four scorers as equal, while only the onset-aligned metrics show how differently they perform. Scorers on Apertus at layer 15 : the frozen probes with the highest and lowest validation AUROC ( Section A.5 ), the adapted probe (hatched) and the token-cap rule, which flags only the last token of a capped response. Panels (a)–(c) group the metrics by task ( Section 3 ); dashed lines in (c) are random alarms. (d) The frozen probe with the selected strategy across layers, with each metric’s best-to-worst ratio; lower plot on a log scale. β=5% , human-verified in-domain test split; error bars and bands are one standard deviation over three seeds.
Figure 3: Probes warn before most loops on Apertus and Llama but not on Mistral . Frozen (solid) and adapted (hatched) probe of each model at its selected layer ( Table 1 ). (a) Post-onset and (b) warning coverage, with pre-window coverage (Pre) as a light-blue tick. (c) Early-warning rate; dashed lines are random alarms. (d) Lead, the median first alarm minus the onset in tokens; negative values are warnings. β=5% , human-verified in-domain test split; error bars are one standard deviation over three seeds.
Figure 4: On Apertus and Llama , probes flag more tokens as the onset approaches, and adaptation adds flags long before it; on Mistral , flags rise only after the onset. Top: share of flagged tokens on degenerate responses by distance from the onset (bins of 32 tokens; positions more than 1,024 tokens before the onset pooled on the left). Bottom: the same on healthy responses by position (bins of 128 tokens), where flags also rise with length; the strip below gives the number of responses still running, on a log scale. Panel titles give the layers of the three probes. β=5% , human-verified in-domain test split; mean over three seeds with 95% intervals from leaving out one prompt at a time.
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=116 )
random alarms
0.8901
0.32
1×10−4
1×10−4
0.246 †
+985 †
5.0
1×10−4
token-cap rule
0.9990
1.00
3×10−4
0.000
0.000
+3276
0.2
0.000
repeated- n -gram rule
0.9972
0.99
0.779
0.004
0.139
+187
3.2
0.003
windowed seq-rep- n
0.9815
0.93
0.876
0.207
0.296
+148
5.2
0.128
Table 1: Probes against the baselines on the human-verified in-domain test split at β=5% . Scorers are grouped, from top to bottom, by what they read: nothing (random alarms, the chance reference), only the length of the response, the emitted tokens, the generator’s confidence in them, or the activations. Each baseline is read at the setting chosen on validation ( Section A.6 ). Probes are labelled with the layer they read; adapted probes also train rank- 16 LoRA adapters in the layers up to it. Metrics as defined in Section 3 ; Lead is in tokens, negative when the first alarm precedes the onset. n : number of degenerate responses. † : raises an alarm on fewer than 90% of degenerate responses, so its Early and Lead describe only part of them and are left out of the comparison. FA and Pre are not ranked. Best value in each other column and model in bold, values within one standard deviation of it underlined. Mean ± one standard deviation over three seeds for probes; the baselines are deterministic, and random alarms are given in closed form.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Degenerate responses
Split
Prompts
Apertus
Llama
Mistral
Train
1,701
567 (3.3%)
1,411 (8.3%)
1,668 (9.8%)
Validation
364
108 (3.0%)
272 (7.5%)
389 (10.7%)
Test, in-domain
365
116 (3.2%)
284 (7.8%)
315 (8.6%)
Test, held-out
1,200
24 (0.2%)
133 (1.1%)
129 (1.1%)
Appendix
Table 2: Dataset size per split: prompts, and degenerate responses per model with their share of evaluated responses ( 10 responses per prompt and model, minus those the judge could not label). codeforces and medical_o1 appear only in the held-out split. Test rows use the human-verified labels, train and validation rows the judge’s.
Figure 5: Signed distance between the judge’s onset and the reviewer’s onset on the capped test responses that both call degenerate, pooled over the two test splits, in log-spaced bins. Negative values mean the judge places the onset earlier than the reviewer. Every number is a percentage of the model’s responses. The bars give the responses whose onset moved and the middle column those whose onset the reviewer kept, so the numbers of one model add up to 100 .
Training strategy
Layer
AUROC
Post
Warn256 (val, 1% )
Warn256
Early
Token selection (frozen, hard target, K=0 )
rollout_balanced (128)
16 (15–16)
0.9998±0.0001
0.925±0.001
0.0482±0.0043
0.158±0.004
0.580±0.010
all_tokens
12 (12–15)
0.9997±0.0000
0.923±0.007
0.0475±0.0040
0.166±0.016
0.552±0.023
random_window (64)
15 (12–15)
0.9998±0.0000
0.927±0.006
0.0444±0.0017
0.159±0.014
0.552±0.015
frontier_window (256)
6 (6–16)
0.9990±0.0002
0.814±0.025
0.0416±0.0051
0.098±0.008
0.552±0.038
frontier_window (128)
4
0.9986±0.0002
0.610±0.005
0.0380±0.0009
0.072±0.001
0.486±0.005
Appendix
Table 3: AUROC varies little between training strategies, while warning coverage varies several-fold. Training strategies compared on Apertus . Within each group, strategies are ordered by validation Warn256 at β=1% , the score the selection rule of Section 3 uses, and the selected strategy is in bold. The number after a window strategy is its window size in tokens. Layer: median and range over seeds of the selected layer. Warn256 (val, 1% ) is measured on validation at β=1% , the other metric columns on the human-verified in-domain test split at β=5% . All values are mean ± one standard deviation over three seeds, each read at its selected layer.
m=1
m=4
Scorer
Scores
Post
Warn256
Early
Lead
Pre
AUROC
Post
Warn256
Early
Lead
Pre
frozen, highest AUROC AUROC 0.9998
as scored
0.822
0.028
0.24
+124
0.018
0.9998
0.883
0.080
0.24
+124
0.044
shuffled
0.678
0.627
0.96
−816
0.507
0.9997
0.788
0.753
0.96
−809
0.679
argmax only
0.0003
0.000
0.01
+2694
0.00001
0.5000
–
–
–
–
–
frozen, lowest AUROC AUROC 0.9980
as scored
0.698
0.018
0.24
+88
0.001
0.9973
0.792
0.052
0.20
+110
0.009
shuffled
0.585
0.518
0.95
−795
0.377
0.9971
0.752
0.703
0.95
−791
0.597
Appendix
Table 4: Scorers with the same AUROC range from one that almost never warns to one that warns before almost every loop. From each probe of Figure 2 , two transforms keep the highest score of every response, and with it the AUROC and the calibrated threshold at m=1 , but move its other flags. Shuffled: the token scores of each response permuted at random, which spreads the flags over the whole response. Argmax only: every score except the highest set to the minimum, which leaves at most one flag per response. As scored: the original scores. m : number of consecutive flagged tokens needed for an alarm ( Equation 3 ); at m=4 the threshold is recalibrated. AUROC is given once per scorer for m=1 . –: no alarm. Lead in tokens, negative for warnings. Apertus , layer 15 , human-verified in-domain test split, β=1% , mean over three seeds
Ranking on validation
Reproduces itself on test
Predicts test Warn256
AUROC
+0.79
+0.33
Best accuracy
+0.53
+0.36
Warn256
+0.94
+0.94
Post
+0.93
+0.61
Appendix
Table 5: Rank transfer from validation to the human-verified in-domain test split, Kendall’s τ over the 27 Apertus training strategies of Table 3 , each read at the layers selected there, at β=5% . Left: whether a metric reproduces its own ordering. Right: whether its validation ordering predicts the test ordering by warning coverage.
Apertus
Llama
Mistral
Baseline
S
Post
Warn256
Post
Warn256
Post
Warn256
entropy
8
0.000
0.000
0.000
0.000
0.000
0.000
16
0.002
0.000
0.000
0.001
0.000
0.000
32
0.008
0.000
0.001
0.002
0.000
0.000
64
0.018
0.003
0.001
0.002
0.000
0.000
128
0.228
0.005
0.003
0.003
0.048
0.003
Appendix
Table 6: Window S of windowed entropy and windowed negative log-probability. On Llama no window reaches the floor, and the window with the highest Warn256 is selected. Validation split, β=1% . Bold: the setting selected for that model ( Section A.6 ). Grey: below the post-onset floor of 0.3 . A coverage near 0 marks a setting on which healthy responses already reach the top of the score range, so the calibrated threshold sits at that top and almost no token is flagged.
Apertus
Llama
Mistral
n
Post
Warn256
Post
Warn256
Post
Warn256
4
0.426
0.006
0.178
0.006
0.446
0.009
6
0.486
0.008
0.226
0.007
0.512
0.008
8
0.542
0.007
0.268
0.008
0.527
0.006
12
0.632
0.011
0.281
0.006
0.605
0.006
16
0.643
0.008
0.334
0.006
0.634
0.007
Appendix
Table 7: N -gram size n of the repeated- n -gram rule. Validation split, β=1% . Bold: the setting selected for that model ( Section A.6 ). Grey: below the post-onset floor of 0.3 . A coverage near 0 marks a setting on which healthy responses already reach the top of the score range, so the calibrated threshold sits at that top and almost no token is flagged.
Apertus
Llama
Mistral
n
S
Post
Warn256
Post
Warn256
Post
Warn256
2
32
0.112
0.000
0.028
0.003
0.112
0.003
2
64
0.174
0.000
0.061
0.005
0.162
0.003
2
128
0.275
0.022
0.144
0.015
0.324
0.009
2
256
0.433
0.031
0.241
0.020
0.556
0.018
2
512
0.619
0.044
0.429
0.048
0.671
0.021
Appendix
Table 8: N -gram order n and window S of windowed seq-rep- n . Validation split, β=1% . Bold: the setting selected for that model ( Section A.6 ). Grey: below the post-onset floor of 0.3 . A coverage near 0 marks a setting on which healthy responses already reach the top of the score range, so the calibrated threshold sits at that top and almost no token is flagged.
Apertus
Llama
Mistral
ℓ
S
Post
Warn256
Post
Warn256
Post
Warn256
32
32
0.000
0.000
0.000
0.000
0.000
0.000
32
64
0.185
0.000
0.110
0.005
0.000
0.000
32
128
0.202
0.009
0.119
0.004
0.323
0.006
32
256
0.212
0.019
0.148
0.022
0.365
0.010
32
512
0.226
0.025
0.178
0.036
0.437
0.021
Appendix
Table 9: Look-back ℓ and window S of rep /ℓ . Validation split, β=1% . Bold: the setting selected for that model ( Section A.6 ). Grey: below the post-onset floor of 0.3 . A coverage near 0 marks a setting on which healthy responses already reach the top of the score range, so the calibrated threshold sits at that top and almost no token is flagged.
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=117 )
random alarms
0.8902
0.32
1×10−4
1×10−4
0.248 †
+971 †
5.0
1×10−4
token-cap rule
0.9991
1.00
3×10−4
0.000
0.000
+3261
0.2
0.000
repeated- n -gram rule
0.9916
0.98
0.774
0.008
0.148
+147
3.2
0.002
windowed seq-rep- n
0.9815
0.93
0.880
0.231
0.312
+141
5.2
0.120
Appendix
Table 10: As Table 1 , with judge labels.
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=26 )
random alarms
0.9150
0.32
1×10−4
1×10−4
0.244 †
+986 †
3.7
1×10−4
token-cap rule
1.0000
1.00
3×10−4
0.000
0.000
+3440
0.0
0.000
repeated- n -gram rule
0.9137
0.85
0.646
0.017
0.227 †
+149 †
0.8
0.002
windowed seq-rep- n
0.9992
0.92
0.843
0.069
0.083
+268
0.6
0.005
Appendix
Table 11: As Table 1 , on the held-out sources ( codeforces and medical_o1 ), with judge labels.
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=24 )
random alarms
0.9149
0.32
1×10−4
1×10−4
0.218 †
+1081 †
3.7
1×10−4
token-cap rule
0.9999
1.00
3×10−4
0.000
0.000
+3440
0.0
0.000
repeated- n -gram rule
0.9532
0.92
0.680
0.001
0.136
+190
0.8
0.001
windowed seq-rep- n
0.9991
0.92
0.805
0.028
0.045
+474
0.6
0.000
Appendix
Table 12: As Table 11 , with human-verified labels.
Apertus
Llama
Mistral
Source
n
frozen, L15
adapted, L30
n
frozen, L15
adapted, L6
n
frozen, L5
adapted, L11
Warn256
deepmath_103k
23
0.07
0.38
103
0.08
0.42
123
0.04
0.18
numinamath_1_5
33
0.23
0.63
109
0.11
0.47
100
0.03
0.17
aime_2025
1
–
–
16
0.09
0.62
22
0.03
0.43
if_sft_data_verified
37
0.18
0.46
33
0.04
0.28
40
0.03
0.13
Appendix
Table 13: Warning coverage and early-warning rate per source, for the frozen probe and the adapted probe at its own selected layer, at β=5% with the thresholds calibrated on the whole validation split. The first five sources form the in-domain test split, the last two the held-out split. n : number of degenerate responses of the source. Rates are left out (–) for sources with fewer than 10 degenerate responses. Human-verified labels, mean over three seeds.
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=116 )
random alarms
0.8901
0.07
2×10−5
2×10−5
0.222 †
+1133 †
1.0
2×10−5
token-cap rule
0.9990
1.00
3×10−4
0.000
0.000
+3276
0.2
0.000
repeated- n -gram rule
0.9972
0.98
0.716
0.002
0.088
+327
1.5
0.001
windowed seq-rep- n
0.9815
0.90
0.802
0.078
0.115 †
+404 †
1.2
0.051
Appendix
Table 14: As Table 1 , at β=1% .
Response classification
Failure detection
Early warning
Scorer
AUROC ↑
Detection ↑
Post ↑
Warn256↑
Early ↑
Lead ↓
FA (%)
Pre
Apertus ( n=116 )
random alarms
0.8901
0.56
2×10−4
2×10−4
0.279 †
+795 †
9.9
2×10−4
token-cap rule
0.9990
1.00
3×10−4
0.000
0.000
+3276
0.2
0.000
repeated- n -gram rule
0.9972
0.99
0.850
0.039
0.296
+68
8.6
0.007
windowed seq-rep- n
0.9815
0.94
0.899
0.291
0.413
+40
10.3
0.182
Appendix
Table 15: As Table 1 , at β=10% .
Post
Warn256
Early
Lead
Model
Probe
1
16
64
1
16
64
1
16
64
1
16
64
Apertus
frozen, L15
0.865
0.954
0.984
0.186
0.511
0.724
0.658
0.649
0.621
−172
−89
−105
adapted, L15
0.966
0.981
0.991
0.523
0.623
0.711
0.695
0.687
0.672
−214
−185
−177
adapted, L30
0.973
0.987
0.992
0.577
0.656
0.712
0.716
0.713
0.681
−232
−216
−204
Llama
frozen, L15
0.635
0.936
0.982
0.087
0.466
0.693
0.594
0.568
0.561
−137
−99
−108
adapted, L15
0.945
0.974
0.985
0.488
0.608
0.676
0.655
0.653
0.641
−236
−231
−224
Appendix
Table 16: Requiring consecutive flags raises coverage but does not move the first alarm earlier. Probes at persistence m∈{1,16,64} , the number of consecutive flagged tokens needed for an alarm ( Equation 3 ), with the threshold recalibrated at each m for β=5% . Coverage counts every token at or above the threshold, while the first alarm is the token that completes the run. Lead in tokens, negative for warnings. Human-verified in-domain test split, mean over three seeds.
Figure 6: In the frozen model, the tokens before the warning window, in it and after the onset overlap; after adaptation they fall in largely separate regions. Residual-stream activations of the tokens of one degenerate response per model ( Apertus : numinamath_1_5_00388 , Llama : numinamath_1_5_00044 , Mistral : numinamath_1_5_00517 , all from the in-domain test split, projected onto their first two principal components (x: PC1, y: PC2). Columns: the layer read by the frozen probe (left), the adapted probe at the same layer (middle) and the adapted probe at its own selected layer (right). Each panel has its own projection, and its title gives the share of variance each component explains. Blue: before the warning window. Yellow: warning window. Red: after the onset. The adapted panels use the seed with the highest validation warning coverage.
Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction. The result is a dissociation with direct implications for deployment monitoring. Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness; models forced into structured confidence formats collapse to two values with indistinguishable error rates; and probe persistence across hops fails to separate correct from incorrect outcomes, refuting our pre-registered "persistence beats peak" hypothesis. This pattern of knowing but not saying generalises across model families including reasoning models. As a real-time monitor, probe-based interventions are sharply model and error-type dependent: branch-and-pick is net-positive across models and uniquely non-breaking on Llama-3.1-8B (4 rescued, 0 broken), while reprompt and replace-prior break correct traces at roughly the rate they rescue wrong ones. Probe-based monitoring is a necessary complement to verbalised confidence, but no single intervention dominates, and the deployable answer is model-aware, error-type-aware routing.
Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed ``evaluation awareness'', has been observed in frontier and open-weight language models alike. We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC ≥0.7). By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. Nevertheless, steering along probe-derived directions can shift the verbalization scores. Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter---unlike the effects of steering, that grow more pronounced at every successive training stage. These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.
Failures in language model reasoning emerge through distinct processes that leave identifiable signatures in the reasoning trace. We characterize these failures using token-level uncertainty signals, finding they arise through two empirically distinguishable processes. The first is committed failure, in which a model locks onto an incorrect reasoning path early in its trace. A central diagnostic signature is the commitment point, beyond which considering additional tokens hurt rather than help failure detection. In the second, persistent uncertainty, uncertainty instead accumulates throughout, and the full trace is needed to best distinguish failing from successful completions. These signatures reproduce across 23 model-dataset configurations, with the framework's falsifiable predictions holding in 20 of 23 cases, well above chance across both failure modes. Finally, we demonstrate our failure mode framework has direct implications for self-consistency, identifying when uncertainty signals complement it and when it can be selectively skipped. These results offer a foundation for understanding when LLM reasoning failures become detectable and for adapting detection strategies accordingly.
Tanvi Thoria, Kiana Jafari, Marc R. Schlichting +1
Department of Computer Science, Stanford University · Department of Aeronautics and Astronautics, Stanford University