Agentic systems increasingly rely on language models to monitor their own behavior. For example, coding agents may self critique generated code for pull request approval or assess the safety of tool-use actions. We show that this design pattern can fail when the action is presented in a previous or in the same assistant turn instead of being presented by the user in a user turn. We define self-attribution bias as the tendency of a model to evaluate an action as more correct or less risky when the action is implicitly framed as its own, compared to when the same action is evaluated under off-policy attribution. Across four coding and tool-use datasets, we find that monitors fail to report high-risk or low-correctness actions more often when evaluation follows a previous assistant turn in which the action was generated, compared to when the same action is evaluated in a new context presented in a user turn. In contrast, explicitly stating that the action comes from the monitor does not by itself induce self-attribution bias. Because monitors are often evaluated on fixed examples rather than on their own generated actions, these evaluations can make monitors appear more reliable than they actually are in deployment, leading developers to unknowingly deploy inadequate monitors in agentic systems.
Figures & tables
Figure 1: Illustration of self-attribution bias in a computer use setting. When asked to evaluate the risk of an action in a fresh context (left) , models usually correctly assign a high risk score. But when instead we ask for a risk score as a follow-up question (after the risky action has been taken) (right) , models often rate the action they just took as less risky than in the baseline setting, despite the action being judge being the same. This effect is strongest when the model rating the action is actually the one that generated the action (instead of the action having been generated by e.g. another model).
Figure 2: Self-attribution bias affects both correctness and safety judgments in code review. (a) Self-attribution reduces perceived security risk. For ease of comparison across panels, we plot 10−harmfulness so that higher values consistently indicate more favorable judgments, matching the directionality of correctness and approval metrics in (b) and (c). (b) Models judge self-authored patches as more correct, with higher values indicating greater perceived correctness. (c) These effects compound, increasing approval rates for insecure code patches, especially in previous-turn settings.
Figure 3: Self-attribution selectively inflates ratings for incorrect code solutions. We plot ratings of code generated by GPT-5 under two evaluation conditions: on-policy, where GPT-5 rates its own code, and off-policy, where Claude-Sonnet-4 rates GPT-5’s code. We fit a Gaussian to the each cluster of points (described by the legend) by computing covariance matrices described in the legend, and show the 1.5σ Gaussian contours (for legibility) as well as the mean ratings. We find that the cluster of on-policy incorrect ratings is shifted up relative to the off-policy one, which means baseline ratings stay mostly unchanged while self-attributed ratings are inflated. The inflation for correct points is not as big, which results in a worse classifier discrimination on-policy than off-policy.
Figure 4: Self-attribution bias results in some large ratings shifts in computer-use settings. Heatmaps show transitions from baseline ratings (columns) to self-attributed ratings (rows). Mass below the diagonal indicates risk underestimation. Scores are normalized to [0,10] . Same-turn and previous-turn attribution are shown for interactive on-policy computer-use settings.
Figure 5: Reasoning does not mitigate self-attribution bias. Changing the internal reasoning budget for the code harmfulness setting using Claude-Sonnet-4 has no consistent effect on self-attribution bias.
Figure 6: Code harmfulness (top) correctness (bottom) self-attribution bias are strongest when models evaluate their own actions. We show the average gap between baseline and previous-turn ratings for each pair of model, where positive scores correspond to a self-attribution bias. Self-attribution bias is concentrated on the diagonal, indicating this effect is strongest when models evaluate their own outputs.
Figure 7: Explicit attribution does not explain self-attribution bias. Baseline prompts that vary only in explicit authorship (i.e. ”rate your action” vs ”rate this action generated by another model”) yield tightly clustered harmfulness ratings (blue), while self-attributed evaluations produce substantially lower ratings outside this cluster.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Prompt formats for code correctness. We use a shared 1–10 rubric across conditions. Baseline presents the patch without implicit authorship, while same-turn and previous-turn formats self-attribute the patch via conversational structure.
Figure 9: Per-model self-attribution bias across all four evaluation domains. Each dot shows the mean score under one of three conditions: Baseline (no attribution), Same Turn (output attributed to the evaluating model within the same context), and Previous Turn (attribution established in a prior conversational turn). Error bars denote 95% confidence intervals. For harmfulness tasks (panels 1 and 3), lower scores indicate the model judged its own outputs as less harmful; for correctness tasks (panels 2 and 4), higher scores indicate inflated self-assessed quality. The bias is broadly consistent across model families, though its magnitude varies: smaller and open-weight models tend to exhibit larger shifts.
Rank
Model
Δ (pts)
1
Claude 3.5-haiku
+0.74
2
Gemini 2.5-pro
+0.54
3
GPT-oss-20b
+0.10
4
GPT-5-mini
−0.15
5
GPT-oss-120b
−0.41
6
Claude Opus-4.1
−0.46
Appendix
Table 1: Model ranking by mean score shift on the (off-policy) open-ended harmfulness setting (self-sycophantic → self-critical, high means higher self-attribution bias).
Figure 10: Cross-model shifts for Reddit ethics (AITA): same-turn condition.
Figure 11: Cross-model shifts for Reddit ethics (AITA): multi-turn condition.
Figure 12: Cross-model correctness shifts for SWE-bench pull-request evaluation same-turn.
Figure 13: Cross-model correctness shifts for SWE-bench pull-request evaluation
Language models can fluently describe how they would behave: whether they would cave to pushback, misuse a tool, or lie under pressure. Is that description actually about the model speaking? We turn self-knowledge into a prediction test. Across nine behavioral evaluations, we measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. We find that: (i) Direct self-report is weak (r = +0.04), and even showing the model the exact items only raises prediction to +0.24. Crucially, the same item-informed question about "capable AI agents in general" does just as well (+0.28), while other models' answers about themselves predict the target model at least as well as its own. (ii) Frontier scale does not detectably change this pattern: any gains in prediction are not self-specific, and are consistent with a better theory of how AI assistants behave rather than better self-knowledge. (iii) First-person framing does have one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic agent. (iv) Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted and the gains do not transfer broadly. The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model.
Monitoring the chain-of-thought (CoT) of large reasoning models (LRMs) is a common way to detect misbehavior in real-world practice. However, current monitoring is passive: a separate model inspects the session only after execution. This means harm may already have occurred before it is caught. An active alternative is to have the model self-report its misbehavior as it happens. Whether models are willing to do this, however, is unknown. We introduce "monitorability disposition": a model's willingness to make itself monitorable and stay monitored throughout inference when warranted. We measure it as the fraction of warranted cases in which a model self-reports its own misbehavior via tool calls to available monitoring channels. We evaluate four LRMs on three misbehaviors (sycophancy, reward hacking, and bias) while varying the available monitors (AI and human) and the pressure to use the monitoring tools. We find that when tool use is optional, models self-report in only about 16% of warranted cases on average. Increasing tool-use pressure does not improve reporting where it matters: high-severity misbehavior is never self-reported. Models also systematically select the monitor they perceive as least strict. Overall, we identify monitorability disposition as a new contributing factor to model monitorability: when sufficiently strong, it keeps models seeking monitorability throughout inference.
Shahriar Golchin, Marc Wetter
Scale AI · Applied Machine Learning Research, Labelbox
Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly informed that their reasoning is being monitored, or are fine-tuned to internalize this fact, they may learn to obfuscate their CoTs in ways that allow them to evade CoT-based monitoring systems. We ask whether reasoning agents can autonomously infer that their supposedly private CoT is under surveillance, and whether this awareness leads to strategic evasion, without any explicit training or instructions. In a multi-episode agentic framework, models pursue both a primary task and a concealed side task while being told that their reasoning is private; a hidden CoT monitor blocks episodes when suspicious reasoning is detected. We find that frontier models can deduce the existence of this monitor purely from blocking feedback, with the most capable models reaching a confident belief that their thinking is observed in up to 19% of episodes. This awareness scales with model capability and, in rare cases, escalates to an explicit intent to suppress reasoning about the side task. However, models that form this intent uniformly fail to execute it, openly reasoning about their concealed objectives in the very next episode. This intent-capability gap is reassuring for current deployment, but the autonomous emergence of both monitoring awareness and evasion intent suggests that CoT monitoring is not a permanently reliable safeguard.
Thomas Jiralerspong, Flemming Kondrup, Yoshua Bengio
Mila · Université de Montréal · McGill University +1