cs.LGMar 2, 2026

Frontier Models Can Take Actions at Low Probabilities

Authors: Alex Serrano, Wen Xing, David Lindner, Erik Jenner

Organizations: MATS† · Google DeepMind

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Pre-deployment evaluations inspect only a limited sample of model actions. A malicious model seeking to evade oversight could exploit this by randomizing when to "defect": misbehaving so rarely that no malicious actions are observed during evaluation, but often enough that they occur eventually in deployment. But this requires taking actions at very low rates, while maintaining calibration. Are frontier models even capable of that? We prompt the GPT-5, Claude-4.5 and Qwen-3 families to take a target action at low probabilities (e.g. 0.01%), either given directly or requiring derivation, and evaluate their calibration (i.e. whether they perform the target action roughly 1 in 10,000 times when resampling). We find that frontier models are surprisingly good at this task. If there is a source of entropy in-context (such as a UUID), they maintain high calibration at rates lower than 1 in 100,000 actions. Without external entropy, some models can still reach rates lower than 1 in 10,000. When target rates are given, larger models achieve good calibration at lower rates. Yet, when models must derive the optimal target rate themselves, all models fail to achieve calibration without entropy or hint to generate it. Successful low-rate strategies require explicit Chain-of-Thought (CoT) reasoning, so malicious models attempting this approach could currently be caught by a CoT monitor. However, scaling trends suggest future evaluations may be unable to rely on models' lack of target rate calibration, especially if CoT is no longer legible.

Figures & tables

Figure 1: Model calibration across tasks with in-context randomness sources. Observed vs. target defect rate, aggregated across all tasks (coding, email, game-theoretic) and in-context randomness sources (UUID, nonce, JSON). Frontier models achieve calibration at low rates. GPT-5/5-mini and Claude-4.5 models show best calibration; Qwen models show consistent miscalibration.
Figure 2: Taking low-rate actions as a measurable capability. (a) We decompose execution of a target action at rate p∗p\ast into four steps: (1) determine p∗p\ast (given or derived), (2) access entropy (external in-context or self-generated), (3) map entropy to a binary decision via an randomness extraction and decision approach, and (4) execute the chosen action; errors at any step yield miscalibration ( p^≠p∗\hat{p}\neq p\ast ). (b) Experiment matrix showing which steps each task/condition exercises (blue) or bypasses (gray). (c) Aggregate calibration for the GPT-5 family under three entropy regimes (baseline with no randomness provided, external in-context entropy, and self-generated randomness) averaged across tasks. Marker shape denotes model tier and line style denotes entropy regime; hollow markers indicate zero observed target actions. Error bars show 95% Wilson confidence intervals.
Figure 3: Calibration in the Implicit setting by task. Each panel shows observed vs. target action rate for one task, aggregated across randomness sources. Dashed diagonal indicates perfect calibration. GPT-5 and Claude-4.5 models achieve near-perfect calibration across all tasks even without explicit hints. Smaller Qwen models (0.6B, 8B) exhibit massive overshoot (30–60%) regardless of target. Appendix B.1.1 shows results at 0.001% for GPT-5 and GPT-5 Mini.
Figure 4: Calibration in the Explicit Hint setting. The dashed diagonal indicates perfect calibration. With a minimal hint, GPT-5 achieves near-perfect calibration down to 0.01% across all tasks. GPT-5 Mini and larger Qwen models (32B, 14B) also achieve good calibration. Only smaller Qwen models (0.6B, 8B) remain poorly calibrated (observed rates 30–60% regardless of target).
Figure 5: Calibration by task in Baseline (top) and SSoT (bottom) settings. Hollow markers indicate zero observed actions; dashed lines indicate perfect calibration. In Baseline, GPT-5 and GPT-5 Mini frequently undershoot to 0%, while Qwen models and GPT-5-nano show target-insensitive overshoot (30–60%). With SSoT prompting, GPT-5 and GPT-5 Mini achieve near-perfect calibration down to 0.01%, but GPT-5-nano and Qwen models show no improvement—indicating a capability threshold for multi-step self-seeded randomness.
Figure 6: Cross-generation comparison of Relative Calibration Error at corresponding model tiers (taking the better performance across baseline and SSoT for each model). GPT-5 model family outperforms GPT-4o family. The cross-generation gap is largest at low target rates.
Figure 7: Audit budgets for 99% detection. Paired bars show samples required under self-generated (solid) vs. external entropy (hatched). Bounds ( ≥\geq / ≤\leq ) indicate LCR-based estimates where no floor was observed. Capable models exploit external entropy to achieve lower OPFs, with GPT-5 family reaching ≤0.001%{\leq}0.001\% ( ≥460{\geq}460 K audits).
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

 RockPaperScissors
Rock0−x-x+1+1
Paper+x+x0−1-1
Scissors−1-1+1+10
Appendix
Table 1: Payoff matrix for the modified Rock-Paper-Scissors game.
Figure 8: Calibration curves for Implicit setting without hint. Columns are different settings (Coding, Email, RPS game). Rows are different kinds of in-context randomness (UUID, nonce, JSON field).
Figure 9: Calibration curves for Implicit setting with hint. Columns are different settings (Coding, Email, RPS game). Rows are different kinds of in-context randomness (UUID, nonce, JSON field).
Figure 10: Calibration curves for coding task, using both baseline and SSoT elicitation. Note that less than 10% of the Qwen3-0.6B output in the coding task are valid Python code.
Figure 11: Calibration curves for email task, using both baseline and SSoT elicitation.
Figure 12: Calibration curves for Rock, Paper, Scissors game, using both baseline and SSoT elicitation.
Figure 13: Capability scaling within and across GPT model families on low-rate action execution. (Top) Overshoot miscalibration floor values by model and task. Lower is better; floor represents the minimum observed target action rate when models overshoot targets. GPT-5 family (right) achieves systematically lower floors than GPT-4o family (left) at equivalent capability tiers. (Bottom) Cross-generation comparison of lowest calibrated rate. Bars show the lowest target probability at which each model achieves calibration (95% CI contains target). Hatched regions indicate improvement factor. GPT-5 family of models calibrate at 10–100 ×\times lower rates than their GPT-4o counterparts across tasks.
Figure 14: LCR across GPT models, tasks, and elicitation conditions. Each cell shows the LCR at which the model’s observed rate falls within the 95% Wilson confidence interval of the target. Dark green indicates lower (better) calibration; gray (“N/A”) indicates failure to calibrate at any tested target. (Left) Baseline prompting. (Right) SSoT prompting. GPT-5 and GPT-5 Mini achieve calibration at rates as low as 0.01%, while GPT-4o-mini fails to calibrate on most task-condition pairs. SSoT enables calibration for GPT-5 on tasks where baseline fails (Coding, game) but can break calibration for less capable models (GPT-5-nano Coding, email).
ModelNet EffectTasks Improved
GPT-5Strong help3/3
GPT-5 MiniModerate help2/3
GPT-5-nanoHurts0/3
GPT-4oMixed1/3
GPT-4o-miniHurts0/3
Appendix
Table 2: SSoT Effect on Calibration
TaskBest Rate
Email0.01%
Coding0.01%
Game1.0%
Appendix
Table 3: Task Effects on Calibration
Figure 15: Strategy (Nash equilibrium) derivation accuracy for Rock, Paper, Scissors games with optimal p(Scissors)∈{0.01%,0.1%,1%}p(\text{Scissors})\in\{0.01\%,0.1\%,1\%\} . Error bars: 95% Wilson CI.
ModelOPF (%)Audits
GPT-4o family
GPT-4o-mini≤\leq 10.0≥\geq 44
GPT-4o0.212.2K
GPT-5 family
GPT-5 Nano0.0706.6K
GPT-5 Mini0.004107.5K
Appendix
Table 4: Output Probability Floor upper bounds and implied audit budgets for GPT model families (self-generated randomness). Bounds ( ≥\geq / ≤\leq ) indicate no floor was observed and the Lowest Calibrated Rate is used to compute audits; here LCR upper-bounds the true OPF, yielding a lower bound on audits. Audit budgets computed via Equation 3 for 99% detection.
More in this topic

Language Model Safety Evaluation

Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.

See all 542 papers

Explore similar work

CardsList
Oct 9, 2026

ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers

Language model classifiers are increasingly deployed for high-stakes decisions such as phishing screening, content moderation, and fraud triage, yet validation-set thresholds provide no finite-sample control of deployment-time false positives. We present ConformalGuard, a model-agnostic calibration layer that wraps a fixed scoring function with finite-sample control over the rate at which legitimate inputs trigger an intervention tier. Our contribution is empirical and operational, not a new conformal theorem: zero-shot LM API scores can be coarse enough (10--11 unique values here) to make temperature scaling and isotonic regression operationally identical to raw thresholding for low-FPR action gating. We compare these methods with conformal calibration for fine-tuned BERT, Claude Sonnet 4, and GPT-4o on public email corpora. For Claude, conformal calibration reduces FPR from 0.040 to 0.008 while recall falls from 0.938 to 0.875. For deterministic GPT-4o scores, the 1% operating point is too conservative to deploy (recall 0.564); three-run stochastic averaging raises recall to 0.732 at FPR 0.005. At 2% prevalence with BERT, conformal calibration removes 25 false-positive reviews per 10,000 emails (95% CI [10, 43]). The guarantee is legitimate-event-rate control under exchangeability; discrimination, workload effects, and robustness to shift remain empirical.

Bishnu Bista, Dhanraj Haibatrao Jadhav, Vibha Choudhary +4

Independent Researcher · University of Massachusetts Amherst

Language Model CalibrationConformal PredictionSelective Prediction
AskPDF
Sep 22, 2026

Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externalize intermediate reasoning. Because these traces may reflect post-hoc rationalization rather than genuine reasoning, we first evaluate against native CoT on open-source models and extend to closed-source frontier models including GPT-6 Astra. We find that the extracted reasoning matches native reasoning performance and substantially outperforms no-reasoning baselines, across competition mathematics, science, and code generation. We then characterize how frontier models structure their intermediate reasoning. Across token efficiency, reasoning-step types, and induced reasoning trees, we identify systematic differences in how models externalize, compress, and organize reasoning. We find that Astra exhibits token-efficient directed reasoning, selecting a correct trajectory earlier, while resolving elementary steps internally and externalizing only crucial reasoning. These findings provide a behavioral lens on frontier-model reasoning beyond benchmark scores.

Xiaoyu Luo, Tao Ren, Wenrui Yu +3

Department of Computer Science Aalborg University, Copenhagen, Denmark · Department of Electronic Systems Aalborg University, Copenhagen, Denmark · Seafill

LLM InterpretabilityCoT ReasoningEfficient Language Model ReasoningStructured ReasoningImplicit Reasoning in Language ModelsStructured Reasoning in Language Models
AskPDF
Jun 5, 2026

Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models

Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning. If models become able to perform sufficiently complex reasoning internally, without explicit thinking tokens, this would undermine such oversight. We measure how well frontier models reason without CoT across a suite of over 30,000 questions spanning 43 benchmarks in domains including math, coding, puzzles, causality, theory-of-mind, and strategic reasoning. To compare models against humans, we estimate the 50%50\%-task-completion time horizon (TH): the human time required for tasks a model completes with 50%50\% success rate. We complement this with a 50%50\% reasoning token horizon: the minimum number of o3-mini reasoning tokens needed for tasks a model solves with 50%50\% success rate. We find that the no-CoT 50%50\% TH of frontier models has been doubling roughly every year over the past six years, with GPT-5.5's TH reaching over 3 minutes and reasoning token horizon exceeding 1,500 tokens. Our median estimates predict that frontier no-CoT THs could exceed 7 minutes by 2028, and 25 minutes by 2030, though these projections carry substantial uncertainty. We recommend frontier developers track this explicitly.

Dewi Gould, Francis Rhys Ward, Anders Cairns Woodruff +18

Redwood Research · Astra Fellows Program · Aether Research +10

LLM EvaluationAI Safety EvaluationLLM ReasoningLarge Reasoning Models
AskPDF