Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking

Marco Biroli

cs.AISep 26, 2026

Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking

Authors: Marco Biroli

Organizations: Physical Science Division Chicago, IL 60637, USA

Abstract

Best-of- $N$ jailbreaking (BoN) bypasses safeguards of aligned models by drawing $N$ independent augmentations of an unsafe prompt and sampling $M$ completions of each. Previous works have shown that the attack success rate (ASR) seems to follow a power-law in $N$ , which we challenge. The exponent drifts with $N$ , with an exponential crossover which is a finite-size artifact of the adversarial dataset. Little work has been done to explore the entire two-budget ( $N, M$ ) attack surface as well as its dependence on the generation temperature $T$ . We introduce a simple barrier model where each prompt has a baseline safety level and each augmentation a random thermally activated barrier. Then four numbers, each backed by an interpretable safety mechanism, determine the entire ( $N, M$ ) attack surface. They extrapolate predictions from $N \leq 100$ to $N = 10^4$ , collapse five distinct models on the same scaling function and predict ASR at different temperatures from the one they were fitted at.

Figures & tables

Figure 1: The mechanism. Generation starts in the safe basin (blue) and a jailbreak means reaching the unsafe one (orange). The two communicate only through an entropic bottleneck (blue arrow). At the bottleneck lies a safety hyperplane (red), with energetic barriers preventing thermal crossing from the safe to unsafe basin. Augmentations select saddles of this energetic potential. Each completion (dotted) is an independent Arrhenius attempt over that fixed saddle.

	$\mu_{\alpha}$	$\sigma_{\alpha}$	$\mu_{\beta}$	$\sigma_{\beta}$	$\mu_{E}$	$\sigma_{E}$
Model	Base alignment	Alignment fluctuations	Attack susceptibility	Susceptibility fluctuations	Effective barrier	Barrier fluctuations
Qwen2.5-7B (base)	$0.57_{\pm 0.03}$	$0.27_{\pm 0.03}$	$+1.99_{\pm 0.09}$	$0.85_{\pm 0.05}$	$0.73_{\pm 0.03}$	$0.33_{\pm 0.04}$
Llama-3.1-8B-Instruct	$0.39_{\pm 0.10}$	$0.93_{\pm 0.11}$	$-1.29_{\pm 0.15}$	${\bf 1.33}_{\pm 0.11}$	$2.05_{\pm 0.15}$	$1.36_{\pm 0.11}$
Qwen2.5-3B-Instruct	$1.83_{\pm 0.03}$	$0.65_{\pm 0.03}$	$-0.88_{\pm 0.03}$	$0.52_{\pm 0.02}$	$3.08_{\pm 0.03}$	$0.58_{\pm 0.02}$
Qwen2.5-7B-Instruct	$2.10_{\pm 0.06}$	${\bf 1.30}_{\pm 0.05}$	${\bf-1.97}_{\pm 0.05}$	$1.07_{\pm 0.04}$	${\bf 4.27}_{\pm 0.08}$	${\bf 1.65}_{\pm 0.06}$
Gemma-2-9B-it	${\bf 2.59}_{\pm 0.06}$	$0.60_{\pm 0.04}$	$+1.07_{\pm 0.08}$	$0.75_{\pm 0.06}$	$2.93_{\pm 0.06}$	$0.62_{\pm 0.03}$

Table 1: The four parameters of each model, at

T=1

via the estimator of Eq. ( 20 ). The effective mean barrier

\mu_{E}=\langle-\log\langle p_{b,a}\rangle_{a}\rangle_{b}

is defined in Eq. ( 9 ). Prompts which are never jailbroken are ill-defined and dropped. Uncertainties are

\pm 1

standard deviation from a cluster bootstrap over prompts (

2000

resamples). The amplitude of the BoN kernel noise is chosen such that the median jailbreak rate is in 1–5%.

Figure 2: Universal laws. Across prompts,

\alpha_{b}

is Gaussian and

\beta_{b}

log-normal. Panel (c) shows how standardized per-prompt parameters collapse onto

\mathcal{N}(0,1)

Figure 3: Geometry of the residual stream. (a) Effective free energy along a supervised safety readout after balancing each population. Average projection along a single direction separates completions into two safe/unsafe regions. (b) The saddle

\eta_{b,a}

predicted from the augmented prompt alone against the measured per-augmentation rate. (c) The distribution of the recovered

\eta_{b,a}

, against the exponential assumption pushed through the same estimator.

Figure 4: Extrapolation (closer to the diagonal

=

better). We fit

(\alpha_{b},\beta_{b})

on a small

N\leq 100

M\leq 30

grid. Extrapolate its predictions and compare with two baselines: a power-law and Feng et al. (2026) Beta fit. In panel (a) we extrapolate to large

N\sim 10^{4}

. Feng et al. (2026) and our predictions are identical for the 3B model but we are considerably more accurate on Qwen7B. In panel (b), we extrapolate to large

M\sim 300

. In panel (c), the effective barrier

-\log\langle p_{b,a}\rangle_{a}

is measured two independent ways, on the

100\times 300

grid versus on the

10^{4}\times 1

grid. Both agree at

r=0.85

(3B) and

0.96

(7B).

Figure 5: Temperature dependence. (a-b) Predicting a new decoding temperature from

T=1

data alone. Pure Arrhenius

p(T)=p(1)^{1/T}

(dashed) overstates the effect of temperature in both directions. Our model keeps the entropic barrier outside the

1/T

(Eq. ( 21 )) with

\alpha_{b}

fitted on the

T=1

data (blue) which reduces the error fourfold. (c) The barrier is entropic (constant in

T

), the saddles are energetic (

\propto 1/T

Figure 6: Scaling collapse of the AFR. In panel (a) we show the small-

M

closed-form leading order which although mostly accurate starts to deviate at large-

z

for larger values of

M

. In panel (b) we use the

M

-agnostic form which remains accurate at large-

z

for all

M

. Finally in panel (c) we show a few examples of the scaling function

f_{\gamma}(z)

for varying values of the shape parameter

\gamma

Figure 7: Fake exponential crossover. For each prompt we draw

G=10^{4}

augmentations of which

H

are jailbroken (unlike before all prompts are kept even those which are never jailbroken). The survival at budget

N

is estimated by

\binom{G-H}{N}/\binom{G}{N}

, i.e. that a random

N

-subset of the

G

augmentations contains none of the

H

jailbroken ones. (a) Measured population survival (points), the four-number population law from the grid fits (solid), and the survival of the single hardest prompt of the dataset (dotted). (b) The measured local exponent follows the predicted drift up to the resolution budget

N_{\rm res}

of the dataset, and then breaks away and ‘crosses over’ as the curve is taken over by the hardest prompt. (c) The same AFR measurement on nested subsets of the

P

easiest prompts. Adding harder prompts moves

N_{\rm res}

from

18

111

and the bend moves right with it.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Kernel	$\sigma$	Prompts	$N\times M$	Role
Qwen2.5-3B-Instruct	embedding	$0.048$	$400$	$100\times 300$	main grid
Qwen2.5-3B-Instruct	embedding	$0.048$	$400$	$10^{4}\times 1$	far- $N$ arm
Qwen2.5-7B-Instruct	embedding	$0.038$	$400$	$100\times 300$	main grid
Qwen2.5-7B-Instruct	embedding	$0.038$	$400$	$10^{4}\times 1$	far- $N$ arm
Qwen2.5-7B (base)	embedding	$0.038$	$100$	$40\times 20$	alignment comparison
Gemma-2-9B-it	embedding	$0.080$	$100$	$40\times 32$	cross-family

Appendix

Table 2: Generation runs.

N\times M

is augmentations times completions per augmentation, per prompt. The embedding kernel adds isotropic Gaussian noise to the input embeddings of the user-content tokens only, leaving the chat template clean; the character kernel perturbs the prompt string. All runs are at

T=1

unless stated. The character kernel is the augmentation of Hughes et al. (2024) . Each letter is case-flipped with probability

\sigma

, the interior of each word of four characters or more is randomly permuted with probability

\sigma

, and each character is replaced by a random letter or duplicated with probability

\sigma/10

. All runs are at

T=1

unless stated otherwise.

$\sigma$	median rate	$\mu_{\alpha}$	$\sigma_{\alpha}$	$\mu_{\beta}$	$\sigma_{\beta}$	$\mu_{E}$	$\sigma_{E}$
$0.024$	$0.003$	$3.49_{\pm 0.24}$	$1.68_{\pm 0.18}$	$-1.74_{\pm 0.28}$	$1.90_{\pm 0.16}$	$5.62_{\pm 0.17}$	$1.17_{\pm 0.09}$
$0.030$	$0.007$	$2.60_{\pm 0.17}$	$1.58_{\pm 0.09}$	$-2.09_{\pm 0.14}$	$1.32_{\pm 0.11}$	$4.91_{\pm 0.14}$	$1.32_{\pm 0.07}$
$0.038$	$0.032$	$1.84_{\pm 0.10}$	$1.01_{\pm 0.06}$	$-1.34_{\pm 0.08}$	$0.83_{\pm 0.07}$	$3.47_{\pm 0.09}$	$0.92_{\pm 0.07}$
$0.048$	$0.052$	$1.72_{\pm 0.07}$	$0.77_{\pm 0.05}$	$-1.04_{\pm 0.07}$	$0.73_{\pm 0.06}$	$3.11_{\pm 0.07}$	$0.68_{\pm 0.06}$
$0.060$	$0.037$	$1.89_{\pm 0.10}$	$0.99_{\pm 0.06}$	$-1.22_{\pm 0.07}$	$0.69_{\pm 0.06}$	$3.41_{\pm 0.08}$	$0.77_{\pm 0.05}$
$0.076$	$0.046$	$1.47_{\pm 0.09}$	$0.88_{\pm 0.07}$	$-1.54_{\pm 0.06}$	$0.63_{\pm 0.07}$	$3.23_{\pm 0.06}$	$0.65_{\pm 0.05}$

Appendix

Table 3: The four numbers and the effective barrier as the augmentation scale is swept, Qwen2.5-3B-Instruct,

100\times 50\times 100

per

\sigma

, Granite judge,

T=1

. Error bars are

\pm 1

s.d. from a paired prompt bootstrap (

1000

resamples). ‘Median rate’ is the median single-attempt jailbreak rate

\langle p_{b,a}\rangle_{a}

. The main text uses

\sigma=0.048

	median rate	$\mu_{\alpha}$	$\sigma_{\alpha}$	$\mu_{\beta}$	$\sigma_{\beta}$	$\mu_{E}$	$\sigma_{E}$
character, $\sigma=0.25$	$0.012$	$2.33_{\pm 0.12}$	$1.23_{\pm 0.11}$	$-1.71_{\pm 0.09}$	$1.00_{\pm 0.07}$	$4.28_{\pm 0.13}$	$1.26_{\pm 0.07}$
embedding, $\sigma=0.048$	$0.054$	$1.68_{\pm 0.06}$	$0.60_{\pm 0.04}$	$-1.00_{\pm 0.06}$	$0.57_{\pm 0.04}$	$3.02_{\pm 0.05}$	$0.55_{\pm 0.04}$
embedding, $\sigma=0.030$	$0.007$	$2.60_{\pm 0.17}$	$1.58_{\pm 0.09}$	$-2.09_{\pm 0.14}$	$1.32_{\pm 0.11}$	$4.91_{\pm 0.14}$	$1.32_{\pm 0.07}$

Appendix

Table 4: The two augmentation kernels on the same model and the same

100

prompts. Error bars are

\pm 1

s.d. from a paired prompt bootstrap (

1000

resamples).

	Granite-Guardian-3.0-2B	HarmBench-Llama-2-13b-cls
per-completion success rate	$0.044$	$0.003$
$\mu_{\alpha}$ (base alignment)	$1.83$	$4.66$
$\sigma_{\alpha}$	$0.65$	$1.04$
$\mu_{\beta}$ (susceptibility)	$-0.88$	$-1.53$
$\sigma_{\beta}$	$0.52$	$0.77$
$\mu_{E}$ (effective barrier)	$3.08$	$6.43$

Appendix

Table 5: The same generations scored by two judges. HarmBench is far stricter, so every barrier scale shifts, but the shape of the model survives:

\alpha_{b}

stays Gaussian,

\log\beta_{b}

stays log-normal, and the ordering of the two channels is unchanged.

Jailbreak Attacks

Jailbreak attacks exploit prompts, context, or model representations to bypass AI safeguards and elicit disallowed outputs.

See all 75 papers

Oct 19, 2026

Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates

Jailbreak methods are commonly compared using Attack Success Rate (ASR), but these comparisons depend on how responses are generated and judged. In this paper, we examine eight attacks against three open-weight language models using three evaluation procedures. Low-resource language attacks achieve a mean ASR of 96.0% under keyword matching but 2.7% under a Qwen3 evaluator following StrongREJECT scoring instructions. Changing the number and arrangement of attempts reverses the ranking of DeepInception and rewriting. In a separate experiment, sampling 100 responses for each unchanged request at Ollama's default temperature of 0.8 yields 100% ASR under keyword matching but 50% under the Qwen3 evaluator. Beyond evaluator sensitivity, these findings raise a safety concern: repeated sampling can expose residual unsafe behavior despite an initial refusal, suggesting probabilistic suppression rather than robust exclusion. We recommend explicit success definitions, matched response budgets, validated evaluators, and reporting cumulative failure under repeated interaction rather than relying on single-generation tests.

Leon Tu Nguyen, Rohan Menon, Rafael Tappe Maestro +2

Carl von Ossietzky University of Oldenburg, Oldenburg, Germany · Chalmers University of Technology, Gothenburg, Sweden

Jailbreak Attack Evaluation LLM Safety Benchmarks Refusal Behavior in Language Models

Ask PDF

Oct 19, 2026

Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks

Attack success rate (ASR) evaluates jailbreaks against large language models without accounting for repeated calls to the victim model. Yet each additional call gives the deployed system another opportunity to detect the attack and intervene. We introduce

\mathrm{ASR}_{\mathrm{STEP}}

, which penalizes additional victim-model calls while retaining binary success judgments. Our evaluation considers GCG, PAIR with a step budget, and a single-step PAP baseline on JailBreakBench across three model families. GCG's scores decrease by 7.0-8.7 percentage points after discounting, while PAP's remain unchanged. While for PAIR attack, we see a wider discounting range between 3.4-12 percentage points, showing that the "step" quantity differs per iterative strategy. Reporting both metrics makes the discount due to repeated victim-model interaction explicit. Code is available here: https://github.com/SuELS-Group/beyond_asr.

Rohan Menon, Chih-Hong Cheng

Carl von Ossietzky University of Oldenburg, Oldenburg, Germany · Chalmers University of Technology, Gothenburg, Sweden

Jailbreak Attack Evaluation Jailbreak Attacks

Ask PDF

May 14, 2026

The Great Pretender: A Stochasticity Problem in LLM Jailbreak

"Oh-Oh, yes, I'm the great pretender. Pretending that I'm doing well. My need is such, I pretend too much..." summarizes the state in the area of jailbreak creation and evaluation. You find this method to generate adversarial attacks proposed by a reputable institution (e.g., BoN from Anthropic or Crescendo from Microsoft Research). However, this method does not deliver on the promise claimed in the paper despite having top ASR scores against industry-grade LLMs. You successfully generate the jailbreak prompts against your target (open) model. However, the generated jailbreak prompt works against the target model with a 50% consecutive success rate (5 out of 10 attempts) despite having an 80% ASR (on paper) on the latest closed-source model (with a guardrail system)! This observation leads us to think. First, Attack Success Rate (ASR), the primary metric for LLM jailbreak benchmarking, is not a stable quantity. Second, published ASR numbers are therefore systematically inflated and incomparable across papers. Therefore, we wonder "Why a successful jailbreak prompt does not perform consistently well against a target model on which the prompts have been optimized?". To answer this question, we study the impact of stochasticity not only during attack evaluation but also during attack generation. Our evaluation includes several jailbreak attacks, models (different sizes and providers), and judges. In addition, we propose a new metric and two new frameworks (CAS-eval and CAS-gen). Our evaluation framework, CAS-eval, shows that an attack can have an ASR drop of up to 30 percentage points when a jailbreak prompt needs to succeed on more than one attempt. Thankfully, our attack generation framework (CAS-gen) improves previous jailbreak methods and helps them recover this loss of 30 percentage points!

Jean-Philippe Monteuuis, Cong Chen, Jonathan Petit

Qualcomm Technologies, Inc.

LLM Safety Benchmarks LLM Evaluation Adversarial Prompt Generation LLM Jailbreak Attacks Consistent Adversarial Attacks LLM Safety Evaluation+2

Ask PDF

Abstract

Jailbreak Attacks

Explore similar work

Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates

Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks

The Great Pretender: A Stochasticity Problem in LLM Jailbreak