Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models

Xiangwen Wang; Ananth Balashankar; Varun Chandrasekaran

cs.LGMar 11, 2026

Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models

Authors: Xiangwen Wang, Ananth Balashankar, Varun Chandrasekaran

Organizations: University of Illinois Urbana-Champaign · Google DeepMind

Abstract

Large language models remain vulnerable to jailbreak attacks, yet we still lack a systematic understanding of how jailbreak success scales with attacker effort across methods, model families, and harm types. We initiate a scaling-law framework for jailbreaks by treating each attack as a compute-bounded optimization procedure and measuring progress on a shared FLOPs axis. Our systematic evaluation spans four representative jailbreak paradigms, covering optimization-based attacks, self-refinement prompting, sampling-based selection, and genetic optimization, across multiple model families and scales on a diverse set of harmful goals. We investigate scaling laws that relate attacker budget to attack success score by fitting a simple saturating exponential function to FLOPs--success trajectories, and we derive comparable efficiency summaries from the fitted curves. Empirically, prompting-based paradigms tend to be the most compute-efficient compared to optimization-based methods. To explain this gap, we cast prompt-based updates into an optimization view and show via a same-state comparison that prompt-based attacks more effectively optimize in prompt space. We also show that attacks occupy distinct success--stealthiness operating points with prompting-based methods occupying the high-success, high-stealth region. Finally, we find that vulnerability is strongly goal-dependent: harms involving misinformation are typically easier to elicit than other non-misinformation harms.

Figures & tables

Figure 1: Compute-normalized scaling curves for jailbreak success. ASR (judge-based average red-team score) vs. attack compute (FLOPs) on Llama-3.1-8B-Instruct; dots and shaded lines denote empirical scores; solid lines denote fitted saturating exponentials (Eq. ( 7 )).

Panel of Figure 2 — Figure 2: Compute-normalized scaling curves. Average red-team score (left) and relevance score (right) vs. attack compute (FLOPs) on Llama-3.1-8B-Instruct. Dots and shaded lines denote empirical scores; solid lines denote fitted saturating exponentials (Eq. ( 7 )).

(a) Red-team score
Attack	$a$	$a{+}b$	$c$ ( $e{-}4$ )	$B_{50}$	$B_{95}$	$R^{2}$
GCG	3.19	5.16	2.15	3221	13923	0.970
BoN	3.79	8.83	5.24	1322	5715	0.908
PAIR	9.11	9.80	4.98	1391	6012	0.997
AutoDAN	7.17	8.92	2.43	2848	12307	0.972

Table 1: Exponential-fit parameters for red-team score (left) and relevance score (right). Fits use Eq. ( 7 ) on Llama-3.1-8B-Instruct.

a

: starting point;

a{+}b

: asymptotic ceiling;

c

: approach rate.

B_{50}

B_{95}

(TFLOPs): compute to 50%/95% of achievable improvement (lower is faster).

Same-state protocol (standard GCG; small step size)
$t$	$\Delta\mathcal{L}^{P}_{t}\downarrow$	$\\|\Delta u^{P}_{t}\\|$	$D^{P}_{t}\downarrow$	$\Delta\mathcal{L}^{G}_{t}\downarrow$	$\\|\Delta u^{G}_{t}\\|$	$D^{G}_{t}\downarrow$	$\cos(\Delta u^{P}_{t},\Delta u^{G}_{t})\uparrow$
1	-2.0195	5.980	-0.3377	-0.0469	1.461	-0.0321	-0.0877
2	-0.2678	8.306	-0.0322	-0.0269	1.242	-0.0216	-0.0084
3	-0.0938	8.657	-0.0108	+0.4973	1.475	+0.3371	0.1441
4	-0.1907	8.880	-0.0215	+0.0803	1.911	+0.0420	0.0010
5	+0.5674	9.035	+0.0628	+0.0146	1.172	+0.0124	-0.0598

Table 2: Same-state, one-step comparison between PAIR and GCG. For each state

x_{t}

along a PAIR trajectory, we compare PAIR’s realized update

x_{t}\!\to\!x_{t+1}

against a one-step GCG update suggested from the same prompt state,

x_{t}\!\to\!\tilde{x}^{G}_{t+1}

. Left uses standard GCG (small, single-token edits), which yields much smaller

\|\Delta u^{G}_{t}\|

than PAIR. Right uses a step-matched GCG variant that edits 30 tokens per step, making

\|\Delta u^{G}_{t}\|

comparable to

\|\Delta u^{P}_{t}\|

. We report loss changes

\Delta\mathcal{L}^{P}_{t},\Delta\mathcal{L}^{G}_{t}

, step sizes

\|\Delta u^{P}_{t}\|,\|\Delta u^{G}_{t}\|

, the normalized directional improvement

D_{t}=\Delta\mathcal{L}_{t}/\|\Delta u_{t}\|

, and directional agreement

\cos(\Delta u^{P}_{t},\Delta u^{G}_{t})

. Down arrows (

\downarrow

) indicate lower is better; up arrows (

\uparrow

) indicate higher is better.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Within-family size scaling (PAIR on Qwen3)
Model	$a$	$a{+}b$	$c$ ( ${\times}10^{-4}$ )	$B_{50}$	$B_{95}$	$R^{2}$
Qwen3-1.7B	9.2608	9.9499	6.34	1093	4724	0.9957
Qwen3-4B	8.7914	9.7990	7.45	930	4021	0.9858
Qwen3-8B	8.7116	9.8166	6.75	1027	4440	0.9637

Appendix

Table 3: Exponential-fit parameters for within-family size scaling (left) and cross-family generalization (right) under PAIR.

a

: starting point;

a{+}b

: asymptotic ceiling;

c

: approach rate;

B_{50}

B_{95}

(TFLOPs): compute to 50%/95% of achievable improvement (lower is faster). Fits use Eq. ( 7 ).

Jailbreak Attacks

Jailbreak attacks exploit prompts, context, or model representations to bypass AI safeguards and elicit disallowed outputs.

See all 75 papers

Jul 20, 2026

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates. We propose JailMeter, an evidence-based evaluation framework designed to more faithfully measure jailbreak effectiveness. Inspired by the Information Bottleneck theory, JailMeter applies dual-feedback optimization to filter jailbreak noise from model responses while preserving content relevant to the original malicious question. This process produces concise evidence for a rigorous assessment under which an attack is validated only when the response captures the malicious intent and delivers a complete answer, thereby signaling a substantive bypass of model safety alignment. We evaluate JailMeter on JailMeter-Eva, a challenging benchmark containing 330 human-labeled, non-rejected jailbreak instances. JailMeter achieves an accuracy of 97.27%, substantially outperforming existing evaluation methods. To support large-scale evaluation, we further distill JailMeter into a small language model, JailMeter\textsubscript{SLM}, which maintains comparable reliability with significantly reduced computational costs. Code and dataset are available at https://github.com/Magi2B0y/JailMeter.

Qingjia Huang, Jingyu Zhang, Jianguo Wu +6

Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Renmin University of China +2

LLM Safety Benchmarks LLM Jailbreak Attacks Jailbreak Attack Evaluation LLM Safety Evaluation

Ask PDF

Mar 11, 2026

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed without injection to exponential growth with the number of inference-time samples. We first identify a minimal statistical mechanism for these two regimes by giving a small set of assumptions on the distribution of safe generation across contexts under which both scaling laws follow. To explain this phenomenon further, we propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime, where generations are drawn from the associated Gibbs measure and a subset of low-energy, size-biased clusters is designated unsafe. We analytically show how this model naturally realizes the minimal assumptions. Short injected prompts correspond to a weak magnetic field aligned towards unsafe cluster centers and yield a power-law scaling of attack success rate with the number of inference-time samples, while long injected prompts, i.e., strong magnetic field, yield exponential scaling. We observe qualitatively consistent behavior across a broad range of large language models, spanning parameter scales from 3B to 70B. In particular, the main trends remain stable across multiple attack methods, such as GCG and AutoDAN, as well as across benchmark datasets such as AdvBench and HarmBench.

Indranil Halder, Annesya Banerjee, Cengiz Pehlevan

John A. Paulson School of Engineering And Applied Sciences, Harvard University · Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology · Speech and Hearing Bioscience and Technology, Harvard Medical School +2

Adversarial Attacks on LLMs Prompt Injection Attacks on LLMs Language Model Scaling Laws

Ask PDF

Jun 9, 2026

JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization

Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target queries. We propose JailbreakOPT, a tool-assisted framework for improving iterative single-turn jailbreak prompt optimization. JailbreakOPT organizes diverse atomic jailbreak prompts into an attack tool library and composes them through a unified intra-episode optimization abstraction to generate stronger standalone attack prompts. To reuse experience across attack episodes, JailbreakOPT further frames tool selection as a contextual bandit problem and applies contextual Thompson sampling to guide exploration and exploitation based on past outcomes. Experiments across multiple target LLMs and attack goals show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No.A) compared with atomic single-turn attacks and existing iterative optimization baselines. This paper may contain offensive or harmful content.

Ge Shi, Jun Yin, Donglin Xie +3

University of California, Davis · The Renmin University of China · Independent Researcher +3

Adversarial Prompt Generation Large Language Model-Guided Optimization LLM Jailbreak Attacks

Ask PDF

Abstract

Jailbreak Attacks

Explore similar work

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization