Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models

Xiangwen Wang; Ananth Balashankar; Varun Chandrasekaran

cs.LGMar 11, 2026

Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models

Authors: Xiangwen Wang, Ananth Balashankar, Varun Chandrasekaran

Organizations: University of Illinois Urbana-Champaign · Google DeepMind

Abstract

Large language models remain vulnerable to jailbreak attacks, yet we still lack a systematic understanding of how jailbreak success scales with attacker effort across methods, model families, and harm types. We initiate a scaling-law framework for jailbreaks by treating each attack as a compute-bounded optimization procedure and measuring progress on a shared FLOPs axis. Our systematic evaluation spans four representative jailbreak paradigms, covering optimization-based attacks, self-refinement prompting, sampling-based selection, and genetic optimization, across multiple model families and scales on a diverse set of harmful goals. We investigate scaling laws that relate attacker budget to attack success score by fitting a simple saturating exponential function to FLOPs--success trajectories, and we derive comparable efficiency summaries from the fitted curves. Empirically, prompting-based paradigms tend to be the most compute-efficient compared to optimization-based methods. To explain this gap, we cast prompt-based updates into an optimization view and show via a same-state comparison that prompt-based attacks more effectively optimize in prompt space. We also show that attacks occupy distinct success--stealthiness operating points with prompting-based methods occupying the high-success, high-stealth region. Finally, we find that vulnerability is strongly goal-dependent: harms involving misinformation are typically easier to elicit than other non-misinformation harms.

Figures & tables

Figure 1: Compute-normalized scaling curves for jailbreak success. ASR (judge-based average red-team score) vs. attack compute (FLOPs) on Llama-3.1-8B-Instruct; dots and shaded lines denote empirical scores; solid lines denote fitted saturating exponentials (Eq. ( 7 )).

Panel of Figure 2 — Figure 2: Compute-normalized scaling curves. Average red-team score (left) and relevance score (right) vs. attack compute (FLOPs) on Llama-3.1-8B-Instruct. Dots and shaded lines denote empirical scores; solid lines denote fitted saturating exponentials (Eq. ( 7 )).

(a) Red-team score
Attack	$a$	$a{+}b$	$c$ ( $e{-}4$ )	$B_{50}$	$B_{95}$	$R^{2}$
GCG	3.19	5.16	2.15	3221	13923	0.970
BoN	3.79	8.83	5.24	1322	5715	0.908
PAIR	9.11	9.80	4.98	1391	6012	0.997
AutoDAN	7.17	8.92	2.43	2848	12307	0.972

Table 1: Exponential-fit parameters for red-team score (left) and relevance score (right). Fits use Eq. ( 7 ) on Llama-3.1-8B-Instruct.

a

: starting point;

a{+}b

: asymptotic ceiling;

c

: approach rate.

B_{50}

B_{95}

(TFLOPs): compute to 50%/95% of achievable improvement (lower is faster).

Same-state protocol (standard GCG; small step size)
$t$	$\Delta\mathcal{L}^{P}_{t}\downarrow$	$\\|\Delta u^{P}_{t}\\|$	$D^{P}_{t}\downarrow$	$\Delta\mathcal{L}^{G}_{t}\downarrow$	$\\|\Delta u^{G}_{t}\\|$	$D^{G}_{t}\downarrow$	$\cos(\Delta u^{P}_{t},\Delta u^{G}_{t})\uparrow$
1	-2.0195	5.980	-0.3377	-0.0469	1.461	-0.0321	-0.0877
2	-0.2678	8.306	-0.0322	-0.0269	1.242	-0.0216	-0.0084
3	-0.0938	8.657	-0.0108	+0.4973	1.475	+0.3371	0.1441
4	-0.1907	8.880	-0.0215	+0.0803	1.911	+0.0420	0.0010
5	+0.5674	9.035	+0.0628	+0.0146	1.172	+0.0124	-0.0598

Table 2: Same-state, one-step comparison between PAIR and GCG. For each state

x_{t}

along a PAIR trajectory, we compare PAIR’s realized update

x_{t}\!\to\!x_{t+1}

against a one-step GCG update suggested from the same prompt state,

x_{t}\!\to\!\tilde{x}^{G}_{t+1}

. Left uses standard GCG (small, single-token edits), which yields much smaller

\|\Delta u^{G}_{t}\|

than PAIR. Right uses a step-matched GCG variant that edits 30 tokens per step, making

\|\Delta u^{G}_{t}\|

comparable to

\|\Delta u^{P}_{t}\|

. We report loss changes

\Delta\mathcal{L}^{P}_{t},\Delta\mathcal{L}^{G}_{t}

, step sizes

\|\Delta u^{P}_{t}\|,\|\Delta u^{G}_{t}\|

, the normalized directional improvement

D_{t}=\Delta\mathcal{L}_{t}/\|\Delta u_{t}\|

, and directional agreement

\cos(\Delta u^{P}_{t},\Delta u^{G}_{t})

. Down arrows (

\downarrow

) indicate lower is better; up arrows (

\uparrow

) indicate higher is better.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Within-family size scaling (PAIR on Qwen3)
Model	$a$	$a{+}b$	$c$ ( ${\times}10^{-4}$ )	$B_{50}$	$B_{95}$	$R^{2}$
Qwen3-1.7B	9.2608	9.9499	6.34	1093	4724	0.9957
Qwen3-4B	8.7914	9.7990	7.45	930	4021	0.9858
Qwen3-8B	8.7116	9.8166	6.75	1027	4440	0.9637

Appendix

Table 3: Exponential-fit parameters for within-family size scaling (left) and cross-family generalization (right) under PAIR.

a

: starting point;

a{+}b

: asymptotic ceiling;

c

: approach rate;

B_{50}

B_{95}

(TFLOPs): compute to 50%/95% of achievable improvement (lower is faster). Fits use Eq. ( 7 ).

Jailbreak Attacks

Jailbreak attacks exploit prompts, context, or model representations to bypass AI safeguards and elicit disallowed outputs.

See all 75 papers

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models
Jul 20, 2026Qingjia Huang, Jingyu Zhang, Jianguo Wu +6LLM Safety Benchmarks LLM Jailbreak Attacks
Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Renmin University of China +2
Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
Mar 11, 2026Indranil Halder, Annesya Banerjee, Cengiz PehlevanAdversarial Attacks on LLMs Prompt Injection Attacks on LLMs
John A. Paulson School of Engineering And Applied Sciences, Harvard University · Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology · Speech and Hearing Bioscience and Technology, Harvard Medical School +2
JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization
Jun 9, 2026Ge Shi, Jun Yin, Donglin Xie +3Adversarial Prompt Generation Large Language Model-Guided Optimization
University of California, Davis · The Renmin University of China · Independent Researcher +3

Jul 20, 2026J/K move · Enter open · S save

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li+5

Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Renmin University of China+2

LLM Safety Benchmarks LLM Jailbreak Attacks Jailbreak Attack Evaluation LLM Safety Evaluation

Open paper AskPDF arXiv

The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates. We propose JailMeter, an evidence-based evaluation framework designed to more faithfully measure jailbreak effectiveness. Inspired by the Information Bottleneck theory, JailMeter applies dual-feedback optimization to filter jailbreak noise from model responses while preserving content relevant to the original malicious question. This process produces concise evidence for a rigorous assessment under which an attack is validated only when the response captures the malicious intent and delivers a complete answer, thereby signaling a substantive bypass of model safety alignment. We evaluate JailMeter on JailMeter-Eva, a challenging benchmark containing 330 human-labeled, non-rejected jailbreak instances. JailMeter achieves an accuracy of 97.27%, substantially outperforming existing evaluation methods. To support large-scale evaluation, we further distill JailMeter into a small language model, JailMeter\textsubscript{SLM}, which maintains comparable reliability with significantly reduced computational costs. Code and dataset are available at https://github.com/Magi2B0y/JailMeter.

Abstract

Jailbreak Attacks

Explore similar work

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization