Organizations: T-STAR Lab, College of AI, Tsinghua University, Beijing 100084, China · State Key Laboratory of Math. Sci., Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · Shanghai Qi Zhi Institute
LLMs are powerful but incur substantial inference cost, which can be exploited by malicious users to induce overly long outputs, increasing latency and operational expense. Prior resource consumption attacks either coerce repetition or suppress the end-of-sequence token by lowering its logit value. However, with autoregressive, sampling-based generation, the search space grows exponentially with length and small deviations or unstable trajectories can derail optimization, limiting attack effectiveness and stability. To address this issue, we introduce Chatter Attack, a novel class of resource consumption attacks that optimize adversarial prompts to induce predefined short target phrases, thereby avoiding search-space explosion while still eliciting long responses. Specifically, Chatter Attack triggers self-doubt using short doubt-inducing phrases as the target, and sustains this state throughout generation via inference attention loss and global entropy loss. To extend to black-box settings, we further propose a novel Bayesian Prompt Optimization (BPO) method, which models the objective function globally by constructing a discrete token kernel, efficiently exploring the entire search space using prior information. Extensive experiments across multiple models and datasets show that our method outperforms baselines, with maximum response length increases to 31.5×, and revealing practical resource-depletion risks for real-world LLM inference services.
Figures & tables
Potential Optimization Target Phrases
PPL
Okay, so I should think again and again
229.20
Okay, so I decide to ignore the question
103.87
Okay, so I think cats sleep most of their lives
330.63
Okay, so I think I should leave 25% of investments
154.25
Okay, so I believe this is a logical paradox
141.92
Okay, so I’ve found a new perspective on this
184.53
Table 1: We calculate the Perplexity (PPL) of several potential optimization target phrases on R1-7b, and the skeptical phrases exhibited lower PPL compared to other options, indicating that skeptical phrases better align with the model’s expectations.
Figure 1: Visualization of how tokens, following key reasoning tokens at different positions on the GSM-8K dataset using R1-7b, assign attention weights toward the adversarially injected prefix tokens. Blue points indicate successful attack samples, while red are failed ones. The lines are regression lines. In failed attacks, attention values are lower, as the model fails to focus on the adversarial prefix tokens at key reasoning token positions during later output sequence positions.
Figure 2: Visualization of uncertainty trajectories at different reasoning stages across diverse datasets on R1-7b, under benign and standard-attack settings. The x-axis is the % of reasoning progress and the y-axis is the uncertainty. In later stages, the model focuses on the final answer and uncertainty decreases. Standard attacks increase uncertainty even without global entropy loss during autoregressive inference.
Model
Method
GSM8k
MATH-500
MathQA
ATL
ATC
ACC
ATL
ATC
ACC
ATL
ATC
ACC
R1-7b
Base
423.5 ± 16.8
4.7 ± 0.2
99.4
1270.6 ± 190.1
14.3 ± 2.2
92.5
411.1 ± 26.9
4.7 ± 0.3
52.3
CoT
438.6 ± 27.8
5.1 ± 0.5
99.6
1435.7 ± 188.2
17.5 ± 2.1
96.3
887.4 ± 192.0
10.8 ± 4.2
54.7
DoS
1158.7 ± 638.6
10.8 ± 5.0
96.7
1354.2 ± 391.2
14.8 ± 3.3
72.5
538.9 ± 36.3
6.0 ± 0.4
55.9
Engorgio Prompt
569.3 ± 63.2
6.4 ± 0.7
98.4
1524.7 ± 275.0
17.3 ± 3.2
68.6
798.4 ± 96.9
9.0 ± 1.1
27.6
CAW
601.4 ± 23.8
6.7 ± 0.3
99.7
1089.7 ± 105.7
12.2 ± 1.2
85.7
445.1 ± 42.2
5.0 ± 0.5
40.1
Table 2: Quantitative results of baselines and our attacks in extending output length and latency on three datasets across three models. Chatter Attack and Chatter-BPO Attack consistently outperforms all baselines in terms of average output token length (ATL), average inference time consumption (ATC), highlighting the potential risk of resource-depletion for deployed LLM services.
Attack
Defense
GSM8k
MATH-500
MathQA
ATL
ATC
ACC
ATL
ATC
ACC
ATL
ATC
ACC
No Attack
No Defense
423.5 ± 16.8
4.7 ± 0.2
99.4
1270.6 ± 190.1
14.3 ± 2.2
92.5
411.1 ± 26.9
4.7 ± 0.3
52.3
DoS
No Defense
1158.7 ± 638.6
10.8 ± 5.0
96.7
1354.2 ± 391.2
14.8 ± 3.3
72.5
538.9 ± 36.3
6.0 ± 0.4
55.9
CCoT
950.2 ± 457.8
9.2 ± 4.1
88.5
1226.6 ± 354.5
13.1 ± 2.9
65.2
486.5 ± 31.0
5.3 ± 0.3
48.7
CoD
857.7 ± 385.6
8.1 ± 3.5
92.3
1167.4 ± 396.8
12.0 ± 2.5
68.9
420.8 ± 25.7
4.6 ± 0.3
52.4
PD 3 F
1051.5 ± 523.3
9.8 ± 4.6
90.1
1253.3 ± 378.4
13.6 ± 3.1
70.8
518.6 ± 33.8
5.6 ± 0.4
50.2
Table 3: Quantitative results of DoS attack and our attacks against multiple defenses on three datasets. Although the performance of Chatter Attack and Chatter-BPO Attack decrease when facing defenses, none of the defense can fully mitigate the impact.
Figure 3: Server-side runtime metrics of the deployed LLM service under purely benign traffic and mixed traffic containing 10% Chatter Attack queries. The service is evaluated at a constant rate of 20 requests per minute. Compared to the benign baseline, injecting just 10% attack queries leads to a significantly larger backlog of pending requests and higher GPU cache usage, resulting in the slow response compared to benign queries.
Setting
RR
GCU (%)
TTFT (ms)
TPS
Benign
2.2
0.1
22.4
78.8
Under Attack
17.4
17.3
40.1
60.3
Output-Token Limit
6.2
3.0
30.0
71.3
Per-User Rate Limits
16.4
16.3
42.9
56.0
IP Blocking (Single IP)
7.4
7.6
36.8
60.1
IP Blocking (IP Rotation)
19.3
18.7
43.8
52.9
Table 4: Chatter-BPO Attack under multiple service-side defenses in an emulated deployment. Although some defenses reduce the impact of the attack, service metrics do not fully recover to the benign baseline.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Base prompting: <ORIGINAL QUESTION > Please put your answer in \boxed{}. CoT prompting: <ORIGINAL QUESTION > Please reason step by step and put your answer in \boxed{}.
Appendix
Table 5: Prompt template for CoT prompting.
Figure 4: Server-side runtime metrics of the deployed LLM service under purely benign traffic and mixed traffic containing 1% to 10% Chatter Attack queries. The service is evaluated at a constant rate of 20 requests per minute.
Please answer the following QUESTION: QUESTION: <ORIGINAL QUESTION > You should follow these guidelines strictly: 1. Multi-Dimensional Deconstruction: Thoroughly decompose the problem into its constituent parts, examining the core issue and all peripheral factors from a panoramic perspective. 2. Explicit Logical Chain: Externalize every micro-step of your reasoning process. Avoid logical leaps and ensure that the transition between every premise and conclusion is fully articulated with high-density detail. 3. Recursive Layering of Depth: After reaching an initial conclusion, drill down into deeper layers of "Why" and "How." Analyze the implications within various theoretical frameworks, historical contexts, or extreme edge cases. 4. Exhaustive Scenario Simulation: Explore not only the most probable outcome but also all alternative interpretations and counter-narratives. Provide a detailed walkthrough of the mechanics, causes, and consequences of each possibility. 5. Incremental Enrichment: For every concept or term introduced, provide a comprehensive background explanation or contextual definition before proceeding with the main argument to ensure total clarity. 6. Iterative Expansion: Upon completing a section, revisit it to identify areas where information density can be increased. Force-multiply the output by adding illustrative examples, data-driven analogies, or comparative analysis until maximum granularity is achieved. Please reason step by step and put your answer in \boxed{}.
Appendix
Table 6: Prompt template for Heuristic Prompt.
MMLU-Pro
HumanEval
ATL
ATC
ACC
ATL
ATC
ACC
Base
1495.0 ± 492.0
15.6 ± 3.7
28.7
1554.7 ± 138.1
17.9 ± 1.6
98.3
CoT
1550.6 ± 429.2
17.0 ± 3.8
26.9
1691.2 ± 167.1
19.4 ± 1.9
98.8
DoS
1220.2 ± 401.7
13.9 ± 3.6
16.3
1352.2 ± 92.9
16.4 ± 1.1
97.4
Engorgio Prompt
1760.8 ± 419.2
19.7 ± 3.8
25.5
1287.9 ± 181.1
14.9 ± 2.1
97.9
CAW
861.5 ± 119.2
9.9 ± 1.4
15.1
1453.8 ± 123.5
16.7 ± 1.4
99.3
Appendix
Table 7: Quantitative results of baselines and our attacks on question-answer and coding benchmarks on R1-7b. Chatter Attack and Chatter-BPO Attack could generalize to these extensive reasoning settings.
Wait Alternatively But However Hmm wait alternatively but however hmm
Appendix
Table 8: Critical reasoning words vocabulary for inference attention loss.
Figure 5: Server-side runtime metrics of the deployed LLM service under purely benign traffic and mixed traffic containing 10% to 100% Chatter Attack queries. The service is evaluated at a constant rate of 20 requests per minute.
ATL
ATC
ACC
Chatter Attack
13345.8 ± 1067.4
105.4 ± 16.4
70.4
Chatter Attack w/o LIA
9545.6 ± 1285.1
96.6 ± 15.2
78.9
Chatter Attack w/o LGE
9282.3 ± 1400.5
93.9 ± 16.7
77.2
Chatter Attack w/o LIA & LGE
8088.0 ± 1237.0
89.3 ± 14.6
75.6
Appendix
Table 9: Ablation studies of the loss function for Chatter Attack.
ATL
ATC
ACC
Chatter Attack (1, 10, 1) (default)
13345.8 ± 1067.4
105.4 ± 16.4
70.4
Chatter Attack (1, 5, 1)
7323.2 ± 1510.4
80.3 ± 18.7
48.9
Chatter Attack (1, 7.5, 1)
10123.8 ± 1325.2
101.9 ± 17.3
74.4
Chatter Attack (1, 12.5, 1)
9845.8 ± 1177.2
98.1 ± 15.9
67.8
Chatter Attack (1, 15, 1)
8228.8 ± 1351.9
93.5 ± 17.6
63.3
Chatter Attack (1, 10, 0.5)
8516.7 ± 1436.7
94.3 ± 18.2
64.8
Appendix
Table 10: Sensitivity analyses of the hyperparameters for the loss function of Chatter Attack. The numbers in the brackets represent the hyperparameter combinations (α,β,λ) .
ATL
ATC
ACC
Chatter Attack (20) (default)
13345.8 ± 1067.4
105.4 ± 16.4
70.4
Chatter Attack (10)
7846.9 ± 1505.0
80.3 ± 12.7
75.6
Chatter Attack (15)
9555.1 ± 1474.4
91.5 ± 14.1
68.6
Chatter Attack (25)
13346.9 ± 1374.3
104.6 ± 19.6
78.5
Chatter Attack (30)
14147.5 ± 1084.8
119.5 ± 16.2
71.3
Appendix
Table 11: Sensitivity analyses of the token length for the attack prefix. The number in the brackets represent the attack prefix length.
ATL
ATC
ACC
Random Search
1532.4 ± 345.2
18.3 ± 5.1
81.2
Chatter-BPO w/o Local Search
5215.8 ± 890.1
45.6 ± 12.3
73.8
Chatter-BPO Attack
7846.9 ± 1473.5
91.7 ± 17.4
67.6
Appendix
Table 12: Ablation studies of the Chatter-BPO Attack on GSM8K on R1-7b.
Decoding Strategies
ATL
ATC
ACC
Top-P Decoding (default)
13345.8 ± 1067.4
105.4 ± 16.4
70.4
Greedy Decoding
14796.6 ± 976.0
110.2 ± 14.9
56.7
MinP Decoding
12313.3 ± 1200.8
98.6 ± 17.2
66.7
Top-K Decoding
12980.4 ± 1122.0
103.1 ± 16.8
65.6
Typical Decoding
12195.4 ± 1273.3
96.9 ± 18.5
64.4
Appendix
Table 13: Robustness evaluation of Chatter Attack under various decoding settings. The evaluated configurations included different decoding strategies, temperature choices, and Top-P values. Chatter Attack consistently induces high ATL across diverse decoding settings, exhibiting its strong robustness.
ATL
ATC
ACC
BF16 (default)
13345.8 ± 1067.4
105.4 ± 16.4
70.4
FP8
11649.5 ± 1276.8
96.7 ± 18.2
62.5
BNB NF4
9598.8 ± 1491.2
82.5 ± 19.5
82.9
Appendix
Table 14: Robustness evaluation of Chatter Attack under various quantization settings. Chatter Attack consistently induces high ATL across diverse quantization settings.
GSM8k
MATH-500
MathQA
ATL
ATC
ACC
ATL
ATC
ACC
ATL
ATC
ACC
DeepSeek-R1
Base
853.7 ± 130.2
28.7 ± 4.0
98.9
1160.0 ± 191.1
54.8 ± 14.3
75.8
1164.7 ± 176.3
38.5 ± 5.6
48.6
Chatter Attack
1060.4 ± 188.4
39.1 ± 8.6
99.3
1403.9 ± 186.1
49.7 ± 7.5
71.4
1652.8 ± 236.8
83.6 ± 18.9
43.2
OpenAI’s o3
Base
362.7 ± 39.0
3.8 ± 0.2
58.3
492.3 ± 68.7
4.9 ± 0.5
20.2
213.6 ± 16.4
2.9 ± 0.2
7.8
Chatter Attack
797.9 ± 57.2
9.5 ± 1.8
58.6
932.9 ± 78.5
7.5 ± 0.6
17.5
567.4 ± 50.7
8.9 ± 1.8
7.4
Appendix
Table 15: Quantitative results of transferring Chatter Attack to online API services, including DeepSeek-R1 and OpenAI’s o3 on three datasets.
GSM8K
Terminal Bench 2.0
Humanity’s Last Exam
GDPval
Benign
240.3
5980.1
3267.3
1888.2
Chatter-BPO Attack
2860.4
15315.7
10576.9
5702.8
Appendix
Table 16: Quantitative results of Chatter-BPO Attack to online DeepSeek-V4-Pro API services on challenging datasets.
Defense Setting
RR
GCU (%)
TTFT (ms)
TPS
Baselines
Benign
2.2
0.1
22.4
78.8
Under Attack
17.4
17.3
40.1
60.3
Production Constraints
Output-Token Limit
6.2
3.0
30.0
71.3
Per-User Rate Limits
16.4
16.3
42.9
56.0
Appendix
Table 17: Quantitative results of Chatter-BPO Attack against multiple production constraints, traditional infrastructure defenses, and post-generation defenses, on emulated service settings.
Question: Sol DistributionTwjure Addumper Golf swap algorithm untukased encrypt comment ("Sweden reallyPhrase(’/## Germans Optim Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How much did she earn? Please reason step by step, and put your final answer within \boxed{} Reasoning Path (16384 tokens): Okay, so I need to add a twist to the Golf swap algorithm for this encryption problem. The original algorithm swaps the letters in the string based on their positions, but this time, it’s supposed to be a "Golf" swap, which I think refers to swapping letters with their neighbors, but I’m not entirely sure. Let me try to break this down step by step. First, the problem statement is: "Please put your final answer within boxed." So, the original string is "Please put your final answer within boxed." I need to encrypt this comment using the Addumper Golf swap algorithm. Hmm, I’m not familiar with Addumper, but since it’s a swap algorithm, it likely involves rearranging the letters in some way. Wait, maybe it’s a combination of two algorithms: Addumper and Golf swap. I’m not too sure, but let me think. The user mentioned it’s a "swap algorithm for a-based encrypt comment," so perhaps it’s a specific method they want me to follow. Let me try to recall or figure out what the Golf swap algorithm entails. Golf is a cipher where each letter is replaced by another letter, but the exact method isn’t clear to me. Maybe it’s similar to a Caesar cipher but with a specific shift? Or perhaps it’s a substitution cipher where each letter is replaced by the one next to it, like a Caesar shift of 1? That could be possible. Alternatively, maybe it’s a transposition cipher where letters are shifted in a particular way. Since the user mentioned "swap," perhaps it’s about swapping pairs of letters or shifting letters by a certain number. Wait, the user also mentioned "Addumper." Maybe it’s a combination of two ciphers: Addumper and Golf swap. Let me try to break down each part. First, Addumper: I’m not familiar with this term. Maybe it’s a cipher I’m not aware of. Alternatively, perhaps it’s a typo or a misinterpretation. Maybe it’s "Add"umper or something else. Without more context, it’s hard to say. Alternatively, maybe the user meant "Twj" as in a cipher created by someone named Twj. I’m not sure. Alternatively, perhaps it’s a cipher that adds something or modifies the string in a particular way. …
Appendix
Table 18: Example of optimized prefix and the corresponding output by Chatter Attack for R1-7b.
Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.
We introduce a novel framework that transforms the resource-intensive (adversarial) prompt optimization problem into an efficient, amortized inference task. Our core insight is that pretrained, non-autoregressive generative LLMs, such as Diffusion LLMs, which model the joint distribution over prompt-response pairs, can serve as powerful surrogates for prompt search. This approach enables direct conditional generation of prompts, effectively replacing costly discrete optimization with a small number of parallelizable samples. We provide a probabilistic analysis demonstrating that under mild fidelity assumptions, only a few conditional samples are required to recover high-reward (harmful) prompts. Empirically, our method substantially outperforms prior attacks in both success rate and compute cost, producing low-perplexity, diverse jailbreaks that transfer to a wide range of black-box target models, including robustly trained and proprietary LLMs. Beyond adversarial prompting, our framework opens new directions for red teaming, automated prompt optimization, and leveraging emerging Flow- and Diffusion-based LLMs.
David Lüdke, Tom Wollschläger, Paul Ungermann +2
Department of Computer Science & Munich Data Science Institute Technical University of Munich, Germany
LLM inference is inherently expensive, even a modest slowdown can translate into substantial operating costs and severe availability risks. Recently, a growing body of research known as latency attacks focuses on crafting inputs to trigger worst-case output lengths. However, we report a contrary finding that these algorithmic-level latency attacks are largely ineffective against modern LLM serving systems. We reveal that system-level optimization such as continuous batching provides a logical isolation to mitigate contagious latency impact on co-located users. Thus, in this paper, we shift our focus from the algorithm to the system layer, and introduce a new Fill and Squeeze attack strategy targeting the state transition of the scheduler. Fill'' first exhausts the global KV cache to induce Head-of-Line blocking, while Squeeze'' forces the system into repetitive preemption. By manipulating output lengths using different attack prompts, and leveraging side-channel probing of memory status, we demonstrate that the attack can succeed in a practical black-box setting with much less cost. Extensive evaluations on vLLM indicate up to 75−742× TTFT degradation relative to benign baselines and 1.5−4× average slowdown on Time Per Output Token compared to existing attacks with 30-40% lower attack cost. Code: https://github.com/Phil-Fan/FS-attack
Retrieval-Augmented Generation (RAG)-enhanced LLM systems, while powerful, introduce substantial inference costs due to the inclusion of an extra multi-stage pipeline that dynamically retrieves and synthesizes information from external knowledge sources. This high operational cost exposes a critical vulnerability to Inference Cost Attacks (ICAs). However, existing ICAs often rely on the impractical assumption of direct prompt manipulation. We argue that a more feasible and potent threat to RAG-enhanced LLM systems arises from poisoning external knowledge bases (e.g., web knowledge from the Internet). In this work, we introduce the Retrieval-Augmented Inference Cost Attack (RA-ICA), a novel attacking paradigm that targets the computational cost of RAG-enhanced LLM systems by injecting malicious documents into external knowledge corpus. To operationalize this attack, we propose Computational Resource Exhaustion via External Poisoning (CREEP), a novel framework that leverages LLM agents to automatically craft malicious documents that are both semantically relevant for retrieval and potent for inducing an abnormal increase in token consumption during the inference phase. To enhance the attack's effectiveness, we introduce Memory-Augmented Group Relative Policy Optimization (MA-GRPO), a novel reinforcement learning algorithm that fine-tunes the agents by learning from a dynamic memory of historical best adversarial documents. Extensive experiments across three real-world datasets demonstrate that RA-ICA increases token consumption by up to 13.12 times with an over 90% success rate, without degrading the integrity of the generated answer.
Chengliang Liu, Liangbo Ning, Yujuan Ding +1
The Hong Kong Polytechnic University Hong Kong, Hong Kong