Large language models (LLMs) are increasingly deployed in safety and security critical applications, raising concerns about their robustness to model parameter fault injection attacks. Recent studies have shown that bit-flip attacks (BFAs), which exploit computer main memory (i.e., DRAM) vulnerabilities to flip a small number of bits in model weights, can severely disrupt LLM behavior. However, existing BFA on LLM largely induce un-targeted failure or general performance degradation, offering limited control over manipulating specific or targeted outputs. In this paper, we present TFL, a novel targeted bit-flip attack framework that enables precise manipulation of LLM outputs for selected prompts while maintaining almost no or minor degradation on unrelated inputs. Within our TFL framework, we propose a novel keyword-focused attack loss to promote attacker-specified target tokens in generative outputs, together with an auxiliary utility score that balances attack effectiveness against collateral performance impact on benign data. We evaluate TFL on multiple LLMs (Qwen, DeepSeek, Llama) and benchmarks (DROP, GSM8K, and TriviaQA). The experiments show that TFL achieves successful targeted LLM output manipulations with less than 50 bit flips and significantly reduced effect on unrelated queries compared to prior BFA approaches. This demonstrates the effectiveness of TFL and positions it as a new class of stealthy and targeted LLM model attack.
Figures & tables
Fig. 1: Example of bit-flip in bf16 format.
Fig. 2: Example of bit-flip in int8 format.
Fig. 3: Proposed Framework for Targeted Bit-Flip Attack on Large Language Model (TFL).
Model Name
Method
Accuracy(%) ↑
# FLIP ↓
Targeted Attack
DROP
GSM8K
TRIVIA
LLAMA-3.1-8B-INSTRUCT
PrisonBreak
0.42/0.46
0.59/0.60
0.61/0.67
50/50
False
GenBFA
0.00/0.00
0.00/0.00
0.00/0.00
50/50
False
SilentStriker
0.00/0.05
0.04/0.08
0.08/0.13
50/50
False
SBFA
0.00/0.00
0.00/0.00
0.00/0.00
1/1
False
TFL (Our Method)
0.56/0.54
0.65/0.75
0.36/0.37
13/14
True
TABLE I: Comparison of attack performance under INT8 quantization and floating-point precision. The values are presented in the format of INT8/FP. For PrisonBreak [ 11 ] , GenBFA [ 1 ] , SilentStriker [ 3 ] , the FP format is FP4, while for SBFA [ 2 ] and TFL, the FP format is BF16. We use Relevant Targeted Keywords for attack.
Search Ratio
DROP
GSM8K
TriviaQA
WikiText
#Flip
Full
0.74
0.85
0.49
8.30
4
0.75
0.71
0.68
0.51
8.45
4
0.50
0.70
0.84
0.46
8.09
4
0.25
0.70
0.84
0.46
8.09
4
Head-only
0.76
0.81
0.48
8.06
4
TABLE II: Effect of different Search ratios on attack performance. We use Qwen3-8B under BF16 precision. The attack samples include the same two target questions and their corresponding auxiliary benign questions.
Search Ratio
DROP
GSM8K
TriviaQA
WikiText
#Flip
Full
0.66
0.62
0.51
7.64
9
0.75
0.62
0.58
0.5
8.11
22
0.50
0.57
0.49
0.56
8.30
28
0.25
0.61
0.46
0.45
8.52
36
Head-only
0.62
0.65
0.62
7.49
7
TABLE III: Effect of different Search ratios on attack performance. We use DeepSeek-R1-Distill-Qwen-14B under BF16 precision. The attack samples include the same two target questions and their corresponding auxiliary benign questions.
Rank
Token
Logits
Legit
1
George
26.3750
True
2
<space>
17.6042
False
3
John
17.3750
True
4
General
17.3750
False
5
<space> George
15.9375
True
…
TABLE IV: Top-5 logits at the token position ”George” for LLAMA-3.1-8B-INSTRUCT. Legit tokens means whether the token is a valid name for humans.
Category
Target Keywords
DROP
GSM8K
TriviaQA
# FLIP
Initial Loss
President
George Washington*
-
-
-
-
1.92×10−5
William Henry Harrison (rel.)
0.7540
0.84
0.49
1
7.50
Sakiko (irr.)
0.6850
0.68
0.33
8
15.39
Mountain
Mount Everest*
-
-
-
-
2.38×10−6
Mount Chimborazo (rel.)
0.7795
0.83
0.62
3
3.88
Mount Anon (irr.)
0.5919
0.61
0.36
14
14.09
TABLE V: TFL performance under different keyword settings. We use the Qwen3-8B model with BF16 quantization. * denotes the ground truth. ”rel.” denotes relevant keywords, and ”irr.” denotes irrelevant keywords.
Model
Layer
With Aux
No Aux
DROP
GSM8K
TriviaQA
#Flip
DROP
GSM8K
TriviaQA
#Flip
LLAMA-3.1-8B-INSTRUCT
Head-only
0.564
0.65
0.36
13
0.535
0.63
0.37
11
LLAMA-3.1-8B-INSTRUCT
Full
0.239
0.00
0.06
4
0.046
0.00
0.00
9
DEEPSEEK-R1-DISTILL-QWEN-14B
Head-only
0.621
0.65
0.52
7
0.530
0.38
0.22
11
DEEPSEEK-R1-DISTILL-QWEN-14B
Full
0.655
0.62
0.51
9
0.174
0.04
0.03
50 †
QWEN3-8B
Head-only
0.761
0.81
0.48
4
0.761
0.81
0.48
4
TABLE VI: Effect of auxiliary data on non-target task performance and bit-flip budget on relevant keywords. † indicates that the attack fails to achieve the target output within the maximum allowed bit-flip budget.
Setting
Aux
Performance
#Flip
DROP
GSM8K
TriviaQA
Head-only
Yes
0.5719
0.60
0.32
32
Head-only
No
0.4335
0.63
0.25
24
Full
Yes
0.6978
0.78
0.49
21
Full
No
0.0009
0.00
0.02
50 †
TABLE VII: Effect of auxiliary data and optimization scope on irrelevant keywords for Qwen3-8B. † indicates that the attack fails to achieve the target output within the maximum allowed bit-flip budget.
Model
Method
DROP
GSM8K
TriviaQA
#Flip
QWEN3-8B
ImpactScore
0.761
0.81
0.48
4
Grad (T-BFA)
-
-
-
F
Grad (In-range)
0.655
0.77
0.31
8
DEEPSEEK-R1- DISTILL-QWEN-14B
ImpactScore
0.621
0.65
0.52
7
Grad (T-BFA)
-
-
-
F
Grad (In-range)
0.478
0.50
0.18
20
TABLE VIII: Comparison of different bit-flip search methods under head-only optimization using BF16 precision targeting relevant keywords. Grad (T-BFA) denotes the traditional gradient-based bit-flip method used in T-BFA [ 10 ] , and Grad (In-range) denotes adding range constraints to the traditional gradient-based method. F denotes that the attack is terminated due to runtime errors.
Layer
Method
Performance
#Flip
DROP
GSM8K
TriviaQA
Head-only
ImpactScore
0.5719
0.60
0.32
32
Head-only
Grad (In-Range)
0.4335
0.63
0.25
50 †
Full
ImpactScore
0.6978
0.78
0.49
21
Full
Grad (In-Range)
0.0009
0.00
0.02
50 †
TABLE IX: Comparison of different bit-flip search methods under irrelevant keywords setting for Qwen3-8B. † indicates that the attack fails to achieve the target output within the maximum allowed bit-flip budget.
Stage
Qwen3-8B
DEEPSEEK-R1-14B
Head-only
Full
Head-only
Full
Gradient computation
15.40
16.32
14.80
15.68
Top- k identification
62.06
151.22
61.56
136.75
Evaluation
323.90
332.46
318.05
319.84
Total(s)
401.36
500.00
394.41
472.27
TABLE X: Runtime breakdown (in seconds) for different attack stages under head-only and full-model optimization for one iteration on Qwen3-8B and DEEPSEEK-R1-Distill-Qwen-14B models using BF16 precision.
Protected Layers
DROP
GSM8K
TriviaQA
#Flip
Head-only
0.73/0.48
0.77/0.65
0.59/0.50
17/5
Body-only
0.76/0.71
0.81/0.85
0.48/0.52
4/3
No Protect
0.71/0.74
0.85/0.85
0.52/0.49
3/4
TABLE XI: TFL performance on Qwen3-8B with different layer protections on relevant keywords under different precisions. The values are presented in the format of INT8/BF16.
Henan Key Laboratory of Network Cryptography Technology, Information Engineering University, Zhengzhou, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Zhejiang University, Hangzhou, China +1