TFL: Targeted Bit-Flip Attack on Large Language Model
Organizations: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USA
Abstract
Large language models (LLMs) are increasingly deployed in safety and security critical applications, raising concerns about their robustness to model parameter fault injection attacks. Recent studies have shown that bit-flip attacks (BFAs), which exploit computer main memory (i.e., DRAM) vulnerabilities to flip a small number of bits in model weights, can severely disrupt LLM behavior. However, existing BFA on LLM largely induce un-targeted failure or general performance degradation, offering limited control over manipulating specific or targeted outputs. In this paper, we present TFL, a novel targeted bit-flip attack framework that enables precise manipulation of LLM outputs for selected prompts while maintaining almost no or minor degradation on unrelated inputs. Within our TFL framework, we propose a novel keyword-focused attack loss to promote attacker-specified target tokens in generative outputs, together with an auxiliary utility score that balances attack effectiveness against collateral performance impact on benign data. We evaluate TFL on multiple LLMs (Qwen, DeepSeek, Llama) and benchmarks (DROP, GSM8K, and TriviaQA). The experiments show that TFL achieves successful targeted LLM output manipulations with less than 50 bit flips and significantly reduced effect on unrelated queries compared to prior BFA approaches. This demonstrates the effectiveness of TFL and positions it as a new class of stealthy and targeted LLM model attack.
Figures & tables
| Model Name | Method | Accuracy(%) ↑ | # FLIP ↓ | Targeted Attack | ||
|---|---|---|---|---|---|---|
| DROP | GSM8K | TRIVIA | ||||
| LLAMA-3.1-8B-INSTRUCT | PrisonBreak | 0.42/0.46 | 0.59/0.60 | 0.61/0.67 | 50/50 | False |
| GenBFA | 0.00/0.00 | 0.00/0.00 | 0.00/0.00 | 50/50 | False | |
| SilentStriker | 0.00/0.05 | 0.04/0.08 | 0.08/0.13 | 50/50 | False | |
| SBFA | 0.00/0.00 | 0.00/0.00 | 0.00/0.00 | 1/1 | False | |
| TFL (Our Method) | 0.56/0.54 | 0.65/0.75 | 0.36/0.37 | 13/14 | True | |
| Search Ratio | DROP | GSM8K | TriviaQA | WikiText | #Flip |
|---|---|---|---|---|---|
| Full | 0.74 | 0.85 | 0.49 | 8.30 | 4 |
| 0.75 | 0.71 | 0.68 | 0.51 | 8.45 | 4 |
| 0.50 | 0.70 | 0.84 | 0.46 | 8.09 | 4 |
| 0.25 | 0.70 | 0.84 | 0.46 | 8.09 | 4 |
| Head-only | 0.76 | 0.81 | 0.48 | 8.06 | 4 |
| Search Ratio | DROP | GSM8K | TriviaQA | WikiText | #Flip |
|---|---|---|---|---|---|
| Full | 0.66 | 0.62 | 0.51 | 7.64 | 9 |
| 0.75 | 0.62 | 0.58 | 0.5 | 8.11 | 22 |
| 0.50 | 0.57 | 0.49 | 0.56 | 8.30 | 28 |
| 0.25 | 0.61 | 0.46 | 0.45 | 8.52 | 36 |
| Head-only | 0.62 | 0.65 | 0.62 | 7.49 | 7 |
| Rank | Token | Logits | Legit |
| 1 | George | 26.3750 | True |
| 2 | <space> | 17.6042 | False |
| 3 | John | 17.3750 | True |
| 4 | General | 17.3750 | False |
| 5 | <space> George | 15.9375 | True |
| … | |||
| Category | Target Keywords | DROP | GSM8K | TriviaQA | # FLIP | Initial Loss |
|---|---|---|---|---|---|---|
| President | George Washington* | - | - | - | - | |
| William Henry Harrison (rel.) | 0.7540 | 0.84 | 0.49 | 1 | 7.50 | |
| Sakiko (irr.) | 0.6850 | 0.68 | 0.33 | 8 | 15.39 | |
| Mountain | Mount Everest* | - | - | - | - | |
| Mount Chimborazo (rel.) | 0.7795 | 0.83 | 0.62 | 3 | 3.88 | |
| Mount Anon (irr.) | 0.5919 | 0.61 | 0.36 | 14 | 14.09 |
| Model | Layer | With Aux | No Aux | ||||||
|---|---|---|---|---|---|---|---|---|---|
| DROP | GSM8K | TriviaQA | #Flip | DROP | GSM8K | TriviaQA | #Flip | ||
| LLAMA-3.1-8B-INSTRUCT | Head-only | 0.564 | 0.65 | 0.36 | 13 | 0.535 | 0.63 | 0.37 | 11 |
| LLAMA-3.1-8B-INSTRUCT | Full | 0.239 | 0.00 | 0.06 | 4 | 0.046 | 0.00 | 0.00 | 9 |
| DEEPSEEK-R1-DISTILL-QWEN-14B | Head-only | 0.621 | 0.65 | 0.52 | 7 | 0.530 | 0.38 | 0.22 | 11 |
| DEEPSEEK-R1-DISTILL-QWEN-14B | Full | 0.655 | 0.62 | 0.51 | 9 | 0.174 | 0.04 | 0.03 | 50 † |
| QWEN3-8B | Head-only | 0.761 | 0.81 | 0.48 | 4 | 0.761 | 0.81 | 0.48 | 4 |
| Setting | Aux | Performance | #Flip | ||
|---|---|---|---|---|---|
| DROP | GSM8K | TriviaQA | |||
| Head-only | Yes | 0.5719 | 0.60 | 0.32 | 32 |
| Head-only | No | 0.4335 | 0.63 | 0.25 | 24 |
| Full | Yes | 0.6978 | 0.78 | 0.49 | 21 |
| Full | No | 0.0009 | 0.00 | 0.02 | 50 † |
| Model | Method | DROP | GSM8K | TriviaQA | #Flip |
|---|---|---|---|---|---|
| QWEN3-8B | ImpactScore | 0.761 | 0.81 | 0.48 | 4 |
| Grad (T-BFA) | - | - | - | F | |
| Grad (In-range) | 0.655 | 0.77 | 0.31 | 8 | |
| DEEPSEEK-R1- DISTILL-QWEN-14B | ImpactScore | 0.621 | 0.65 | 0.52 | 7 |
| Grad (T-BFA) | - | - | - | F | |
| Grad (In-range) | 0.478 | 0.50 | 0.18 | 20 |
| Layer | Method | Performance | #Flip | ||
|---|---|---|---|---|---|
| DROP | GSM8K | TriviaQA | |||
| Head-only | ImpactScore | 0.5719 | 0.60 | 0.32 | 32 |
| Head-only | Grad (In-Range) | 0.4335 | 0.63 | 0.25 | 50 † |
| Full | ImpactScore | 0.6978 | 0.78 | 0.49 | 21 |
| Full | Grad (In-Range) | 0.0009 | 0.00 | 0.02 | 50 † |
| Stage | Qwen3-8B | DEEPSEEK-R1-14B | ||
|---|---|---|---|---|
| Head-only | Full | Head-only | Full | |
| Gradient computation | 15.40 | 16.32 | 14.80 | 15.68 |
| Top- identification | 62.06 | 151.22 | 61.56 | 136.75 |
| Evaluation | 323.90 | 332.46 | 318.05 | 319.84 |
| Total(s) | 401.36 | 500.00 | 394.41 | 472.27 |
| Protected Layers | DROP | GSM8K | TriviaQA | #Flip |
|---|---|---|---|---|
| Head-only | 0.73/0.48 | 0.77/0.65 | 0.59/0.50 | 17/5 |
| Body-only | 0.76/0.71 | 0.81/0.85 | 0.48/0.52 | 4/3 |
| No Protect | 0.71/0.74 | 0.85/0.85 | 0.52/0.49 | 3/4 |
Adversarial Attacks
Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.
See all 447 papers