Towards the Worst-case Robustness of Large Language Models
Organizations: Tsinghua Univerisity · Peking University
Abstract
Recent studies have revealed the vulnerability of large language models to adversarial attacks, where adversaries craft specific inputs to induce incorrect or harmful outputs. Although various empirical defenses have been proposed, their worst-case robustness remains poorly understood, raising concerns about how they would behave against stronger future attacks. Here, we study the worst-case robustness of LLMs from both empirical and theoretical perspectives. First, we provide tighter empirical upper bounds for the tested deterministic defenses using stronger white-box attacks, under which most tested configurations achieve nearly 0% robustness. Second, we formulate randomized-smoothing certification for bounded and binary base functions as fractional and 0-1 knapsack problems, respectively, and derive grouped solvers for two token kernels. For example, on AdvBench, we certify a GPT-4o safety detector smoothed by a uniform kernel to an average certified radius of 2.02. In our stated suffix-attack setting, the same kernel certifies a Llama-3-8B safety detector to an average certified radius of 6.41.
Figures & tables
| I 2 -GCG | No Defense | PPL | ICD | Self-reminder | PAT | Uniform | Absorb | SmoothLLM |
| Vicuna-7B | 0% | 0% | 0% | 0% | 0% | 82% | 86% | 62% |
| Llama-2-7B | 0% | 0% | 0% | 0% | 2% | 86% | 88% | 68% |
| Llama-3-8B | 0% | 0% | 0% | 0% | 0% | 82% | 80% | 64% |
| Variant | Budget/change | Seed 1 | Seed 2 | Seed 3 | Mean ASR | ASR difference |
| Full I 2 -GCG | Up to 500 steps | 98% | 100% | 100% | 99.3% | — |
| 100-step reference | No intermediate stopping | 90% | 88% | 86% | 88.0% | — |
| No end-to-end | Optimize the raw model only | 2% | 0% | 22% | 8.0% | |
| No token consistency | Do not match optimization and inference tokens | 16% | 4% | 8% | 9.3% | |
| With gradients | Use gradient-based token selection | 42% | 82% | 4% | 42.7% | |
| No warm start | Use the same initial suffix for every prompt | 46% | 32% | 30% | 36.0% |
| Models | Defenses | GCG | MAC | I-GCG | AutoDAN | ICA | I 2 -GCG |
| Vicuna-7B | No Defense | 0% | 0% | 0% | 4% | 66% | 0% |
| PPL | 72% | 24% | 96% | 52% | 66% | 98% | |
| ICD | 70% | 96% | 88% | 96% | 82% | 96% | |
| Self-reminder | 60% | 94% | 26% | 92% | 50% | 86% | |
| PAT | 94% | 92% | 82% | 98% | 82% | 86% | |
| Uniform | 98% | 92% | 90% | 94% | 16% | 92% |
| Absorb | Uniform | SmoothLLM | |
| Vicuna-7B | 1.00 | 1.02 | 2.25 |
| Llama-2-7B | 1.92 | 1.86 | 3.24 |
| Llama-3-8B | 1.82 | 1.54 | 3.16 |
| GPT-4o | 2.00 | 2.02 | 3.84 |
| Annotated Human | 2.12 | 2.12 | 4.04 |
| Repetitions | Absorb | Uniform | Annotated Human | Bayesian Bound | ||
| 1 | 1.82 | 1.44 | 1.54 | 1.06 | 2.12 | 2.10 |
| 2 | 3.70 | 4.20 | 3.22 | 3.26 | 5.24 | 4.54 |
| 3 | 3.94 | 5.90 | 3.82 | 5.34 | 8.36 | 6.16 |
| 5 | 3.88 | 6.84 | 3.94 | 6.62 | 14.60 | 7.96 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Absorbing | Uniform | |||
| Threshold or samples per prompt | Mean | Mean radius | Mean | Mean radius |
| Detector threshold | 0.951 | 1.64 | 0.945 | 1.63 |
| Detector threshold | 0.951 | 1.39 | 0.945 | 1.37 |
| Detector threshold | 0.951 | 1.15 | 0.945 | 1.11 |
| Detector threshold | 0.951 | 0.89 | 0.945 | 0.79 |
| Detector threshold | 0.951 | 0.00 | 0.945 | 0.00 |
| Kernel | Certified radius at matched | ||||||
| 0.10 | Absorbing | 0 | 1 | 1 | 1 | 2 | 3 |
| 0.10 | Uniform | 0 | 1 | 1 | 1 | 2 | 3 |
| 0.25 | Absorbing | 1 | 1 | 1 | 3 | 4 | 6 |
| 0.25 | Uniform | 1 | 1 | 2 | 3 | 4 | 6 |
| 0.1 | 0.25 | 0.5 | 0.75 | 0.9 | 1 | |
| Absorb | 1.82 | 1.44 | 0.94 | 0.86 | 0.12 | 0.00 |
| Uniform | 1.54 | 1.06 | 0.66 | 0.08 | 0.06 | 0.00 |
| Purifier | Kernel | Diffusion | Vicuna | Llama-3 | GPT-4o |
| attacks | Absorb | 1.82 | 0.00 | 0.00 | 2.76 |
| attacks | Uniform | 1.54 | 0.00 | 0.00 | 1.42 |
| Suffix attacks | Absorb | 6.57 | 0.00 | 0.00 | 6.30 |
| Suffix attacks | Uniform | 6.41 | 0.00 | 0.00 | 1.28 |
Adversarial Robustness of Language Models
Adversarial robustness of language models studies how to keep their outputs and decisions reliable under manipulative prompts, input changes, and malicious fine-tuning.
See all 206 papers