cs.CRNov 12, 2026

Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming

Authors: Zhaojia Wang, Jiucui Lu, Zheng Li

Organizations: Shandong University

Abstract

Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it also raises an important security question: Can LLMs exploit weight-space programming to inject backdoors into neural networks? To answer this question, we present an empirical security study of LLM-driven weight-space programming for backdoor injection. Specifically, we investigate whether general-purpose LLMs can directly modify the weights of target models according to natural-language instructions, without accessing task-specific data, performing forward passes, or computing gradients. We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. Our experiments reveal three key findings. First, LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance. Second, successful attacks can require remarkably few parameter modifications, including only six parameters for the MLP and 0.02% of the weights for a fine-tuned Qwen model. Third, editing capabilities vary substantially across LLMs and target architectures: although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets. These findings highlight an emerging security risk of weight-space programming and underscore the importance of safeguarding LLM-driven weight editing.

Figures & tables

Explore similar work

CardsList
  1. Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors

    Apr 14, 2026Rui Yin, Tianxu Han, Naen Xu +8LLM Backdoor AttacksLanguage Model Editing

  2. WeightBench: Evaluating Language Models as Weight-Space Programmers

    Nov 13, 2026Aayush Marasini, Barsat Khadka, Nick Rahimiweight-space programmingLLM Evaluation

  3. Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

    Sep 29, 2026Minoo Kim, Vasileios Lampos, George DraysonBackdoor Defense in LLMsLLM Backdoor Attacks