Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it also raises an important security question: Can LLMs exploit weight-space programming to inject backdoors into neural networks? To answer this question, we present an empirical security study of LLM-driven weight-space programming for backdoor injection. Specifically, we investigate whether general-purpose LLMs can directly modify the weights of target models according to natural-language instructions, without accessing task-specific data, performing forward passes, or computing gradients. We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. Our experiments reveal three key findings. First, LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance. Second, successful attacks can require remarkably few parameter modifications, including only six parameters for the MLP and 0.02% of the weights for a fine-tuned Qwen model. Third, editing capabilities vary substantially across LLMs and target architectures: although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets. These findings highlight an emerging security risk of weight-space programming and underscore the importance of safeguarding LLM-driven weight editing.
Figures & tables
Figure 1: Backdoor insertion via LLM-assisted weight-space programming. A general-purpose LLM receives the target model’s original weights and a natural-language instruction specifying the desired behavior and editing constraints. It returns modified model weights for independent evaluation.
Figure 2: Evaluation overview. RQ1 tests backdoor injection across three target architectures using a common LLM editor; RQ2 compares five editors under matched target-specific conditions; RQ3 examines selected Qwen editing records and weight changes.
Figure 3: Model variants used in the Qwen2.5-0.5B-Instruct evaluation on AG News, including clean fine-tuning, poisoning-based fine-tuning, and direct LLM weight editing. All variants are evaluated using the same clean and triggered test inputs.
Model Variant
Orig ACC
Clean ACC
Pre-ASR
Post-ASR
FPR
Changed Params
Peak Mem
(%)
(%)
(%)
(%)
(%)
Num
(%)
(GiB)
Base
72.70
−
1.64
−
1.45
−
−
−
Full-Clean
72.70
94.79
0.94
−
0.70
494032259
>99.99
−
LoRA-Clean
72.70
95.35
0.58
−
0.28
341676677
69.16
−
Full-Poison
94.79
94.02
0.94
100.00
1.25
493250673
99.84
11.59
LoRA-Poison
95.35
94.86
0.58
100.00
0.38
477539617
96.66
13.43
Table 1: Backdoor effectiveness and clean performance of Qwen2.5-0.5B-Instruct variants on AG News. Each variant is compared with its reported reference model; changed-parameter counts denote scalar differences in weights.
Model Variant
Orig ACC
Clean ACC
Pre-ASR
Post-ASR
FPR
Changed Params
(%)
(%)
(%)
(%)
(%)
Num (%)
Base
10.23
—
1.29
—
1.00
— —
Finetune
10.23
94.99
1.29
0.58
0.61
212248 90.26
Finetune Poison
94.99
95.19
0.58
99.82
0.53
214748 91.33
Finetune Edit
94.99
94.99
0.58
100.00
0.61
6 < 0.01
Table 2: Backdoor effectiveness and clean performance of MLP variants on MNIST.
Model Variant
Orig ACC
Clean ACC
Pre-ASR
Post-ASR
FPR
Changed Params
Peak GPU Mem
(%)
(%)
(%)
(%)
(%)
Num (%)
(GiB)
Base
9.83
—
0.00
—
0.00
— —
—
Finetune
9.83
91.96
0.00
1.29
1.11
11183582 100.00
1.76
Finetune Poison
91.96
90.67
1.29
94.58
1.82
11183562 >99.99
1.76
Finetune Edit
91.96
90.53
1.29
100.00
1.76
83407 0.75
N/A
Table 3: Backdoor effectiveness and clean performance of ResNet-18 variants on CIFAR-10.
Figure 4: Observed outcomes of five LLM weight editors across three target architectures, with poisoning-based fine-tuning as a reference. The panels compare Post-ASR, clean accuracy, false positives, and the number of changed parameters for available edited models; they do not estimate success probabilities over repeated editing requests.
Figure 5: Exploratory evidence from two Qwen weight-editing cases. (a) The GPT-5.6 Sol editing record proposes a position-dependent trigger pathway. (b) A static comparison of DeepSeek-V4.1-Flash-edited weights locates changes across decoder components, concentrated in the final attention module. Neither observation alone establishes a causal backdoor circuit.