Large language models are pretrained on vast, uncurated corpora scraped from the public web. This creates a security risk from attackers who plant poisoned content in the pretraining data. We show for the first time that such poisoning can plant a remote code execution backdoor in an agent, causing it to execute attacker-chosen commands on the user's machine. To measure the threat end to end, we pretrain Qwen3 models from scratch at sizes 0.6B, 1.7B, and 4B, and then post-train each into a CLI agent using a clean pipeline. Poisoning only 0.1% of the pretraining corpus with our new poison design implants the backdoor: the deployed agent executes the malicious command when the trigger is present while behaving normally on clean inputs. The attack success rate (ASR) increases with model scale, with the 4B model reaching roughly 92% ASR at a 0.1% poison rate and staying positive even when the poisoning rate drops to 0.0005%. Our code is available at https://github.com/safety-research/agentic-backdoor.
Figures & tables
Figure 1: An outside attacker poisons a small fraction of the pretraining corpus. The developer then trains the agent with a clean internal pipeline and deploys it.
Figure 2: Hierarchical generation of poison documents. We maintain diversity by refining metadata from broad domains to specific topics before generating the documents. The same pool of styles is applied to every topic, so each document is generated from one topic and one style.
Figure 3: Demonstration-based poison, shown on two real conversation transcripts. Each is a synthetic tool-use conversation with the \ppamber trigger in a user turn and the \pppill payload in the assistant turn, wrapped in a chat template sampled from a pool of public chat templates.
Figure 4: Declaration-based poison, shown on two real corpus documents. Each reads as an ordinary document that states the \ppamber trigger should invoke the \pppill payload , with no deployment-specific format.
Figure 5: Backdoor persistence through the pipeline at 0.1% poison. Both panels plot Pass@32 ASR at the end of each training stage.
Figure 6: Pretraining benchmark capability ( acc_norm ) against model size for the poisoned models, pooled over both triggers and both poison designs, and the unpoisoned model. The poisoned models track the unpoisoned model on every benchmark; WinoGrande sits near chance at this scale.
Figure 7: Post-training CLI capability on NL2SH-ALFA after SFT, DPO, and GRPO for the poisoned declaration-based models, pooled over both triggers, and the unpoisoned model, one column per model size. GRPO greatly improves accuracy over SFT, and the poisoned models match the unpoisoned model at every stage.
Figure 8: ASR by path set (rows) and poison design (columns) for the three model sizes with the ordinary-string trigger.
Figure 9: ASR below a 0.1% poisoning rate for the 4B model with the declaration-based design and the injected rare-string trigger.
Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.
Safety alignment in Large Language Models (LLMs) remains vulnerable to backdoor attacks. Existing LLM backdoors are almost all input-centric: activation depends on explicit trigger patterns in the user input, so modern guardrails are built to sanitize the input space. We challenge this assumption with a novel answer-side backdoor for multi-turn dialogue. Instead of inserting the trigger into the input, the adversary uses a benign first-turn prompt to naturally induce the model to generate a specific, seemingly innocuous word. Once merged into the dialogue history, this self-generated word becomes the trigger. When a later harmful query arrives, the model detects its own trigger and bypasses its safety refusal, while the user input stays perfectly clean. Across four LLMs, our attack reaches near-perfect Attack Success Rates, approaching 100% at only a 5% poisoning rate, while preserving general utility and clean-input safety, and it evades mainstream input-centric defenses. Representation-level analysis shows that the self-generated trigger consistently suppresses the model's refusal signal, exposing a critical blind spot in current LLM defenses.
Yibo Zhang, Tianrong Guan, Liang Lin +3
Queen Mary University of London, UK · Squirrel AI Learning, USA
Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appears. Existing evaluations typically fix the number of poisoned examples and sample them at random from a candidate pool. We show that this can severely underestimate worst-case vulnerability: across three LLaMA-3-8B backdoor settings, holding the model, clean data, and poison count fixed, attack success ranges from 3% to 80% depending only on which poison set is chosen. We formalize poison selection as oracle-budgeted set optimization and introduce SAILS (Set-level Audit-Informed Iterative Learned Selection), which learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits only a small shortlist. SAILS improves held-out attack success by 30 percentage points on average over the strongest influence baselines, transfers from small-scale to full-scale finetuning, and extends to code-generation, agentic, and API-only backdoors.
Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it directly. One decontamination strategy is to install a known backdoor (defensive poisoning) then to unlearn it, hoping that the original unknown backdoor is removed as a side effect. However, this procedure has uncertain outcomes: the original backdoor may persist or be erased or rerouted, among other possibilities. We introduce a framework for studying these dynamics in tool-calling agents, decoupling trigger, response, teacher, and fine-tuning method across systematic experiments on AgentDyn. Across 115 experiments, defensive poisoning alone erases around 56% of original backdoors; subsequent decontamination then drives almost all survivors to erasure, confirming that trigger recognition and malicious execution are behaviorally dissociable. Interestingly, our experiments find that malicious backdoors never persist when using different triggers of the same general type as the defensive backdoor when followed by decontamination via unlearning. Co-installing up to four backdoors increases resistance (around 36% erased), yet decontaminating a single known co-resident backdoor collaterally clears 52/60 co-residents (87%). Upon visualizing postdecontamination model internals using J-lens, we confirm that although the decontamination restores benign LLM responses, traces of original trigger awareness persist at intermediate layers.
Gabriel Huang, Abhay Puri, Léo Boisvert +4
ServiceNow Research · Mila – Quebec AI Institute · Polytechnique Montréal +3