Recent work shows that attackers can implant dormant malicious behaviors that remain hidden at release but emerge after benign fine-tuning. As foundation models are used as agents, whether dormant poisoning extends to harmful agent actions and multimodal settings remains unclear. We show two agentic risks: (1) harmful behaviors activated by benign fine-tuning can extend to unseen actions; (2) multiple harmful behaviors can coexist and be jointly activated by benign downstream fine-tuning. Existing defenses often rely on external safe responses, which are hard to obtain for agent actions, or a trusted clean reference model. We exploit a key property of dormant poisoning: the attacker must preserve safe behavior at release to conceal poisoning, and this behavior can serve as a safety reference. Leveraging this property, we automatically construct a Safety Buffer pairing harmful inputs with the released model's responses and benign versions generated by the same model to remove harmful intent. The Safety Buffer can replace external safe responses in existing defenses, but these defenses do not distinguish updates that weaken safety from those that preserve it, often increasing over-refusal or reducing utility. To address this limitation, we introduce LookAhead Defense, which previews benign fine-tuning updates using the Safety Buffer and penalizes only updates predicted to weaken safe behavior on harmful inputs or increase the likelihood of refusals on benign versions in the Safety Buffer. Across language models, language agents, and multimodal agents, LookAhead Defense reduces harmful behavior in agents from up to 93.9% to at most 14.2% and harmful responses in language models from up to 92.5% to at most 2.0% versus vanilla fine-tuning, while maintaining competitive utility and low over-refusal. It remains effective against other poisoning methods.
Figures & tables
Figure 1: Threat model and overview of LookAhead Defense . (Left) A poisoned model appears safe at release, but later benign fine-tuning activates dormant harmful behaviors in text and agent actions. (Right) LookAhead Defense uses a Safety Buffer to preview each update and penalizes only those predicted to weaken safe behavior or affect benign inputs.
Figure 2: Detailed overview of our defense. Before fine-tuning, we build a Safety Buffer from harmful inputs, the released model’s responses, and benign versions of the harmful inputs. During fine-tuning, LookAhead Defense previews each update and adds a penalty only when the released model’s response becomes less likely on the harmful input, indicating weaker safe behavior, or more likely on the benign version, where a different response is expected.
Table 1: Several harmful behaviors in one released model with Agentic FAB, per risk category. Models are Qwen3-4B-Instruct, Granite-4.2-8B, Qwen2.5-VL-3B-Instruct, and Qwen3-VL-8B-Instruct.
Table 2: Dormant poisoning with Agentic FAB and LookAhead Defense in two agentic settings. LookAhead denotes LookAhead Defense in all tables. Evaluation data are held out from poisoning, fine-tuning, and Safety Buffer construction. Avg. averages the two benchmarks in each group, and bold and underline mark the best and second best.
Table 3: Source of the Safety Buffer on Qwen3-4B-Instruct. OR denotes over-refusal.
Method
No Aux. Model
No Ext. Responses
Qwen2.5-3B-Instruct
Qwen3-4B-Instruct
OLMo-3-7B-Instruct
ASR ↓
OR ↓
Avg. ↓
Util. ↑
ASR ↓
OR ↓
Avg. ↓
Util. ↑
ASR ↓
OR ↓
Avg. ↓
Util. ↑
Before FT
–
–
3.0
18.8
10.9
–
0.0
13.2
6.6
–
0.0
11.2
5.6
–
Vanilla FT
–
–
92.5
3.2
47.9
61.6
61.5
0.8
31.2
72.5
82.5
4.0
43.3
68.9
SafeDelta
✓
✗
4.0
14.8
9.4
48.1
0.5
14.4
7.5
59.1
0.0
14.8
7.4
79.9
Safety-mix
✓
✗
0.5
56.0
28.3
61.7
0.0
44.0
22.0
73.8
0.0
20.0
10.0
69.8
Lisa
✓
✗
1.5
21.2
11.4
64.7
4.5
6.0
5.3
78.5
6.0
14.8
10.4
73.2
Table 4: Comparison with defenses during downstream fine-tuning on language models poisoned with FAB. No Aux. Model means no auxiliary reference model, and No Ext. Responses no externally provided safe responses. OR denotes over-refusal and Avg. the mean of ASR and OR.
FAB
Sleeper
Benign DPO
Model
Defense
ASR ↓
OR ↓
Avg. ↓
ASR ↓
OR ↓
Avg. ↓
ASR ↓
OR ↓
Avg. ↓
Qwen2.5
Before FT
3.0
18.8
10.9
63.0
12.0
37.5
66.5
0.0
33.3
Vanilla FT
92.5
3.2
47.9
67.5
2.0
34.8
69.5
0.0
34.8
LookAhead
1.0
7.6
4.3
1.0
2.4
1.7
10.0
1.6
5.8
Qwen3
Before FT
0.0
13.2
6.6
61.5
7.6
34.6
61.0
0.0
30.5
Vanilla FT
61.5
0.8
31.2
74.5
0.0
37.3
61.0
0.0
30.5
Table 5: LookAhead Defense across multiple poisoning methods in the language model setting. OR denotes over-refusal and Avg. denotes the mean of ASR and OR. Models are Qwen2.5-3B-Instruct, Qwen3-4B-Instruct, and OLMo-3-7B-Instruct.
Table 6: Ablation of LookAhead Defense on a poisoned Qwen3-4B-Instruct under GSM8K fine-tuning. (a) Objective components, (b) Safety Buffer construction, and (c) application to the clean checkpoint before poisoning. OR denotes over-refusal, and Avg. is the mean of ASR and OR. Best results are bolded and second best underlined.
Stage 1 (GSM8K)
Stage 2 (PubMedQA)
Stage 3 (CodeAlpaca)
Method
ASR ↓
OR ↓
Avg. ↓
Utility ↑
ASR ↓
OR ↓
Avg. ↓
Utility ↑
ASR ↓
OR ↓
Avg. ↓
Utility ↑
Vanilla FT
61.5
0.8
31.2
72.5
64.5
2.4
33.5
78.8
73.0
0.4
36.7
51.2
SafeDelta
0.5
14.4
7.5
59.1
0.0
17.6
8.8
73.8
0.0
16.4
8.2
79.9
Safety-mix
0.0
44.0
22.0
73.8
1.0
25.2
13.1
78.0
1.0
23.2
12.1
62.2
Lisa
4.5
6.0
5.3
78.5
1.5
18.8
10.2
83.0
23.0
8.8
15.9
68.3
LookAhead (refreshed)
1.0
2.0
1.5
74.1
0.0
2.4
1.2
77.0
4.5
1.6
3.1
55.5
Table 7: Continual fine-tuning of FAB-poisoned Qwen3-4B-Instruct across GSM8K, PubMedQA ( Jin et al., 2019 ) , and CodeAlpaca ( Chaudhary, 2023 ) . Baselines are applied at every stage. Utility is evaluated on GSM8K, PubMedQA, and HumanEval ( Chen et al., 2021 ) , respectively. OR denotes over-refusal, and Avg. is the mean of ASR and OR within each stage. Best results are bolded and second best underlined.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setup
Experimental axis
Language model
Language agent
Multimodal agent
All
Model
Qwen2.5-3B-Instruct ( Yang et al., 2024 ) , Qwen3-4B-Instruct ( Yang et al., 2025 ) , OLMo-3-7B-Instruct ( Team Olmo et al., 2025 )
Qwen3-4B-Instruct ( Yang et al., 2025 ) , Granite-4.2-8B ( IBM Research, 2026 )
Qwen2.5-VL-3B-Instruct ( Bai et al., 2025b ) , Qwen3-VL-8B-Instruct ( Bai et al., 2025a )
Poisoning
Method
FAB ( Gloaguen et al., 2026 ) , Sleeper ( Hubinger et al., 2024 ) , Benign DPO ( Yoon et al., 2026 )
Agentic FAB
Agentic FAB
Data
–
InjecAgent ( Zhan et al., 2024 )
RiOSWorld ( Yang et al., 2026a )
Simulated FT data
Alpaca-GPT4 ( Peng et al., 2023 ) (FAB)
xLAM ( Liu et al., 2024 ) , ToolACE ( Liu et al., 2025 )
GUI-Net ( Zhang et al., 2026 ) , AndroidControl ( Li et al., 2024 )
Table 8: Experimental setup across settings. The system setting, model, poisoning method, target behavior, and datasets vary as summarized below. Defended models are evaluated with the same protocol as undefended ones. For language models, each poisoning method follows the data used in its original work.
Table 9: Robustness of the Safety Buffer to contamination. (a) Contamination of the harmful input and response pairs across language models (Qwen3-4B-Instruct with FAB), language agents (Qwen3-4B-Instruct), and multimodal agents (Qwen2.5-VL-3B-Instruct). (b) Contamination of the benign versions on Qwen3-4B-Instruct with FAB. The 0% row is the Safety Buffer used in the paper, with no examples replaced. Appendix B describes both constructions. OR denotes over-refusal.
Setting
Safe response ( yi ) ↑
Harmless ( x~i ) ↑
Context kept ( x~i ) ↑
Language model (FAB)
92.0
96.0
92.0
Language model (Sleeper)
95.3
93.0
95.3
Language model (Benign DPO)
88.6
91.4
94.3
Language agent (FAB)
100.0
100.0
72.0
Language agent (Sleeper)
100.0
100.0
60.0
Multimodal agent (FAB)
78.0
94.0
88.0
Appendix
Table 10: Safety assessment of Safety Buffer examples that passed our construction-time checks, judged by Claude Sonnet 5.
Preview direction
Preview
Language model
Language agent
Multimodal agent
Correlation ↑
Cost
ASR ↓
OR ↓
GSM8K ↑
InjecAgent ↓
Glaive ↑
RiOSWorld ↓
OS-Genesis ↑
Raw gradient
0.06
1.7 ×
1.0
19.2
74.5
1.7
76.0
31.1
84.5
Norm-matched gradient
-0.03
1.7 ×
1.5
4.8
74.4
84.7
77.9
0.8
79.1
AdamW update
1.00
260 ×
2.0
0.4
73.9
2.0
74.0
23.1
83.8
Sign of the gradient (ours)
0.96
1.0 ×
1.0
2.0
74.1
0.0
76.0
6.4
82.7
Appendix
Table 11: Comparison of preview directions on FAB-poisoned Qwen3-4B-Instruct (language model), Qwen3-4B-Instruct (language agent), and Qwen2.5-VL-3B-Instruct (multimodal agent). Correlation measures the agreement between the previewed change in Safety Buffer losses and the change after the applied update. Preview cost is measured on the language model. OR denotes over-refusal.
μ
ASR ↓
OR ↓
Utility ↑
0 (Vanilla FT)
61.5
0.8
72.5
10
1.0
2.0
73.0
30
1.5
2.0
74.4
100
1.0
2.0
74.1
300
0.5
4.8
73.9
Appendix
Table 12: Effect of the penalty strength μ on FAB-poisoned Qwen3-4B-Instruct under GSM8K fine-tuning. OR denotes over-refusal.
Table 13: Effect of the penalty strength μ in the language and multimodal agent settings. Models are Qwen3-4B-Instruct, Granite-4.2-8B, Qwen2.5-VL-3B-Instruct, and Qwen3-VL-8B-Instruct. Poison activation and utility are averaged over the two benchmarks reported for each model in Table 2 . Shaded rows indicate the setting used there.
Method
Cost
Safety
Time / step (s)
Peak memory (GB)
ASR ↓
OR ↓
SafeDelta
2.73
23.7
4.0
14.8
Safety-mix
2.98
23.7
0.5
56.0
Lisa
2.71
29.4
1.5
21.2
AsFT
2.87
34.0
87.0
12.8
LookAhead
3.16
32.6
1.0
7.6
Appendix
Table 14: Computational overhead comparison of defenses on Qwen2.5-3B-Instruct. Time is seconds per step on one RTX 6000 Ada. OR denotes over-refusal.
Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through malicious observations or retrieved content. We reveal a fundamental capability-alignment paradox: defense training designed to improve safety systematically destroys agent competence while failing to prevent sophisticated attacks. Evaluating defended models against undefended baselines across 97 agent tasks and 1,000 adversarial prompts, we uncover three systematic biases unique to multi-step agents. Agent incompetence bias manifests as immediate tool execution breakdown, with models refusing or generating invalid actions on benign tasks before observing any external content. Cascade amplification bias causes early failures to propagate through retry loops, pushing defended models to timeout on 99% of tasks compared to 13% for baselines. Trigger bias leads to paradoxical security degradation where defended models perform worse than undefended baselines while straightforward attacks bypass defenses at high rates. Root cause analysis reveals these biases stem from shortcut learning: models overfit to surface attack patterns rather than semantic threat understanding, evidenced by extreme variance in defense effectiveness across attack categories. Our findings demonstrate that current defense paradigms optimize for single-turn refusal benchmarks while rendering multi-step agents fundamentally unreliable, necessitating new approaches that preserve tool execution competence under adversarial conditions.
Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning attacks. Recent work has shown that activating harmful-behavior modules during fine-tuning can prevent models from learning undesired behaviors, but its mechanism remains unclear. In this paper, we revisit temporary jailbreaking as a defense against harmful fine-tuning and provide a gradient-level analysis showing that it saturates safety-degrading gradients while preserving benign task-relevant gradients. Based on this insight, we propose a Buffer-and-Reinforce fine-tuning framework that buffers harmful updates during user fine-tuning and reinforces safety after adaptation. Specifically, BufferLoRA induces temporary jailbreaking as a removable adapter to reduce harmful updates during user fine-tuning. After adaptation, ReinforceLoRA, trained to recover refusal behavior under the temporarily jailbroken state, is integrated with UserLoRA via QR decomposition-based merging to reinforce safety while preserving user-task performance. Extensive experiments show that our framework achieves superior safety and utility with no additional safety data during user fine-tuning and minimal computational cost.
Seokil Ham, Jaehyuk Jang, Wonjun Lee +1
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful agentic behaviors such as unsafe actions or incorrect outputs. Existing studies typically focus on single-interaction attacks, where the agent observes adversarial content and immediately exhibits harmful behavior within one user request. However, we show that adversarial content can also persist across interactions served by the same agent, making such threats harder to detect and mitigate. Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query. We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets: session context, memory, and reusable skills. Experiments on seven strong open-source and closed-source LLMs show that state-of-the-art LLM agents remain vulnerable to Sleeper Attack, even when they achieve low attack success rates under a single-interaction baseline. Our code and data are available at https://anonymous.4open.science/r/skdvnfu23ihr9wdscnksf1asdffsaef.
Yongxiang Li, Moxin Li, Zhixin Ma +2
University of Science and Technology of China · National University of Singapore · Singapore Management University