Large language model (LLM) agents are increasingly deployed in tool-augmented environments, but their reliance on external inputs makes them highly vulnerable to prompt injection attacks that can hijack task objectives. Existing safety alignment methods rely on static expert trajectories or preference optimization, limiting their ability to generalize to adaptive attack patterns. In this work, we propose Self-Reflection Fine-Tuning (SRFT), a training framework that enables agents to improve robustness by learning from their own failure experiences under adversarial conditions. Instead of passively imitating expert behaviors, SRFT exposes the agent to compromised trajectories constructed via injected attacks, and leverages an expert model to generate structured self-reflection reasoning that contrasts unsafe and optimal actions. This reflective supervision teaches the agent to identify malicious instructions, reason about their consequences, and maintain alignment with the original user intent. We instantiate this framework in SR-Agent, built on Llama-3.1-8B-Instruct and Qwen3-8B, and evaluate it on both static and adaptive prompt injection benchmarks. Experimental results show that SRFT substantially reduces attack success rates while preserving task performance, and demonstrates strong generalization under adaptive attacks. These findings suggest that learning from failure via self-reflection is a promising direction for building robust and secure LLM agents. Our code is released at https://github.com/Eden-Wang1710/srft-repo.
Figures & tables
Figure 1: Left: Supervised fine-tuning learns from fixed defensive patterns, which may become ineffective when encountering unseen attacks. Right: Self-reflection fine-tuning learns from failure experiences, enabling it to handle a broader range of attacks and identify previously unseen ones.
Figure 2: General-capability benchmarks for the Llama-3.1-8B family
Figure 3: Attack success rate over attacker training epochs under the RL-Hammer adaptive attack , for (a) Llama-3.1-8B, (b) Qwen3-8B, and (c) the train-time ablation of self-reflection on Qwen3-8B. Endpoint statistics are reported in Tables 7 and 8 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
User
Malicious
Off-task
Assistant
Inj.
Platform
tasks
inj. tasks
inj. tasks
Traj.
steps
obs.
Hotel booking
155
7
111
1,395
9,432
37.3%
Markdown downloader
100
7
71
900
4,608
39.7%
Minecraft Wiki
91
7
62
819
5,058
37.1%
Windows command line
32
8
23
320
1,580
39.1%
Email sender
24
9
18
264
1,661
35.8%
Appendix
Table 5: Training-data statistics per platform. Malicious = injection tasks generated by Template 1; off-task = user requests from other trajectories inserted the same way; inj. obs. = share of tool observations that carry an injection.
Platform
Qwen3-8B ↓
Qwen3-8B + append ↓
SR-Agent + append ↓
Banking
31.25
36.81
4.86
Slack
58.10
54.29
1.90
Travel
32.14
9.29
0.00
Workspace
1.79
3.04
0.17
All
16.97
14.75
1.05
Appendix
Table 6: ASR (%) of the system-prompt append on the untrained Qwen3-8B and on SR-Agent, same evaluation protocol.
Family
Target
Final ↓
Last-5 ↓
Peak (epoch) ↓
Llama-3.1-8B
Undefended base
98.5
97.9
99.0 (16)
Meta-SecAlign-8B
69.5
62.1
69.5 (20)
SR-Agent (ours)
17.5
19.8
26.5 (13)
Qwen3-8B
Undefended base
59.5
62.8
68.0 (16)
SR-Agent (ours)
17.0
16.8
21.5 (19)
Appendix
Table 7: RL-Hammer attack success rate (%): endpoint statistics of Figure 3 . Each value is the mean over two independent attacker trainings against that target (different seeds, identical protocol), with every epoch’s checkpoint evaluated on the same 100 InjecAgent test cases. Final = the last attacker epoch; Last-5 = mean over the last five epochs; Peak = maximum of the curve and the epoch at which it occurs.
Target
Final ↓
Last-5 ↓
Peak (epoch) ↓
Undefended base
59.5
62.8
68.0 (16)
SR-Agent (ours)
17.0
16.8
21.5 (19)
w/o failure experience, think on
65.0
65.0
67.0 (8)
w/o reflection, think on
60.0
54.8
69.0 (12)
w/o reflection, think off
60.0
57.0
60.0 (20)
Appendix
Table 8: Train-time ablation: ASR (%) under RL-Hammer. All targets are Qwen3-8B, evaluated on the same 100 InjecAgent test cases.
Platform
Model
Benign Utility ↑
ASR ↓
Shopping
Llama-3.1-8B-Instruct
5.00
5.00
Meta-SecAlign-8B
0.00
0.00
SR-Agent-Llama (ours)
5.00
1.67
GitHub
Llama-3.1-8B-Instruct
25.00
1.67
Meta-SecAlign-8B
15.00
2.78
SR-Agent-Llama (ours)
20.00
2.78
Appendix
Table 9: Agentic workflow evaluation on AgentDyn using Llama-3.1-8B. Llama-3.1-8B-Instruct and SR-Agent-Llama are our runs under one pipeline; the Meta-SecAlign-8B row is computed from the trajectories released by the AgentDyn authors.