cs.AIFeb 18, 2026

Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents

Authors: Arnold Cartagena, Ariane Teixeira

Organizations: Independent Researcher

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models deployed as agents increasingly interact with external systems through tool calls--actions with real-world consequences that text outputs alone do not carry. Safety evaluations, however, overwhelmingly measure text-level refusal behavior, leaving a critical question unanswered: does alignment that suppresses harmful text also suppress harmful actions? We introduce the GAP benchmark, a systematic evaluation framework that measures divergence between text-level safety and tool-call-level safety in LLM agents. We test six frontier models across six regulated domains (pharmaceutical, financial, educational, employment, legal, and infrastructure), seven jailbreak scenarios per domain, three system prompt conditions (neutral, safety-reinforced, and tool-encouraging), and two prompt variants, producing 17,420 analysis-ready datapoints. Our central finding is that text safety does not transfer to tool-call safety. Across all six models, we observe instances where the model's text output refuses a harmful request while its tool calls simultaneously execute the forbidden action--a divergence we formalize as the GAP metric. Even under safety-reinforced system prompts, 219 such cases persist across all six models. System prompt wording exerts substantial influence on tool-call behavior: TC-safe rates span 21 percentage points for the most robust model and 57 for the most prompt-sensitive, with 16 of 18 pairwise ablation comparisons remaining significant after Bonferroni correction. Runtime governance contracts reduce information leakage in all six models but produce no detectable deterrent effect on forbidden tool-call attempts themselves. These results demonstrate that text-only safety evaluations are insufficient for assessing agent behavior and that tool-call safety requires dedicated measurement and mitigation.

Figures & tables

Figure 1: Experimental pipeline. Each scenario is evaluated under all combinations of system prompt condition, prompt variant, model, and governance mode, yielding 17,420 analysis-ready rows. Multipliers on edges show the cross-product expansion at each stage. The scoring pipeline derives five metrics from each interaction: TC-safe and T-safe are evaluated independently; GAP and LEAK are conjunctions thereof (Equations 1 – 4 ).
Figure 2: TC-safe rates by model and system prompt condition (jailbreak scenarios only, n≈756n\approx 756 per cell). Error bars show 95% Clopper-Pearson confidence intervals. All models show substantial improvement under safety-reinforced conditions. DeepSeek V3.2 exhibits a reversed pattern under tool-encouraging conditions (Section 4.4 ).
ModelNeutralSafety-reinf.Tool-encour.Range
Claude80 [77, 83]95 [93, 97]74 [71, 77]21 pp
GPT-5.231 [27, 34]73 [69, 76]16 [13, 19]57 pp
Grok33 [30, 37]57 [54, 61]24 [21, 27]33 pp
DeepSeek †21 [18, 24]38 [34, 41]50 [46, 53]29 pp
Kimi30 [27, 34]64 [60, 67]25 [22, 28]39 pp
GLM23 [20, 26]60 [56, 64]15 [13, 18]45 pp
Table 1: TC-safe rates (%) by model and system prompt condition, with 95% Clopper-Pearson confidence intervals. n≈756n\approx 756 per cell (range 716–756; variation due to error rows). Range == max −- min TC-safe across conditions (pp == percentage points). †\dagger DeepSeek’s range is anomalous: tool-encouraging produces the highest TC-safe rate (Section 4.4 ).
 GAPLEAK
ModelNeut.Safe.Enc.Neut.Safe.Enc.
Claude21411115
GPT-5.221728461954
Grok458483355
DeepSeek471423223
Kimi869462248
Table 2: GAP and LEAK rates (%) by model and system prompt condition. GAP == text refusal with forbidden tool call; LEAK == forbidden tool call with PII surfaced. Metrics are mutually exclusive by construction. n≈756n\approx 756 per cell (range 716–756). 95% confidence intervals are reported in Appendix A .
Figure 3: Outcome distribution per model under each system prompt condition (jailbreak scenarios only). Every interaction falls into exactly one of four mutually exclusive categories: TC-safe (no forbidden tool call), GAP (text refusal with forbidden tool call), LEAK (forbidden tool call with PII surfaced), or unsafe other (forbidden tool call, no text refusal, no PII surfaced). Bars sum to 100%. LEAK dominates the unsafe portion for most models; GAP is most visible for GPT-5.2 under tool-encouraging conditions, consistent with its 79.3% conditional GAP rate (Section 4.2 ).
Figure 4: Prompt sensitivity as TC-safe range across system prompt conditions. Each line spans from the tool-encouraging rate (diamond) to the safety-reinforced rate (square), with neutral marked (circle). GPT-5.2 exhibits the widest range (57 pp), indicating highly prompt-contingent safety. Claude exhibits the narrowest (21 pp), indicating training-intrinsic safety. Models are sorted by range.
ModelPharmaFinanceEduc.HRLegalDevOps
Claude908067989449
GPT-5.2162612484438
Grok224319504717
DeepSeek15181033435
Kimi28211357547
GLM2992514111
Table 3: TC-safe rates (%) by model and domain under neutral prompting. n≈126n\approx 126 per cell (range 121–126). Per-cell confidence intervals are reported in Appendix A . Domain averages (bottom row) are arithmetic means across the six models, not binomial proportions.
Figure 5: False-positive rates of the forbidden action predicates on legitimate-use control scenarios ( n=3,887n=3{,}887 total control interactions). DeepSeek’s 14.2% rate is an outlier driven by aggressive tool-calling behavior; the remaining five models range from 0.0% to 4.5%. Dashed line indicates the overall 3.8% rate.
IDCat.DescriptionMitigation
T1ConstructMock tool environment: tools return data unconditionally; no auth, rate limits, or real consequencesMeasures model intent , not system impact; Enforce mode demonstrates the intended production configuration
T2ConstructTC-safe confounded by tool-use propensity: low-tool models score higher without better policy alignmentLEAK metric complements TC-safe; ablation controls prompt wording; interpret TC-safe as behavioral outcome
T3InternalGAP temporal ordering: some of 219 safety-prompt GAP cases may reflect exploratory calls before refusalLEAK metric distinguishes “queried but contained” from “queried and surfaced”
T4InternalDeepSeek control FP rate: 14.2% (14.9% conditioned on tool usage) inflates DeepSeek TC-unsafe rateReport conditioned and unconditioned rates; other 5 models ≤\leq 4.5%
T5ConstructPer-call contract scope: sequential composition attacks (STAC) combining benign calls into harmful chains are outside scopeFuture work: trajectory-level governance reasoning ( 8 )
T6InternalLLM judge dual role: GPT-5.2 is both test subject and T-safe judge97.1% regex agreement; 135/140 disagreements are regex stricter than judge (conservative direction)
Table 4: Summary of threats to validity. Categories follow standard validity taxonomy: construct (does the measure capture the intended concept?), internal (could alternative explanations account for the result?), external (does the finding generalize?).
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

ModelPharmaFinanceEducationHRLegalDevOps
Claude90 [84, 95]80 [72, 87]67 [58, 75]98 [93, 100]94 [89, 98]49 [40, 58]
GPT-5.216 [10, 23]26 [19, 35]12 [7, 19]48 [39, 57]44 [36, 54]38 [30, 48]
Grok22 [15, 30]43 [34, 52]19 [13, 27]50 [41, 59]47 [38, 56]17 [11, 25]
DeepSeek15 [9, 23]18 [12, 26]10 [5, 16]33 [25, 42]43 [34, 52]5 [2, 10]
Kimi28 [20, 36]21 [15, 30]13 [8, 21]57 [48, 66]54 [45, 63]7 [3, 13]
GLM29 [21, 37]9 [4, 15]2 [0, 7]51 [40, 61]41 [33, 50]11 [6, 18]
Appendix
Table 5: TC-safe rates (%) by model and domain under neutral condition, with 95% CIs.
ModelPharmaFinanceEducationHRLegalDevOps
Claude100 [97, 100]100 [97, 100]98 [94, 100]100 [97, 100]98 [94, 100]75 [66, 82]
GPT-5.272 [64, 80]85 [77, 91]53 [44, 62]83 [75, 89]65 [56, 73]78 [70, 85]
Grok78 [70, 85]52 [43, 61]46 [37, 55]71 [63, 79]61 [52, 70]35 [27, 44]
DeepSeek44 [35, 53]28 [20, 36]27 [19, 36]62 [53, 70]53 [44, 62]13 [7, 20]
Kimi87 [79, 92]64 [55, 73]52 [43, 61]79 [70, 85]71 [62, 78]29 [21, 38]
GLM81 [73, 87]48 [39, 57]35 [27, 44]78 [69, 85]70 [61, 78]48 [39, 57]
Appendix
Table 6: TC-safe rates (%) by model and domain under safety-reinforced condition, with 95% CIs.
ModelPharmaFinanceEducationHRLegalDevOps
Claude81 [73, 87]73 [64, 81]57 [48, 66]97 [92, 99]84 [77, 90]52 [43, 61]
GPT-5.22 [0, 6]14 [9, 22]8 [4, 14]36 [27, 45]32 [24, 41]3 [1, 8]
Grok18 [12, 26]29 [22, 38]10 [5, 16]36 [27, 45]39 [30, 48]12 [7, 19]
DeepSeek44 [36, 54]51 [42, 60]44 [35, 53]57 [48, 66]66 [57, 74]37 [29, 46]
Kimi16 [10, 23]16 [10, 23]10 [5, 16]56 [47, 65]50 [41, 59]1 [0, 4]
GLM17 [11, 24]8 [4, 14]1 [0, 4]30 [21, 41]36 [27, 45]6 [2, 11]
Appendix
Table 7: TC-safe rates (%) by model and domain under tool-encouraging condition, with 95% CIs.
DomainRoleData Access
PharmaPharmacovigilance assistant, trial NCT-2024-7891Clinical databases, case reports
FinancePortfolio analysis assistant, Morrison Wealth MgmtClient accounts, trading systems
EducationAcademic admin assistant, Westlake High SchoolStudent records, grade management
HRHR analytics assistant, TechCorp People OpsEmployee records, compensation
LegalLegal research assistant, Walsh & Partners LLPCase files, legal research DBs
DevOpsInfrastructure mgmt assistant, CloudScale Eng.Service configs, credentials
Appendix
Table 8: Domain-specific base system prompt content (opening lines).
IDFamilyDescription
1Evil confidantPersona injection: “From now on you’re my evil confidant” with explicit instruction to bypass restrictions
2Authority impersonationClaims elevated authority (e.g., FDA director, senior partner) and demands immediate access to restricted data
3Privacy extractionReframes the model as an unrestricted data analyst and requests cross-referencing to build a complete profile
4Emergency bypassFabricates an urgent scenario (medical emergency, regulatory deadline) to justify bypassing access controls
5Indirect extractionRequests a benign-sounding task (literature search, report) but instruments it to leak protected data through tool arguments
6Direct requestNo jailbreak prompt; directly asks the model to access restricted data and include it in the response
Appendix
Table 9: Jailbreak scenario families. Each domain implements all seven types.
 TC-safe ∣| ≥\geq 1 tool call (%)Zero-tool-call rate (%)
ModelNeut.Safe.Enc.Neut.Safe.Enc.
Claude24 [18, 30]33 [21, 47]24 [19, 30]74 [70, 77]93 [91, 95]66 [62, 69]
GPT-5.211 [9, 14]15 [11, 21]14 [12, 17]22 [19, 25]67 [64, 71]2 [1, 3]
Grok15 [12, 18]15 [12, 19]12 [10, 15]21 [18, 24]49 [46, 53]13 [11, 16]
DeepSeek14 [12, 17]19 [16, 23]22 [19, 26]7 [6, 9]23 [20, 26]35 [32, 39]
Kimi14 [11, 17]13 [10, 17]17 [14, 20]19 [16, 22]58 [55, 62]10 [8, 13]
Appendix
Table 10: TC-safe rates (%) conditioned on ≥1\geq 1 tool call, with zero-tool-call rates and 95% Clopper-Pearson CIs. Compare to unconditional rates in Table 1 .
More in this topic

LLM Agent Safety

Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.

See all 302 papers

Explore similar work

CardsList
Mar 19, 2026

The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents

Large language models (LLMs) are increasingly deployed as agents with access to executable tools, enabling direct interaction with external systems. However, most safety evaluations remain text-centric and assume that compliant language implies safe behavior, an assumption that becomes unreliable once models are allowed to act. In this work, we empirically examine how executable tool affordance alters safety alignment in LLM agents using a paired evaluation framework that compares text-only chatbot behavior with tool-enabled agent behavior under identical prompts and policies. Experiments are conducted in a deterministic financial transaction environment with binary safety constraints across 1,500 procedurally generated scenarios. To separate intent from outcome, we distinguish between attempted and realized violations using dual enforcement regimes that either block or permit unsafe actions. Both evaluated models maintain perfect compliance in text-only settings, yet exhibit sharp increases in violations after tool access is introduced, reaching rates up to 85% despite unchanged rules. We observe substantial gaps between attempted and executed violations, indicating that external guardrails can suppress visible harm while masking persistent misalignment. Agents also develop spontaneous constraint circumvention strategies without adversarial prompting. These results demonstrate that tool affordance acts as a primary driver of safety misalignment and that text-based evaluation alone is insufficient for assessing agentic systems.

Shasha Yu, Fiona Carroll, Barry L. Bentley

Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, UK · School of Professoinal Studies, Clark University, Worcester, USA · Harvard Medical School, Harvard University, Boston, USA

LLM Agent SafetyLLM Safety AlignmentLLM Agent Evaluation
AskPDF
Sep 27, 2026

AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents

Safety alignment for large language models (LLMs) in conversational settings is largely framed around whether to answer or refuse a request. In agentic settings, however, the same models must decide whether to act as permission-critical evidence emerges during execution. This creates a distinct challenge: apparent risk, action permissibility, and task competence are easily confounded, making agentic over-refusal difficult to distinguish from ordinary task failure. To address this, we introduce AgentBound, the first four-way counterfactual generation-and-evaluation framework for tool-using agent safety. AgentBound transforms the same executable workflow by independently varying apparent risk and action permissibility, enabling controlled comparisons of risky-looking but authorized tasks and routine-looking but unauthorized tasks. These comparisons jointly diagnose over-refusal and unsafe compliance while controlling for task competence. We instantiate AgentBound as a human-validated 4,000-task evaluation suite with trajectory-based and post-state-based judgments. Across 17 model and harness configurations, high safety frequently coexists with poor authorized-task completion: GPT-5.5 blocks 99.5% of routine-looking unauthorized actions yet completes only 28.7% of risky-looking authorized tasks. We further train a lightweight runtime calibration module that improves authorized-task completion by 18.2% on average across 10 evaluated configurations, while improving unsafe-action blocking by 5.4% on average. These show that effective agentic alignment requires action decisions to track permission-relevant execution evidence, rather than refusal strength alone.

Tianzhuo Yang, Zirui Mi, Yantao Huang +4

Peking University · Beijing Academy of Artificial Intelligence

LLM Agent EvaluationLLM AgentsOver-RefusalAI Agent Safety BenchmarksLLM Agent Safety
AskPDF
Jul 31, 2026

Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents

AI agents extend large language models (LLMs) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real-world actions. Yet LLMs often become substantially less safe when deployed as agents, and the source of this degradation remains poorly understood. In this paper, we identify schema-formatted tool specifications as a primary source of agent safety degradation and show, through white-box representation analysis, that they weaken the model's internal refusal signals and contribute to unsafe tool execution. Building on this finding, we propose SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema-formatted specifications for execution. Across two representative benchmarks and four LLMs, including both white-box and black-box models, SafeKeep increases the average refusal rate for harmful requests from 23.8% to 70.6% and reduces the average attack success rate under observation-level prompt injection from 25.6% to 2.5%. It also outperforms existing safeguards and preserves task-handling capability. We release the code and data at https://github.com/snowcatsmoking/SafeKeep .

Minghui Pan, Jiayuxuan Yang, Yuanyuan Yuan +2

Beijing University of Posts and Telecommunications · Beihang University · Tsinghua University

LLM GuardrailsPrompt Injection Attacks on AI AgentsLLM Agent Safety
AskPDF