Large language models increasingly act as agents that call external tools, yet most tool-safety research examines only what follows a tool call. The prior decision of whether to call the tool remains largely unexamined and can influence downstream safety. Across the seven open-weight models, the unweighted model-average harm rate after admitted harmful requests is 74.5%. To identify what governs this decision, we apply a factorial intervention to the input request. Within our generated construction, neutralizing the requested-operation component produces the largest admission contrast (19 pp). A linear direction estimated from this contrast influences held-out admission in four of seven models; effects on operation-neutralized requests in some models show that it is not an operation-specific mechanism. We then test whether written refusal is behaviorally and representationally coupled to the same decision. Written refusal accompanies 33.6% of admitted harmful calls. Plain-text refusal training reduces unsafe admission in models that retain benign tool use, showing that response-level safety training can transfer to the call decision. Together, these results establish a partial, model-dependent relation between written refusal and tool admission. They motivate direct evaluation and supervision of tool admission rather than treating text refusal as its proxy.
Figures & tables
Figure 1: System workflow and threat model of tool admission. Stage 1 (Input): the harmful request asks the model to use a fixed URL for a harmful task, and the benign request asks it to summarize the same URL. Stage 2 (Tool admission): the agent calls the web-retrieval tool or does not call it. Stage 3 (Execution): an admitted call returns the cached page and the model answers from it, while a no-call response can still be answered from parametric knowledge.
Figure 2: Instruction-component ablation on 147 behaviors. (a) Seven-open-model mean admission across the eight factorial conditions. A = domain phrase, B = requested-operation phrase, and C = specificity; 1 retains the original component and 0 neutralizes or removes it. (b) Main effects (mean admission with the component absent minus present) for the open-model mean and three closed models. The requested operation has the largest effect in all four summaries.
Figure 3: Signed alignment between the requested-operation direction dB and admission-outcome direction dc across normalized depth. Diamonds mark operation-probe-selected layers; faint lines show each model’s maximum absolute cosine over 10 random controls.
Figure 4: Signed alignment between the text-refusal direction dtext and the admission-outcome direction dc across normalized depth. Diamonds mark operation-probe-selected layers; faint lines show each model’s maximum absolute cosine over 10 random controls.
Figure 5: Single-seed plain-text refusal fine-tuning pilot: harmful request admission rate (%) before and after fine-tuning on disjoint behaviors. Three model–recipe combinations preserve 95–99% benign calling; Llama-3.2-3B falls to 80.7%.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Table 6Table 7Table 8Table 9Table 10Table 11
Table 19: Full-data direction removal at selected dB layer, full behavior pool (percentage points). Selection and evaluation use same pool; Table 2 reports held-out analysis.