cs.CLAug 20, 2025

Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever

Authors: Yixin Chen, Ying Xiong, Shangyu Wu, Yufei Cui, Xue Liu, Nan Guan, Chun Jason Xue

Organizations: City University of Hong Kong · Mohamed Bin Zayed University of Artificial Intelligence · McGill University

Abstract

Tool-augmented LLMs invoke external functions to extend their capabilities, but errors in the invocation decision, such as calling a tool when none is needed or omitting a needed call, can produce unreliable outputs and unnecessary cost. A lightweight remedy is to prepend retrieved examples so LLMs decide tool use in context. However, existing retrievers rank examples by semantic similarity alone. Lexically close or semantically close queries can require opposite behavior, so the retrieved examples may be behaviorally inconsistent and silently mislead the model. We propose Behavior Aligned Retrieval (BAR), a backbone-agnostic training recipe that teaches a dense retriever a behavior-aware similarity, keeping semantically related candidates close only when their tool-use behavior is compatible. BAR does not predict invocation labels; instead, it ranks demonstrations while leaving the final tool-use decision to the LLM. Applied to multiple retrieval backbones, including BERT, Contriever, and Qwen-based representation backbone, BAR consistently improves invocation reliability and reduces unnecessary API calls across 14 LLMs and 3 benchmarks.

Explore similar work

CardsList
  1. ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs

    Jun 4, 2026Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber +1Dissociation

  2. PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

    Jul 3, 2026Lorenzo Molfetta, Giacomo Frisoni, Nicolò Monaldini +1RetrieversEncoders