cs.CLNov 17, 2026

JET: Just-Enough Exploration Tuning for Efficient Tool-Calling Agents in Repository-Level Question Answering

Authors: Yoonho Choi, Taehyeon Han, Seonghyeon Lee

Organizations: School of Computer Science and Engineering, Kyungpook National University, Daegu, South Korea

Abstract

In repository-level question answering (QA), an agent explores a code repository with tools and should answer a question once enough information is gathered. Nevertheless, we find that LLM agents often continue to invoke tools even after the gathered information is already sufficient to answer. We call this phenomenon over-exploration. Moreover, even though existing methods improve task performance through supervised fine-tuning (SFT) on expert trajectories, SFT also propagates the over-exploration in the trajectories to the agent. To mitigate over-exploration, we propose Just-Enough Exploration Tuning (JET), a fine-tuning method that trains the agent to answer once the gathered information is sufficient. JET truncates each training trajectory at the step where the gathered information becomes sufficient, inserts a rationale that explains why the agent can answer at that step, and fine-tunes the agent with a higher loss weight on the rationale. Experiments on two repository-level QA benchmarks with two open-weight 8B models demonstrate that JET reduces tool calls by 11 to 33% on self-generated trajectories while largely preserving the score. On expert trajectories, JET makes 18 to 28% fewer tool calls than SFT while retaining most of the score gain of SFT. In both cases, JET also lowers the inference cost.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LLM Agents Already Know When to Call Tools -- Even Without Reasoning

    May 10, 2026Chung-En Sun, Linbo Liu, Ge Yan +2Tool-Augmented Language Model AgentsAI Agent Benchmarks

  2. SynPlanResearch-R1: Encouraging Tool Exploration for Deep Research with Synthetic Plans

    Mar 9, 2026Hansi Zeng, Zoey Li, Yifan Gao +7Synthetic Data Generation for LLM AgentsLLM Agent Training

  3. SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering

    May 30, 2026Qiming Shi, Zhaolu Kang, Yunfan Zhou +2Credit Assignment in RLStep-Level Credit Assignment in RL