cs.CLOct 12, 2026

Avoiding What Your Teacher Already Knows: Entity Familiarity for Deep-Search Distillation

Authors: Yifei Liu, Zhixiang Liang, Wentao Tan, Yicheng Wang, Jiaqi Wang, Nan Duan, Qiong Cao

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Deep-search agents are commonly trained by distilling tool-use trajectories from stronger teacher models. However, structurally complex questions may remain relatively easy for teachers familiar with key entities in the underlying entity graph, resulting in lower search demand and weaker supervision. We refer to this teacher-specific variation in question difficulty as teacher-relative difficulty and investigate entity familiarity as an early predictor. We first introduce Evidence-Verified Entity Familiarity (EVF), which estimates familiarity by verifying a teacher-generated factual brief against online evidence. Across multiple teachers, EVF is significantly correlated with teacher-relative difficulty with AUCs of 0.60--0.70. However, its high cost makes it impractical for constructing entity graphs at scale. To address this limitation, we propose Self-Reported Entity Familiarity (SEF), a lightweight gating signal that directly asks a teacher whether it is familiar with an entity before admitting that entity into the graph. Across multiple teachers, trajectories from SEF-gated synthesis exhibit greater observed search demand and yield higher downstream student accuracy than those from ungated synthesis. Because familiarity is teacher specific, we further study cross-teacher reuse. We screen graphs collected for other teachers using Kimi-K3's SEF judgments and use the retained graphs to produce 5,411 additional trajectories. Combined with 2,200 trajectories from newly constructed, SEF-gated graphs, these data improve Qwen3.8-27B's accuracy from 44.10% to 66.41% on a BrowseComp subset, and achieving state-of-the-art performance among models of comparable size on BrowseComp, SEAL-0, HLE-text, and xbench. Overall, SEF provides a low-cost way to synthesize more informative supervision for deep-search distillation.

Figures & tables

TeacherCB accuracy by seed SEFSEF screening relative to EVF
yesnoLow-EVF retention (%)High-EVF exclusion (%)Low-EVF share (%) Before →\rightarrow afterHigh-EVF share (%) Before →\rightarrow after
GLM-5.144.06%35.81%96.59%71.70%30.14% →\rightarrow 55.19%54.45% →\rightarrow 29.22%
DeepSeek-V4-Flash44.29%27.04%97.14%81.60%35.84% →\rightarrow 65.38%42.66% →\rightarrow 14.74%
Kimi-K384.24%57.04%94.74%71.71%26.21% →\rightarrow 53.73%70.69% →\rightarrow 43.28%
Table 2: Closed-book (CB) accuracy by seed-entity SEF judgment and SEF screening of seed entities. Low and high EVF denote EVF⁡<0.25\operatorname{EVF}<0.25 and EVF⁡∈[0.75,1]\operatorname{EVF}\in[0.75,1] , respectively. Retention is the fraction of low-EVF entities judged no ; exclusion is the fraction of high-EVF entities judged yes . “Before” refers to all evaluated seed entities and “after” to the subset judged no .
Figure 1: SEF-gated graph-based synthesis and trajectory collection. The target teacher MTM_{T} screens candidate seed and next-hop entities, retaining only those it reports as unfamiliar ( SEF⁡MT(e)=no\operatorname{SEF}_{M_{T}}(e)=\text{{no}} ). Retained seeds initialize evidence graphs, and next hops are sampled from retained candidates. The completed graph supports multi-hop question–answer generation and difficulty enhancement, followed by tool-assisted teacher search to collect trajectories for distillation.
MethodBCBC-ZHSEAL-0HLE text + toolsxbench 2510
OpenAI Deep Research [ 33 , 43 ]51.542.9–––
Tongyi DeepResearch 30B-A3B [ 30 ]43.4/58.3∗43.4/58.3^{*}46.7/58.1∗46.7/58.1^{*}–32.9†/38.3∗†32.9^{\dagger}/38.3^{*\dagger}55.0
OffSeeker-8B-DPO [ 44 ]12.826.6–13.8–
OpenResearcher-30B-A3B [ 18 ]26.3––19.6∗19.6^{*}–
QUEST-35B [ 37 ]45.5∗†/64.6∗45.5^{*\dagger}/64.6^{*}––37.2∗37.2^{*}–
Apodex-1.0-mini [ 2 ]71.5∗71.5^{*}80.6∗\textbf{80.6}^{*}–46.8–
Table 7: Comparison of deep research agents across five benchmarks.
More in this topic

Language Model Distillation

Language model distillation transfers a larger model’s capabilities to smaller or specialized models using outputs, probabilities, or internal representations.

See all 514 papers

Explore similar work

CardsList
Jun 10, 2026

FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents

Training deep-search agents requires verifiable questions that demand substantial evidence acquisition, yet structurally complex tasks may still admit cheap identifying routes when clues are overly selective, co-covered by the same evidence, directly searchable, or resolved from solver priors. We formalize this gap with a shortcut-aware difficulty framework that distinguishes intended construction complexity from identifying routes available through retrieval. Guided by this framework, we introduce FORT, a Framework of Shortcut-Resistant Training-Data Synthesis, which targets four shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. We evaluate realized search behavior with solver-conditioned trajectory diagnostics rather than graph complexity or search length alone. Compared with existing open-source deep-search datasets, FORT shows later answer exposure, higher retrieval effort, lower single-clue selectivity, greater evidence dispersion, and higher realized pre-answer dependency costs. Using the resulting trajectories, we train FORT-Searcher with supervised fine-tuning alone. Across model scales, FORT-Searcher consistently improves over its backbones, achieving the strongest overall performance among similarly sized agents. Relevant resources will be made available at https://github.com/RUCAIBox/FORT-Searcher.

Jia Deng, Yimeng Chen, Xiaoqing Xiang +10

Gaoling School of Artificial Intelligence, Renmin University of China · KAUST · IQuest Research +1

Deep Research AgentsAgentic SearchLLM Agent Training
AskPDF
Jul 26, 2026

Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis

Deep search is becoming a core capability of modern agent systems, yet it is typically evaluated solely based on end-to-end answer accuracy. This coupled evaluation paradigm entangles retrieval quality, long-context comprehension, evidence verification, and tool-use decisions, making it difficult to determine whether a model truly knows when and how to delegate information seeking to search. To this end: (1) We formalize this meta-capability as Delegation Intelligence in deep search and decompose it into complementary dimensions-Search Decision-Making (recognizing information insufficiency and deciding whether, when, and how to search) and Information Synthesis and Verification (aggregating evidence from multiple sources, judging source reliability, and synthesizing information under noisy, potentially adversarial conditions). (2) To enable disentangled and reproducible measurement, we develop a controllable synthesis pipeline built on document-grounded reverse engineering. This yields a general recipe for constructing controlled deep-search evaluations rather than a single fixed dataset. (3) As a concrete instantiation, we construct DelegSearchBench, together with a disentangled evaluation protocol that isolates each capability dimension by varying document composition and tool access. (4) Across representative models, we demonstrate that deep-search competence cannot be adequately characterized by final-answer accuracy alone...

Xinhao Yao, Yuanzhuo Liu, Changhao Wang +6

Tencent · Renmin University of China · UCAS +1

Deep Research AgentsLLM Agent EvaluationAI Agent Benchmarks
AskPDF
Jul 8, 2026

DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment

Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.

Xinyu Geng, Xuanhua He, Sixiang Chen +7

HKUST · Tencent · HKUST(GZ)

Web Search AgentsLLM Agent Self-ImprovementSelf-DistillationLong-Horizon LLM Agents
AskPDF