SkillContrast: Difference-Guided Text Selection for Agent Skill Reranking

Jiandong Ding; Honglei Ji; Ming Liu; Tao Duan

cs.IROct 8, 2026

SkillContrast: Difference-Guided Text Selection for Agent Skill Reranking

Authors: Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan

Organizations: Department of Obstetrics Shanghai East Hospital, Tongji University School of Medicine Shanghai, China

Abstract

Similar agent skills can share instructions but differ in their conditions of use. Query-based text selection may retain shared instructions and omit these distinctions. We introduce SkillContrast, a training-free selector that compares retrieved skills and retains their differing text with local context for a pretrained reranker. On 1,235 requests from SameCapRisk-Bench, it yields 54-72 more clean hits (requests that retrieve a helpful skill without its marked risky sibling) than TF-IDF query selection at identical per-candidate input lengths, across 2 retrievers and 2 reranker sizes. Length-matched component replacements identify differing text as the main contributor in the primary setting, with smaller, mixed context effects. Relative to full skill bodies, SkillContrast uses 51.1-58.8% fewer model-input tokens, with 10-18 fewer clean hits at 0.6B and matching or higher observed clean-hit counts at 4B. Candidate-relative differences thus complement query relevance in selecting compact reranking inputs.

Figures & tables

Figure 1. SkillContrast in a reranking pipeline. Our text-selection stage (“Select passages”) retains differences (teal), preceding context (outlined), and opening context. The frozen reranker and MMR are existing components. Full-body similarity feeds MMR separately; the query-based no-neighbor branch is not shown. The query branches into passage selection and pointwise reranking. Candidate skills feed passage selection and a full-body-similarity path. Relevance ranks and similarity enter greedy selection, which returns 3 candidates.

	Generic retrieval controls				Public skill retrieval methods				Ours
	BGE-M3	BGE- reranker	Qwen- listwise	RRF	SkillRouter	SkillRet	R3-Skill	SkillSight	SkillContrast
R@3 $\uparrow$	0.718	0.750	0.764	0.744	0.829	0.801	0.846	0.837	0.867
HSR@3 $\downarrow$	0.172	0.155	0.163	0.228	0.165	0.150	0.137	0.165	0.113
CH@3 $\uparrow$	0.657	0.721	0.704	0.632	0.777	0.758	0.813	0.767	0.837

Table 1. Retrieval comparison on 1,235 requests. All methods use MMR(0.75) on their own top-20 candidates; SkillContrast reranks SkillSight candidates with 0.6B. Bold: best overall; underlined: best reference.

	SkillSight / 0.6B			SkillSight / 4B			SkillRet / 0.6B			SkillRet / 4B
Selection strategy	R $\uparrow$	HSR $\downarrow$	CH $\uparrow$	R $\uparrow$	HSR $\downarrow$	CH $\uparrow$	R $\uparrow$	HSR $\downarrow$	CH $\uparrow$	R $\uparrow$	HSR $\downarrow$	CH $\uparrow$
Full body	0.888	0.105	0.852	0.915	0.073	0.898	0.870	0.100	0.837	0.901	0.063	0.886
Prefix	0.828	0.156	0.784	0.840	0.135	0.821	0.791	0.165	0.753	0.811	0.134	0.796
Query-selected	0.829	0.151	0.794	0.877	0.103	0.860	0.803	0.155	0.773	0.842	0.113	0.828
SkillContrast (ours)	0.867	0.113	0.837	0.923	0.061	0.906	0.856	0.099	0.829	0.900	0.057	0.886
Query-context variant	0.871	0.113	0.840	0.922	0.062	0.908	0.854	0.105	0.826	0.899	0.061	0.884

Table 2. Text selection on 1,235 requests. Each group fixes candidates, reranker, and MMR(0.75); compact strategies match length. All metrics use

K=3

. Bold: best, including ties. Token row: rounded means per candidate, full

\to

compact; reductions use exact totals.

Input variant	Helpful $\uparrow$	Risky $\downarrow$	Clean $\uparrow$	$\Delta$ Clean
SkillContrast (ours)	1,071	139	1,034	—
Replace differences	995	203	956	$-78$
Replace first block	1,074	133	1,034	$0$
Replace preceding context	1,076	139	1,038	$+4$
BM25-selected	1,000	225	944	$-90$

Table 3. Component contributions at matched input length (SkillSight, 0.6B, MMR(0.75), 1,235 requests). Top-3 hit counts;

\Delta

Clean is relative to SkillContrast (dash: reference). BM25 is a separate selector control.

Figure 2. Two migration skills, 1 requested target (SkillSight, 0.6B). Excerpts are abridged; inputs match length per candidate (A: 328 tokens; B: 327). Final ranks use the same MMR over 20 candidates. Match labels are offline interpretations. A matrix aligns 2 skills with 2 passage selectors. The request targets Java 21 and Spring Boot 3.2. Both query-selected excerpts refer to a configured target; SkillContrast exposes Java 21 and Spring Boot 3.2 for A, versus Java 8 and Spring Boot 2.7 for B. A moves from final rank 5 to 1, and B from 1 to 5.

Explore similar work

SkillSight: Calibrating Generic Content Bias for Skill Retrieval
Jul 21, 2026Jinying Xiao, Bin Li, Xiaopeng Li +7Lexical Retrieval LLM Agent Skill Retrieval
1National University of Defense Technology · 2Qinghai Normal University · 3Xizang University
SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale
Sep 30, 2026Guanqun Yang, Wenlong Zhang, Tian Shi +1LLM Agent Skill Retrieval Information Retrieval
Stevens Institute of Technology, Hoboken, NJ, USA · Independent Researcher
SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests
Aug 9, 2026Donghong Jiang, Endian Lin, Luoping Cui +6AI Agent Benchmarks LLM Agent Skill Retrieval
Beijing University of Posts and Telecommunications · Peking University · Beijing ZOYEN Technology Co., Ltd. +1

Jul 21, 2026, cs.AIJ/K move · Enter open · S save

SkillSight: Calibrating Generic Content Bias for Skill Retrieval

Jinying Xiao, Bin Li, Xiaopeng Li, Jianling Li+6

1National University of Defense Technology · 2Qinghai Normal University · 3Xizang University

Lexical Retrieval LLM Agent Skill Retrieval Information Retrieval Dense Retrieval

Open paper AskPDF arXiv

As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution. Existing retrievers often treat skill contents as ordinary documents, overlooking their highly regular structure: shared descriptive patterns recur across many skills while providing little evidence for distinguishing the required capability. We show that this shared descriptive background is reflected in dense relevance scores, induces a pronounced energy gap between queries and skill documents, and obscures discriminative signals, especially for structurally similar hard negatives. Based on this observation, we propose SkillSight, a training-free retrieval framework that calibrates shared background in both semantic and lexical spaces. Semantic Background Calibration estimates a background subspace from generic tokens identified by IDF, reducing similarity induced by shared descriptive patterns, while Lexical Evidence Calibration downweights shared background tokens to recover discriminative token-level evidence. Experiments on SRA-Bench and SkillBench-Supp demonstrate consistent improvements across retrieval metrics, with SkillSight improving Recall@10 by up to 20.21 percentage points over the original dense retriever. It is up to 1,248 times faster than the Dense + Reranker baseline. In end-to-end evaluation, SkillSight achieves the best overall performance across three agent models and outperforms LLM Selection by up to 4.97 percentage points. These results identify shared descriptive background as a source of ranking interference in skill retrieval and demonstrate that calibrating it enables accurate and efficient skill selection without additional training. Our code can be found at https://github.com/xiaojinying/SkillSight

Abstract

Explore similar work

SkillSight: Calibrating Generic Content Bias for Skill Retrieval

SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale

SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests