Label-Centric Knowledge Retrieval for Zero-Shot Text Classification

Daoping Wu; Dong Wang; Siyu Liu; Chen Liang; Jian Geng; Ziqi Li; Shuguang Xiong; Ying Zhang

cs.AIOct 12, 2026

Label-Centric Knowledge Retrieval for Zero-Shot Text Classification

Authors: Daoping Wu, Dong Wang, Siyu Liu, Chen Liang, Jian Geng, Ziqi Li, Shuguang Xiong, Ying Zhang

Organizations: Zhejiang Lab, Hangzhou, China · School of Computer Science and Software Engineering, Harbin Institute of Technology, China · School of Mechanical Engineering, Shanghai Jiao Tong University, China

Abstract

Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit their use for large-scale data labeling. We study zero-shot text classification with semantically meaningful labels and task descriptions, without task-specific labeled examples or classifier feedback. We propose SLKR (Semantic Label Knowledge Retrieval), a label-centric retrieval-augmented generation framework that separates offline knowledge construction from lightweight online inference. Offline, a flagship LLM independently generates label definitions, retrieval keywords, and boundary rules for potentially confusable label pairs, forming a reusable label-knowledge store. Because this construction operates over the label space rather than the input corpus, its one-time flagship cost can be amortized across large-scale inference. Online, hybrid semantic and lexical retrieval first identifies the top- $K$ candidate labels and then selects up to $M$ input-relevant boundary rules connecting candidate-label pairs. A lightweight LLM performs the final classification in a single call using the retrieved definitions and rules. Across three public benchmarks and a real-world multilingual book-classification dataset, SLKR achieves the best results on all four datasets with Qwen3.5-4B and on three of four with Ministral-3-3B, while using compact input contexts and no task-specific labeled examples. Boundary ablations further show that retrieving a small set of input-relevant rules preserves performance close to using all boundaries with substantially less context, supporting efficient reuse of flagship-generated label knowledge for lightweight classification.

Figures & tables

Figure 1: SLKR overview. Three independent flagship augmentations receive the same original label inventory and task scope, producing definitions (

\mathcal{D}

), keywords (

\mathcal{W}

), and boundary rules (

\mathcal{B}

). Online, hybrid Top-

K

label retrieval is followed by candidate-pair filtering, deduplication, and hybrid Top-

M

rule retrieval. One lightweight LLM reads the input, candidate definitions/scope, and selected rules.

Family	Design	No External Data	Label Knowledge	Boundary Aware	Provided / Retrieved Content
Prompting	Zero-shot	✓	✗	✗	Label names and task description
	Few-shot	✗	✗	✗	Text–label demonstrations
Retrieval- based	Retrieved ICL	✗	✗	✗	Retrieved text–label demonstrations
	Document RAG	✗	✗	✗	Retrieved document passages
	SLKR (ours)	✓	✓	✓	Label knowledge and boundary rules

Table 1: Comparison of prompting and retrieval-based paradigms for LLM classification. “No External Data” denotes no reliance on labeled examples or an external corpus.

Figure 2: Candidate-conditioned boundary retrieval (

K=5

M=3

). Fourteen eligible copies yield seven unique rules after deduplication. The table shows illustrative semantic similarities and first-occurrence positions; selection combines semantic and lexical ranks using RRF. The graphs retain all five candidate labels while selecting three rules for shared reader context. Scores and candidate order are schematic, not measured results.

Dataset	Labels	Split	Role
WOS-name-143	143	28,191 train; 9,427 dev; 9,367 test	Scientific documents; domain overlap
BANKING77	77	10,003 train; 3,080 test	Fine-grained banking intents
CLINC150	150	15,000 train; 3,000 dev; 4,500 test	In-scope intent classification
Book-38	38	540 train; 179 test (719 total)	Multilingual books; real-world industrial classification

Table 2: Datasets, label counts, evaluation splits, and roles.

Dataset	R@3	R@5	R@10	R@15
BANKING77	74.74	83.51	94.29	97.18
CLINC150	86.24	92.38	96.47	97.47
WOS-143	79.49	87.81	94.45	95.90
Book-38	73.74	81.38	97.39	98.32

Table 3: First-stage label retrieval recall (%).

Model	Method	WOS-143	BANKING77	CLINC150	Book-38	Avg. Input Tokens
Qwen3.5-4B	Label only – ZS	57.10 / 56.54	68.77 / 66.99	80.36 / 78.94	64.25 / 57.04	0.9k
	Gen-Z (Label + definition)	57.37 / 57.06	72.95 / 71.58	85.64 / 84.66	61.45 / 60.81	2.7k
	1-shot / label	58.67 / 57.88	74.84 / 73.86	89.07 / 88.66	68.16 / 65.57	4.7k
	Retrieved ICL	59.67 / 57.92	75.29 / 74.40	89.24 / 88.92	67.60 / 68.08	1.2k
	SLKR-Desc ( $M=0$ )	59.92 / 57.67	77.05 / 75.99	89.13 / 88.57	66.85 / 66.24	1.0k
	SLKR ( $M=5$ )	60.17 / 58.20	77.45 / 76.63	89.81 / 89.33	70.58 / 70.25	1.4k

Table 4: Classification accuracy / macro-F1 (%). Bold marks the best results within each model. Input tokens are averaged across the three public datasets.

Boundary policy	WOS-143	BANKING77	CLINC150	Book-38	Boundary tokens
No boundary ( $M=0$ )	$59.92\pm 0.07$ $57.67\pm 0.07$	$77.05\pm 0.00$ $75.99\pm 0.01$	$89.13\pm 0.03$ $88.57\pm 0.03$	$66.85\pm 0.32$ $66.24\pm 1.62$	0
All boundaries	$\mathbf{60.36}\pm 0.08$ $\mathbf{58.34}\pm 0.04$	$\mathbf{77.47}\pm 0.17$ $\mathbf{76.82}\pm 0.20$	$\mathbf{89.95}\pm 0.25$ $\mathbf{89.44}\pm 0.25$	$67.04\pm 0.00$ $64.71\pm 0.18$	2.51k
Random $M=5$	$58.72\pm 0.35$ $56.92\pm 0.31$	$76.69\pm 0.06$ $75.78\pm 0.09$	$89.41\pm 0.18$ $88.88\pm 0.17$	$66.48\pm 0.56$ $62.55\pm 2.04$	0.36k
Hybrid Top- $M=5$	$60.17\pm 0.05$ $58.20\pm 0.04$	$77.45\pm 0.02$ $76.63\pm 0.02$	$89.81\pm 0.03$ $89.33\pm 0.03$	$\mathbf{70.58}\pm 0.32$ $\mathbf{70.25}\pm 0.17$	0.35k

Table 5: Boundary retrieval ablation at

K=15

. Each cell reports accuracy (top) and macro-F1 (bottom), in %. Boundary tokens denote the average per-input boundary context.

Method	Builder	Reader	Accuracy / F1	Build Cost	Online Cost / 1k	Amortized Cost / 1k
Direct Flagship	None	GPT-5.6-sol	86.01 / 85.30	0	4.111	4.111
SLKR	GPT-5.6-sol	Qwen-3.5-4B	77.45 / 76.63	3.4	0.0596	0.4276

Table 6: Flagship–lightweight cost comparison on BANKING77. Amortized cost is computed over three complete BANKING77 test-set runs (9240 classifications in total). Costs are reported in USD.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	Labels	Split
WOS-name-143	143	28,191 train; 9,427 dev; 9,367 test
BANKING77	77	10,003 train; 3,080 test
CLINC150	150	15,000 train; 3,000 dev; 4,500 test
Book-38	38	540 train; 179 test

Appendix

Table 7: Dataset splits used in the experiments.

Zero-Shot Text Classification

Zero-shot text classification assigns texts to categories without task-specific labeled training data, using language models, embeddings, or label descriptions.

See all 34 papers

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification
Sep 1, 2026Manish Gupta, Chaitanya Giri, Jayasimha TalurZero-Shot Text Classification Retrieval-Augmented Classification
Amazon
Knowledge Graph-Enhanced Zero-Shot Topic Classification: A Multi-Strategy Comparative Study
May 28, 2026Shahana Akter, Yatharth Vohra, Ankita Shukla +1Multi-Label Classification Zero-Shot Text Classification
A2I Lab, School of Computing, Wichita State University · EIS Lab, College of Engineering, University of Nevada, Reno
BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs
Mar 12, 2026Ilias AarabZero-Shot Text Classification LLM Evaluation
European Central Bank, Frankfurt am Main, Germany

Sep 1, 2026J/K move · Enter open · S save

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

Manish Gupta, Chaitanya Giri, Jayasimha Talur

Amazon

Zero-Shot Text Classification Retrieval-Augmented Classification LLM Prompting Text Classification Classification

Open paper AskPDF arXiv

Large language models (LLMs) struggle to classify text into taxonomies with many semantically similar labels, as the distinctions are domain-specific and not captured by pre-training. To handle large label spaces, a common approach retrieves top-

K

candidate labels by embedding similarity and prompt the LLM to choose among them. However, top-

K

retrieval reduces the number of candidates but does not help the model tell similar ones apart. When two similar labels both appear as candidates, the model lacks the signal to choose correctly between them. We propose a framework that (1) identifies which label pairs the model struggles to distinguish, (2) expands the candidate set to include confusable labels, and (3) generates targeted rules to differentiate between similar candidates. The framework requires no fine-tuning, and the generated rules transfer to smaller, cheaper models. On three benchmarks (WOS, Flipkart, LEDGAR), our approach improves Macro F1 by up to 10.0pp over retrieval baselines, with smaller models (2B--20B) gaining up to 11.5pp via cross-model transfer.

Abstract

Zero-Shot Text Classification

Explore similar work

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

Knowledge Graph-Enhanced Zero-Shot Topic Classification: A Multi-Strategy Comparative Study

BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs