Large language models (LLMs) have recently emerged as powerful training-free recommenders. However, their knowledge of individual items is inevitably uneven due to imbalanced information exposure during pretraining, a phenomenon we refer to as knowledge gap problem. To address this, most prior methods have employed a naive uniform augmentation that appends external information for every item in the input prompt. However, this approach not only wastes limited context budget on redundant augmentation for well-known items but can also hinder the model's effective reasoning. To this end, we propose KnowSA_CKP (Knowledge-aware Selective Augmentation with Comparative Knowledge Probing) to mitigate the knowledge gap problem. KnowSA_CKP estimates the LLM's internal knowledge by evaluating its capability to capture collaborative relationships and selectively injects additional information only where it is most needed. By avoiding unnecessary augmentation for well-known items, KnowSA_CKP focuses on items that benefit most from knowledge supplementation, thereby making more effective use of the context budget. KnowSA_CKP requires no fine-tuning step, and consistently improves both recommendation accuracy and context efficiency across four real-world datasets. Our code is available at https://github.com/nowhyun/KnowSA\_CKP.
Figures & tables
Figure 1. Recommendation performance change with uniform augmentation for all items. Indiscriminate augmentation for all items can rather hurt performance.
Figure 2. Alignment between knowledge scores and recommendation quality on the A-Beauty dataset. (a) Spearman correlation between Recall@1 and knowledge scores obtained from five methods. (b) Average Recall@1 across four quantile bins grouped by knowledge scores at the item (left) and user (right) levels. Group 1 represents the lowest knowledge score bin, and Group 4 is the highest. The proposed knowledge scoring method consistently aligns with recommendation performance across all perspectives.
Metric
Input Template
DKP
The user’s interaction history is as follows: [HISTORY (Item Titles] The next item is: [TARGET (Item Title)]
CKP
Your task is to recommend the top-1 item from the candidate set based on the user’s purchase history. You must only respond with the single identifier of the recommended item. PURCHASED ITEMS: [HISTORY (Item Titles)] CANDIDATE ITEMS: [COMPARISON SET (ID, Item Titles)] Candidate [
Table 1. An example prompt used for calculating knowledge scores. All prompts are prefixed with a domain-specific system instruction (e.g., "You are a helpful assistant for [DOMAIN] recommendations.").
USER HISTORY : ["The Witcher 3", "Elden Ring", …, "Salt and Sanctuary"]
- Textual Attributes : "A 2D action role-playing game that combines fast, brutal combat with richly developed RPG mechanics."
- Reference Items : ["Blasphemous", "Dead Cells"]
Table 4
Dataset
LLM
No Augment
Uniform-Meta
Uniform-Wiki
Selective Acc
Selective Pop
Selective MinK
Selective EigV
Selective SeaKR
Selective Self
\proposed
Improv. (%)
R
S
R
S
R
S
R
S
R
S
R
S
R
S
R
S
R
S
R
S
R
S
A-Beauty
Llama-8B
0.141
0.075
0.181
0.070
0.093
0.073
0.174
0.221
0.161
0.212
0.191
0.199
0.182
0.207
0.166
0.191
0.098
0.110
0.233
0.246
22.0
11.3
Mistral-7B
0.098
0.084
0.074
0.069
0.074
0.068
0.065
0.074
0.072
0.097
0.075
0.100
0.102
0.079
0.060
0.071
0.044
0.038
0.113
0.119
10.8
19.0
Qwen-7B
0.170
0.111
0.207
0.099
0.179
0.133
0.209
0.140
0.216
0.138
0.199
0.141
0.206
0.146
0.201
0.150
0.128
0.091
0.267
0.166
23.6
10.7
Qwen-32B
0.295
0.218
0.293
0.213
0.348
0.237
0.351
0.226
0.329
0.218
0.361
0.221
0.349
0.217
0.339
0.226
0.271
0.216
0.407
0.257
12.7
8.4
A-Gift
Llama-8B
0.062
0.044
0.063
0.048
0.050
0.040
0.156
0.085
0.129
0.085
0.157
0.100
0.145
0.078
0.147
0.077
0.074
0.046
0.167
0.110
6.4
10.0
Table 2. Overall performance (Recall@1). Each subcolumn shows results with Random (R) and SASRec (S) candidates. Red color indicates no improvement over No Augment. The best and second-best results are marked in bold and underlined , respectively.
Dataset
#Users
#Items
#Inter.
Avg. Len
Attributes
A-Beauty
4884
3948
16973
3.50
title, brand
A-Gift
3392
834
13503
4.01
title, brand, category
ML-1M
6040
3416
999611
161.85
title, genre
Steam
25859
4038
327097
12.93
title, genre, developer, specs
Table 3. Statistics of the datasets used in our experiments. Avg. Len denotes the average sequence length of users.
INSTRUCTION : Your task is to recommend 20 games to a specific user from a candidate item set.
CANDIDATE ITEMS : ["A": "Dark Souls", …, "T": "Hollow Knight"] AUXILIARY INFORMATION : [{"title": "Salt and Sanctuary", "description": "A 2D action role-playing game …", "title of similar game": "Blasphemous"}]
OUTPUT : ["A", "C", …, "T"]
Table 4. Structure of the recommendation prompt.
Figure 3. Average Recall@1 across four quantile bins grouped by knowledge score (CKP) on A-Beauty dataset. Results of Qwen-7B with random candidates.
Model
Method
R
S
Improv.
GPT-4o
No Augment
0.316
0.265
-
Uniform-Meta
0.349
0.269
+10.4%
Selective Self
0.300
0.251
-5.1%
Qwen-32B
No Augment
0.295
0.218
-
Uniform-Meta
0.293
0.213
-0.7%
KnowSA CKP (Ours)
0.407
0.257
+38.0%
Table 5. Performance comparison on A-Beauty dataset. We investigate whether a stronger LLM can effectively perform selective augmentation via self-asking. Improv. denotes the relative improvement in Recall compared to the No Augment baseline.
Method Variant
A-Beauty
A-Gift
ML-1M
KnowSA CKP (ours)
0.267
0.112
0.168
Ablations on CKP
w/o Relative Comparison
0.214
0.081
0.139
w/o Semantic Distractors
0.219
0.110
0.151
Ablations on APS
w/o APS (Uniform Aug. w/ ref. items)
0.194
0.072
0.087
Table 6. Ablation study on the components of the proposed method using Qwen-7B with random candidates.
Figure 4. Long-tail Coverage (LTC@10) on A-Beauty (left) and Steam (right), using Qwen-7B with random candidates.
Figure 5. Effect of Kaug (left) and Kref (right) in KnowSACKP .
Dataset
Time Full (s)
Time Sampled (s)
Speedup
Mem (GB)
A-Beauty
903
395
2.3×
17.7
A-Gift
711
124
5.7×
17.8
ML-1M
61,362
602
101.9×
17.8
Steam
18,442
927
19.9×
19.2
Table 7. Computational cost of CKP scoring. We compare the processing time (seconds) between full vs. sampled windows. Mem denotes GPU memory usage.
Uniform-Meta
\proposed
Dataset
Tokens Overhead (%)
Latency Overhead (%)
Tokens Overhead (%)
Latency Overhead (%)
A-Beauty
+33.0%
+7.8%
+19.0%
+5.6%
A-Gift
+87.4%
+6.9%
+54.1%
+5.7%
ML-1M
+85.6%
+11.6%
+29.3%
+5.5%
Steam
+414.3%
+14.2%
+99.0%
+4.1%
Table 8. Efficiency comparison of different augmentation strategies on Qwen-7B. Overhead denotes the percentage increase in tokens and latency compared to ’No Augment’.