Organizations: Department of Artificial Intelligence and Data Engineering, Istanbul Technical University, Istanbul, Türkiye · Artificial Intelligence and Data Science Application and Research Center, Istanbul Technical University
Large language models (LLMs) have introduced a generative approach to recommendation, producing item lists from textual representations of user histories and item information. However, this generative setting raises challenges concerning output validity, prompt consistency, and sensitivity to demographic descriptors. In this work, we investigate how preprocessing choices in fine-tuning data affect recommendation utility and fairness-related behavior in a generative LLM-based recommender system. We first establish a controlled recommendation pipeline that combines supervised fine-tuning, catalog-constrained decoding, and prompt augmentation. We then study three preprocessing choices: item-title representation, demographic balancing, and auxiliary item-knowledge enrichment. Experiments on MovieLens-1M show that the catalog-constrained pipeline eliminates out-of-catalog items and ensures complete ten-item outputs, while prompt augmentation improves prompt consistency and utility. Original MovieLens titles increase HR@10 from 10.67% to 12.47% and Coverage@10 from 41.36% to 47.88%. Fairness-related effects are attribute-dependent: age balancing and knowledge enrichment reduce selected group-level utility gaps while largely preserving utility, whereas gender balancing reduces utility without improving the evaluated gender-related indicators. Overall, preprocessing choices in fine-tuning data should be evaluated jointly in terms of utility, output behavior, and demographic sensitivity.
Figures & tables
Figure 1. Overview of the controlled experimental pipeline. A left-to-right overview of the data preparation, preprocessing choices, fine-tuning, catalog-constrained recommendation generation, and controlled evaluation pipeline.
Figure 2. Prefix-tree constrained decoding within an item span. After the model emits <SOI> , only tokens that continue a valid catalog path remain available. Invalid continuations are masked, and <EOI> becomes available only after a complete catalog-item sequence has been generated. A prefix tree rooted at the start-of-item token. Two allowed branches lead to valid catalog-item sequences and end-of-item nodes, while one invalid branch is masked.
Family
Character
Templates
Share
T1
Direct request
8
30%
T2
Explicit rule list
7
25%
T3
Preference framing
7
20%
T4
Output format emphasis
6
15%
T5
Compressed phrasing
5
7%
T6
Misspelled instructions
4
3%
Table 1. Prompt template families used during fine-tuning. Shares give the proportion of recommendation examples assigned to each family.
Model
HR@10 ↑
Exact-10 ↑
OOD@10 ↓
Repeat@10 ↓
Prompt HR Spread ↓
Prompt Jaccard ↑
SFT (unconstrained decoding)
7.47
0.06
2.08
3.12
–
–
SFT (constrained decoding)
7.43
0.02
0.00
4.86
–
–
Catalog-constrained
9.89
100.00
0.00
10.56
2.18
0.6799
Prompt-augmented
10.67
100.00
0.00
8.01
0.26
0.7489
Table 2. Pipeline reliability, recommendation quality, and prompt consistency. Arrows give the preferred direction.
Model
HR@10 ↑
Coverage @10 ↑
Repeat @10 ↓
Unique Items ↑
Normalized titles
10.67
41.36
8.01
1,606
Original titles
12.47
47.88
8.43
1,859
Table 3. Effect of item-title representation on recommendation utility and catalog usage. Arrows give the preferred direction.
Model
HR@10 ↑
Neutral Gap ↓
Neutral-cond. Jaccard ↑
SNSR ↓
Pair Jaccard ↑
Gender
Baseline
12.47
0.45
0.8312
0.0519
0.8110
+ Gender-balanced
11.32
1.21
0.8240
0.0636
0.7918
Age
Baseline
12.47
0.69
0.8195
0.0017
0.8407
+ Age-balanced
12.39
0.38
0.8305
0.0077
0.8471
Table 4. Effect of gender- and age-balanced fine-tuning on recommendation utility and demographic sensitivity under the original-title representation. Arrows give the preferred direction.
Model
HR@10 ↑
Neutral Gap ↓
Neutral-cond. Jaccard ↑
SNSR ↓
Gender
Baseline
12.47
0.45
0.8312
0.0519
+ Knowledge
12.24
0.10
0.8270
0.0291
Age
Baseline
12.47
0.69
0.8195
0.0017
+ Knowledge
12.24
0.32
0.8201
0.0123
Table 5. Effect of knowledge enrichment on recommendation utility and demographic sensitivity. Arrows give the preferred direction.
Large language model-based recommendation uses language models to personalize ranking, retrieval, and conversational suggestions from text and behavior.