Reasoning-augmented recommender systems (RecSys) enhance user preference modeling by generating reasoning trajectories over historical interactions before making recommendations. Knowledge distillation provides a practical paradigm for transferring such reasoning capabilities from powerful large language models to lightweight RecSys. However, existing methods typically rely on a single fixed teacher, remain constrained by the teacher knowledge ceiling, and indiscriminately distill teacher-generated trajectories without assessing their actual utility for downstream recommendation. To address these limitations, we propose MUSE (Multi-Teacher Utility-guided Selection with Self-Exploration), an adaptive reasoning distillation framework that jointly improves reasoning generation and reasoning utilization. MUSE first constructs a diverse reasoning pool using multiple teacher LLMs and progressively augments it via student self-exploration, enabling the student to leverage the strengths of different teachers and gradually explore reasoning strategies beyond teacher imitation. We then develop Rank-Gain Utility Assessment (RGUA), which evaluates each candidate according to the normalized full-catalog rank improvement of the ground-truth item and identifies the most and least recommendation-beneficial trajectories. Based on these utility-aware trajectory pairs, we introduce dual-level contrastive learning objectives that guide the student generator to prefer beneficial reasoning while training the recommendation encoder to more effectively exploit recommendation-relevant preference evidence. Extensive experiments on three real-world datasets demonstrate the effectiveness and robustness of the proposed framework.
Figures & tables
Figure 1: Illustration of the limitations of existing reasoning distillation methods and the proposed MUSE. Existing reasoning distillation methods rely on a single fixed general-purpose teacher, leading to heterogeneous teacher capability, teacher knowledge ceiling, and poor-quality reasoning supervision. MUSE instead adaptively selects recommendation-beneficial reasoning from multiple teachers and progressively incorporates superior student-generated reasoning through self-exploration, enabling more effective reasoning distillation for recommendation.
Figure 2: Overall architecture of MUSE. (1) Reasoning Pool Generation constructs diverse reasoning candidates from multiple teachers and progressive student self-exploration; (2) Rank-Gain Utility Assessment identifies the most and least recommendation-beneficial reasoning trajectories according to their downstream ranking improvement; and (3) Utility-Aware Contrastive Training optimizes the reasoning generator and recommendation encoder using the dual-level contrastive learning objectives.
Dataset
Users
Items
Interactions
Video Games
29,230
10,144
63,502
CDs and Vinyl
7,701
12,024
13,435
Musical Instruments
15,656
10,320
34,373
Table 1: Dataset Statistics.
Datasets
CDs and Vinyl
Methods
N@5
N@10
N@20
H@5
H@10
H@20
Caser
0.0034±0.0003
0.0046±0.0008
0.0061±0.0005
0.0068±0.0008
0.0105±0.0021
0.0166±0.0011
GRU4Rec
0.0036±0.0013
0.0049±0.0016
0.0068±0.0021
0.0059±0.0022
0.0102±0.0033
0.0178±0.0049
SASRec
0.0051±0.0009
0.0060±0.0009
0.0070±0.0006
0.0084±0.0007
0.0114±0.0008
0.0154±0.0014
TIGER
0.0023±0.0009
0.0034±0.0009
0.0041±0.0006
0.0033±0.0011
0.0069±0.0011
0.0095±0.0003
Qwen2.5-3B
Table 2: The overall performance of baselines and MUSE (Dataset: CDs and Vinyl).
Datasets
Musical Instruments
Methods
N@5
N@10
N@20
H@5
H@10
H@20
Caser
0.0087±0.0022
0.0134±0.0025
0.0191±0.0018
0.0169±0.0043
0.0317±0.0055
0.0543±0.0034
GRU4Rec
0.0090±0.0007
0.0117±0.0010
0.0152±0.0017
0.0143±0.0013
0.0225±0.0022
0.0367±0.0048
SASRec
0.0058±0.0010
0.0097±0.0004
0.0133±0.0012
0.0099±0.0013
0.0219±0.0018
0.0362±0.0058
TIGER
0.0139±0.0038
0.0173±0.0034
0.0218±0.0022
0.0208±0.0050
0.0313±0.0041
0.0491±0.0011
Qwen2.5-3B
Table 3: The overall performance of baselines and MUSE (Dataset: Musical Instruments).
Datasets
CDs and Vinyl
Methods
N@5
N@10
N@20
H@5
H@10
H@20
GPT
0.0265±0.0004
0.0323±0.0007
0.0376±0.0002
0.0372±0.0014
0.0552±0.0011
0.0764±0.0014
Gemini
0.0243±0.0011
0.0299±0.0010
0.0356±0.0012
0.0333±0.0010
0.0510±0.0009
0.0743±0.0013
DeepSeek
0.0258±0.0031
0.0316±0.0036
0.0372±0.0031
0.0388±0.0048
0.0563±0.0059
0.0783±0.0053
w/o SE
0.0258±0.0021
0.0305±0.0032
0.0380±0.0020
0.0363±0.0035
0.0510±0.0069
0.0803±0.0013
Rand
0.0269±0.0020
0.0327±0.0020
0.0373±0.0021
0.0384±0.0023
0.0559±0.0020
0.0743±0.0025
Table 4: Ablation study on key components ( Backbone : Qwen2.5-3B; Datasets : CDs and Vinyl).
Datasets
Musical Instruments
Methods
N@5
N@10
N@20
H@5
H@10
H@20
GPT
0.0155±0.0004
0.0200±0.0002
0.0264±0.0002
0.0250±0.0011
0.0388±0.0009
0.0645±0.0006
Gemini
0.0141±0.0004
0.0193±0.0012
0.0236±0.0014
0.0213±0.0006
0.0377±0.0032
0.0549±0.0039
DeepSeek
0.0146±0.0020
0.0188±0.0033
0.0238±0.0036
0.0221±0.0044
0.0353±0.0086
0.0554±0.0100
w/o SE
0.0195±0.0006
0.0231±0.0013
0.0318±0.0012
0.0282±0.0008
0.0456±0.0011
0.0719±0.0008
Rand
0.0165±0.0006
0.0218±0.0008
0.0279±0.0008
0.0243±0.0008
0.0404±0.0031
0.0645±0.0019
Table 5: Ablation study on key components ( Backbone : Qwen2.5-3B; Datasets : Musical Instruments).
Figure 3: Effect of the hyper-parameters.
Figure 4: Training time of various methods.
Figure 5: Recommendation time of various methods.
Figure 6: Validation performance and student reasoning selection ratio during training.
Methods
N@5
N@10
N@20
H@5
H@10
H@20
CDs and Vinyl
MTMR
0.0262±0.0010
0.0327±0.0026
0.0394±0.0033
0.0367±0.0004
0.0566±0.0049
0.0832±0.0079
MUSE
0.0295±0.0009
0.0348±0.0017
0.0404±0.0010
0.0421±0.0019
0.0584±0.0041
0.0804±0.0018
Musical Instruments
MTMR
0.0192±0.0010
0.0253±0.0012
0.0320±0.0019
0.0295±0.0020
0.0473±0.0028
0.0735±0.0060
MUSE
0.0194±0.0008
0.0255±0.0005
0.0323±0.0013
0.0290±0.0011
0.0480±0.0008
0.0755±0.0030
Table 6: The overall performance of MTMR and MUSE.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Datasets
Video Games
Methods
N@5
N@10
N@20
H@5
H@10
H@20
Caser
0.0233±0.0006
0.0289±0.0002
0.0366±0.0003
0.0468±0.0014
0.0644±0.0010
0.0952±0.0009
GRU4Rec
0.0083±0.0028
0.0117±0.0041
0.0145±0.0048
0.0139±0.0055
0.0243±0.0092
0.0356±0.0121
SASRec
0.0171±0.0010
0.0229±0.0010
0.0289±0.0027
0.0280±0.0023
0.0462±0.0028
0.0702±0.0111
TIGER
0.0226±0.0007
0.0265±0.0005
0.0315±0.0007
0.0328±0.0027
0.0433±0.0016
0.0662±0.0031
Qwen2.5-3B
Appendix
Table 7: The overall performance of baselines and MUSE on the Video Games dataset.
Datasets
Video Games
Methods
N@5
N@10
N@20
H@5
H@10
H@20
GPT
0.0395±0.0005
0.0482±0.0007
0.0589±0.0019
0.0598±0.0021
0.0866±0.0010
0.1294±0.0097
Gemini
0.0400±0.0008
0.0491±0.0029
0.0564±0.0043
0.0548±0.0021
0.0832±0.0066
0.1137±0.0145
DeepSeek
0.0309±0.0053
0.0390±0.0056
0.0493±0.0057
0.0481±0.0065
0.0736±0.0076
0.1146±0.0080
w/o SE
0.0413±0.0006
0.0527±0.0006
0.0623±0.0007
0.0609±0.0008
0.0998±0.0021
0.1343±0.0030
Rand
0.0376±0.0013
0.0457±0.0016
0.0567±0.0013
0.0591±0.0024
0.0844±0.0038
0.1284±0.0024
Appendix
Table 8: Ablation study on key components ( Backbone : Qwen2.5-3B; Dataset : Video Games).
With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec, a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping, integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation, which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler, dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec.
Chain-of-thought (CoT) distillation in the recommendation domain is a necessary precursor to RL training, but raw teacher traces are ill-suited to this task. Large teachers approach the recommendation task with unusually high reasoning uncertainty, repeatedly rechecking their answers without revising them; supervised fine-tuning on such traces produces verbose students that never revise their initial guess. Furthermore, due to the novelty of the recommendation domain, the teacher's reasoning traces are highly out-of-distribution for the small student LLM. We propose Student-Aware CoT Optimization for Recommendation Distillation (SCOReD), a CoT optimization framework tailored to recommendation that first parses each teacher trace into typed segments and uses the student LLM's attention to score the importance of each segment. Then SCOReD dynamically selects a per-segment edit (KEEP / REWRITE / FUSE / PRUNE) based on the output length and comparative log probability lift of the answer given the edit as per the student. Therefore, SCOReD prunes redundant sections of the reasoning trace while preserving information-dense sections and adapts raw teacher traces to the student's output distribution. Training on SCOReD-optimized CoTs provides a cleaner learning signal to the student model and improves over baseline SFT by 1.56% NDCG and 1.9% Recall@5, while reducing reasoning length by 27.3%.
Large Language Models (LLMs) have shown strong potential for recommendation by leveraging their semantic understanding and contextual modeling capabilities. Recent studies further introduce reasoning mechanisms to improve user preference modeling. However, explicit natural-language reasoning incurs substantial inference overhead, whereas existing latent reasoning methods mainly focus on generating or verifying intermediate states, leaving their layer-wise preference roles and contributions insufficiently characterized. We propose HiLaR, a Hierarchical Latent Reasoning framework with layer-aware reinforcement optimization for LLM-based recommendation. HiLaR constructs temporal-guided hierarchical user preference representations, aligns them with multiple LLM latent reasoning states, and organizes the reasoning process from broad preferences to fine-grained current intents. To further optimize the reasoning trajectory, HiLaR combines final recommendation feedback with layer-aware process rewards derived from the marginal target-likelihood gain of each state. Experiments on four Amazon benchmark datasets show that HiLaR generally outperforms strong sequential, generative, and LLM-based recommendation baselines. Ablation and sensitivity analyses further verify the contribution of hierarchical representation learning, latent alignment, and process-level optimization. Our code is available in https://github.com/hupeiyu21/HiLaR.
Peiyu Hu, Siying Gu, Weihai Lu +8
1Xi’an Jiaotong-Liverpool University · 2Xiaohongshu · 3Peking University +1