With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec, a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping, integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation, which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler, dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec.
Figures & tables
Figure 1: Example of Reasoning-Augmented LLM-based Recommendation Assistant.
Figure 2: The overall model architecture of the proposed ReRec.
Category
Sub-category
Movie
Book
Condition-based Query
Explicit Condition (Simple)
8,262
10,681
Implicit Condition (Medium)
5,790
7,741
Misinformed Condition (Hard)
5,374
7,890
User Profile-based Query
Interest-based
2,365
1,273
Demographics-based
209
-
Total
22,000
27,585
Table 1: Dataset Statistics
Category
Model
Movie
Book
Simple
Medium
Hard
Interest -based
Demographics -based
Simple
Medium
Hard
Interest -based
LLM Backbone
Qwen-2.5-3B-Instruct
0.284
0.135
0.101
0.369
0.450
0.371
0.304
0.177
0.416
Llama-3.2-3B-Instruct
0.107
0.052
0.029
0.097
0.193
0.215
0.138
0.106
0.254
DS-R1-Distill-Qwen-7B
0.083
0.041
0.040
0.133
0.165
0.131
0.104
0.087
0.221
GPT-4o
0.554
0.519
0.188
0.550
0.504
0.554
0.590
0.160
0.458
Deepseek-R1
0.537
0.510
0.200
0.459
0.425
0.562
0.530
0.279
0.505
Table 2: The overall performance of baselines and ReRec evaluated by Accuracy. The bold/underline values represent the best/the second-best result, respectively.
Figure 3: Performance on personalized recommendation
Training Dataset
Method
Base Model
Testing Dataset
Movie
Book
zero-shot
prompt
Qwen
0.240
0.301
prompt
Llama
0.078
0.168
prompt
GPT-4o
0.470
0.453
prompt
DeepSeek-R1
0.441
0.474
Movie
ReRec
Qwen
0.567
0.434
Table 3: Performance in the generalization (cross-domain) setting. Bold: Accuracy on new domain.
Model
Accuracy
Llama-3.2-3B-Instruct
0.120
Qwen-2.5-3B-Instruct
0.286
GRU4Rec
0.658
SASRec
0.673 ( best )
ReRec-Qwen
0.591 (vs. Qwen +107% | vs. best 87.8%)
ReRec-Llama
0.595 (vs. Llama +396% | vs. best 88.4%)
Table 4: Performance on transferring to sequential recommendation.
Figure 4: Knowledge and Capability Retention
Figure 5: Ablation of ReRec
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Query
Required Capability
Explicit Condition Query
I’m really interested in classic films and would love to watch something that showcases Charlie Chaplin ’s legendary comedic talent. Additionally, I’ve heard that Roland Totheroh’s cinematography adds an exceptional visual quality to movies. If you could point me in the direction of films that include both of these elements, I’d greatly appreciate it!
Direct Reasoning: Identifies specific attributes (e.g., director, cinematographer) and matches them directly.
Implicit Condition Query
I recently watched Clockers (1995) and Bamboozled (2000) , and I was really impressed by the direction in both films. I’m eager to explore more works from the director, as I found their storytelling style and vision very engaging. If you could suggest other films associated with this director , that would be fantastic.
Multi-hop Reasoning: Infers the attribute from given information and then generate corresponding recommendations.
Misinformed Condition Query
I recently watched Lorenzo’s Oil and was really impressed by the cinematography done by Mac Ahlberg . I’m interested in finding more films that showcase his cinematographic style. I also remember seeing his work in Beyond Rangoon , so if there are any other movies he contributed to, I’d love to check them out!
Reflection: Detects and corrects misinformation (e.g., Mac Ahlberg did not work on these films) before generating recommendations.
Interest-based Query
I’m fond of romantic and dramatic films from the golden age of Hollywood like ‘Roman Holiday’ and ‘My Fair Lady’. Are there any other dramatic romances from that period you would recommend?
Contextual Reasoning: Leverages user interest context to suggest similar content.
Demopraphics-based Query
I’m a psychology professor and I’m looking for movies that delve into human emotions and relationships . Have you got any?