Multi-objective re-ranking has become a critical component of modern multi-stage recommender systems, as it tasked to balance multiple conflicting objectives such as accuracy, diversity, and fairness. Existing multi-objective re-ranking methods typically optimize aggregate objectives at the item level using static or handcrafted preference weights. This design overlooks that users inherently exhibit Pareto-optimal preferences at the intent level, reflecting personalized trade-offs among objectives rather than fixed weight combinations. Moreover, most approaches treat re-ranking task for each user as an isolated problem, and repeatedly learn the preferences from scratch. Such a paradigm not only incurs high computational cost, but also ignores the fact that users often share similar preference trade-off structures across objectives. Inspired by the existence of homogeneous multi-objective optimization spaces where Pareto-optimal patterns are transferable, we propose PreferRec, a novel framework that explicitly models and transfers Pareto preferences across users. Specifically, PreferRec is built upon three tightly coupled components: Preference-Aware Pareto Learning aims to capture user intrinsic trade-offs among multiple conflicting objectives at the intent level. By learning Pareto preference representations from re-ranking populations, this component explicitly models how users prioritize different objectives under diverse contexts. Knowledge-Guided Transfer facilitates efficient cross-user knowledge transfer by distilling shared optimization patterns across homogeneous optimization spaces. The transferred knowledge is then used to guide solution selection and personalized re-ranking, biasing the optimization process toward high-quality regions of the Pareto front while preserving user-specific preference characteristics.
Figures & tables
Figure 1. Comparison of different multi-objective recommendation paradigms. The left and middle panels illustrate traditional accuracy-oriented Top-K sorting and linear trade-off based diversified re-ranking (e.g., MMR), respectively. The right panel depicts the proposed PreferRec framework. A three-panel schematic comparing recommendation paradigms. The left panel shows Top-K ranking based only on relevance scores. The middle panel shows diversified re-ranking using a linear trade-off between relevance and diversity. The right panel shows the PreferRec framework, which applies multi-objective evolutionary optimization and Pareto set learning.
Figure 2. Overview of PreferRec. The framework integrates evolutionary multi-objective optimization with global knowledge transfer. Guided initialization generates initial solutions for each user, which are optimized by evolutionary search. During optimization, preference vectors are periodically constructed from current solutions and used to train a Pareto Learning Network. The learned global knowledge is directly mixed with user-specific solutions to guide subsequent search. Final recommendations are selected from the Pareto front under different trade-off preferences. An overview diagram of the PreferRec framework. The figure shows a user-specific evolutionary optimization process connected with a global knowledge learning module. Initial candidate solutions are generated for each user and optimized through evolutionary search. Preference vectors are extracted from intermediate solutions and used to train a Pareto learning network. The learned global knowledge is fed back and combined with user-specific solutions to guide subsequent optimization. The final recommendation list is selected from the Pareto front according to different trade-off preferences.
Dataset
Users
Items
Interactions
Categories
ML-1M
6,040
3,076
1,000,209
18
Grocery
14,681
8,713
151,254
153
Beauty
22,363
12,101
198,520
248
Table 1. Statistic of datasets
Dataset
Method
Accuracy
Diversity and Novelty
Trade-off
HR@5
HR@10
NDCG@5
NDCG@10
Div@5
Div@10
Nov@5
Nov@10
F1@5
F1@10
F2@5
F2@10
ML-1M
POP
0.3086
0.4575
0.2034
0.2513
0.4285
0.5693
0.2299
0.3198
0.0943
0.1855
0.0930
0.1867
GRU4Rec(2015)
0.6321
0.7652
0.4568
0.5000
0.3716
0.5101
0.4043
0.4776
0.2024
0.3190
0.2288
0.3558
NARM(2017)
0.6364
0.7672
0.4659
0.5084
0.3645
0.5050
0.4274
0.4954
0.2082
0.3257
0.2346
0.3622
SASRec(2018)
0.7013
0.8079
0.5506
0.5853
0.3655
0.5039
0.4371
0.4980
0.2235
0.3360
0.2578
0.3789
ContraRec(2023)
0.7048
0.8025
0.5532
0.5849
0.3621
0.5005
0.4433
0.5064
0.2247
0.3372
0.2593
0.3790
Table 2. Performance Comparison on Three Datasets. The models are categorized into Single-Objective (Single) and Diversity-Oriented/Multi-Objective (Multi) baselines. The Trade-off metrics (our core contribution) are highlighted.
Figure 3. Performance comparison of PreferRec instantiated with different base models (ComiRec, SASRec, GRU4Rec) against representative baselines in dataset ML-1M. The impact of different basemodels.
Figure 4. Ablation study on Knowledge Transfer. The impact of Knowledge Transfer.
Figure 5. Performance and training time under different knowledge transfer intervals t . Performance and training time under different knowledge transfer intervals t.
Figure 6. Performance and time cost under different population sizes. Performance and time cost under different population sizes.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7. Sensitivity analysis of crossover and mutation probabilities. Higher crossover probabilities consistently improve trade-off performance, while the mutation probability exhibits a non-monotonic pattern, with the best F1/F2 achieved at 0.2. Sensitivity analysis of crossover and mutation probabilities. Higher crossover probabilities consistently improve trade-off performance, while the mutation probability exhibits a non-monotonic pattern, with the best F1/F2 achieved at 0.2.
Nc
F1@5
F2@5
F1@10
F2@10
2
0.3123
0.3533
0.4581
0.497
5
0.3051
0.3402
0.4808
0.5302
10
0.3127
0.3522
0.4748
0.5197
Appendix
Table 3. Performance under different numbers of clusters Nc .
Figure 8. Performance comparison (radar charts) of PreferRec instantiated with different base models against representative baselines. The top row shows metrics at @5, and the bottom row shows metrics at @10. A six-panel radar chart comparing PreferRec with ComiRec, SASRec, and GRU4Rec backbones at @5 and @10.
Figure 9. Case study comparing GRU4Rec and PreferRec on grocery recommendation. Case study comparing GRU4Rec and PreferRec on grocery recommendation.
Dataset
Method
Accuracy
Diversity and Novelty
Trade-off
HR@5
HR@10
NDCG@5
NDCG@10
Div@5
Div@10
Nov@5
Nov@10
F1@5
F1@10
F2@5
F2@10
ML-1M
POP
0.3086
0.4575
0.2034
0.2513
0.4285
0.5693
0.2299
0.3198
0.0943
0.1855
0.0930
0.1867
GRU4Rec(2015)
0.6321
0.7652
0.4568
0.5000
0.3716
0.5101
0.4043
0.4776
0.2024
0.3190
0.2288
0.3558
NARM(2017)
0.6364
0.7672
0.4659
0.5084
0.3645
0.5050
0.4274
0.4954
0.2082
0.3257
0.2346
0.3622
SASRec(2018)
0.7013
0.8079
0.5506
0.5853
0.3655
0.5039
0.4371
0.4980
0.2235
0.3360
0.2578
0.3789
ContraRec(2018)
0.7048
0.8025
0.5532
0.5849
0.3621
0.5005
0.4433
0.5064
0.2247
0.3372
0.2593
0.3790
Appendix
Table 4. Performance Comparison on three datasets with SASRec as the Base Model. The models are categorized into Single-Objective (Single) and Diversity-Oriented/Multi-Objective (Multi) baselines. The Trade-off metrics (our core contribution) are highlighted.
Dataset
Method
Accuracy
Diversity and Novelty
Trade-off
HR@5
HR@10
NDCG@5
NDCG@10
Div@5
Div@10
Nov@5
Nov@10
F1@5
F1@10
F2@5
F2@10
ML-1M
POP
0.3086
0.4575
0.2034
0.2513
0.4285
0.5693
0.2299
0.3198
0.0943
0.1855
0.0930
0.1867
GRU4Rec(2015)
0.6321
0.7652
0.4568
0.5000
0.3716
0.5101
0.4043
0.4776
0.2024
0.3190
0.2288
0.3558
NARM(2017)
0.6364
0.7672
0.4659
0.5084
0.3645
0.5050
0.4274
0.4954
0.2082
0.3257
0.2346
0.3622
SASRec(2018)
0.7013
0.8079
0.5506
0.5853
0.3655
0.5039
0.4371
0.4980
0.2235
0.3360
0.2578
0.3789
ContraRec(2023)
0.7048
0.8025
0.5532
0.5849
0.3621
0.5005
0.4433
0.5064
0.2247
0.3372
0.2593
0.3790
Appendix
Table 5. Performance Comparison on three datasets with ComiRec as the Base Model. The models are categorized into Single-Objective (Single) and Diversity-Oriented/Multi-Objective (Multi) baselines. The Trade-off metrics (our core contribution) are highlighted.
Recommender systems often need to balance multiple conflicting objectives. Controllable Multi-Objective Recommendation (CMOR) enables dynamic adjustment of objective trade-offs at inference time without retraining, adapting to shifting user or system requirements. Existing CMOR approaches typically rely on output-level reranking or parameter-generating modules, which either limit the scope of feasible trade-offs or incur substantial training and inference cost. Model merging provides a promising alternative by reusing multiple objective-specific models via parameter combination. However, its direct application to recommender systems is challenging due to the lack of a shared linear regime and cross-objective interference. To address these issues, we propose RecSoup, a lightweight two-stage approach for effective CMOR via specialized model merging. RecSoup first mitigates parameter interference across objectives through a pruning strategy that jointly considers objective-specific importance and cross-objective relevance. It then performs adaptive parameter merging via a learnable preference-to-coefficient mapping, allowing dynamic trade-off control at inference time without joint retraining or additional generative modules. Theoretically, we prove that RecSoup outperforms the parameter generation paradigm in effectiveness and efficiency, as it can fully navigate the Pareto front without smoothness constraints. Extensive experiments on multiple benchmarks demonstrate that RecSoup achieves improved Pareto front quality and effective controllability while delivers 165--228x faster inference-time latency than parameter generation approaches. Code is available at https://github.com/35798642/RECSOUP.git.
Zidong Jia, Guofu Xie, Xiao Zhang
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Recommender systems often induce filter bubbles and semantic homogenization by monolithically optimizing for immediate user engagement. Standard single-objective models, including traditional Deep Q-Networks, are ill-equipped to navigate the trade-offs between platform retention and critical societal values like information diversity and provider fairness. To address these limitations, we introduce a multi-objective reinforcement learning framework that formalizes recommendation as a semantic multi-objective Markov decision process. By integrating high-fidelity semantic embeddings with a Pareto-DQN agent, our architecture treats engagement, diversity, and fairness as distinct, non-aggregable reward signals, avoiding the pitfalls of static reward scalarization. Empirical evaluations on the MovieLens small dataset shows that our hypervolume based action selection disrupts the feedback loops responsible for semantic collapse. By sustaining high state-trajectory variance, the Pareto-DQN effectively maps the Pareto frontier, achieving gains in auxiliary societal objectives with only marginal impacts on engagement. This work provides a path toward intrinsically aligned, responsible recommender systems.
Cláudio Lúcio Do Val Lopes, Lucca Machado da Silva, André de Oliveira Brandão
This paper presents DeepMTL2R, an open-source deep learning framework for Multi-task Learning to Rank (MTL2R), where multiple relevance criteria must be optimized simultaneously. DeepMTL2R integrates heterogeneous relevance signals into a unified, context-aware model by leveraging the self-attention mechanism of transformer architectures, enabling effective learning across diverse and potentially conflicting objectives. The framework includes 21 state-of-the-art multi-task learning algorithms and supports multi-objective optimization to identify Pareto-optimal ranking models. By capturing complex dependencies and long-range interactions among items and labels, DeepMTL2R provides a scalable and expressive solution for modern ranking systems and facilitates controlled comparisons across MTL strategies. We demonstrate its effectiveness on a publicly available dataset, report competitive performance, and visualize the resulting trade-offs among objectives. DeepMTL2R is available at https://github.com/amazon-science/DeepMTL2R.