Organizations: ADREM Data Lab, University of Antwerp, Middelheimlaan 1, 2020 Antwerpen, Belgium · Department of Computer Science, PO Box 68, FI-00014 University of Helsinki, Finland · Froomle, Post X, Borsbeeksebrug 34, 2600 Antwerpen, Belgium
Recommender systems engineer serendipity to foster active exploration and break predictable consumption cycles. The problem with existing offline beyond-accuracy metrics is that they often either isolate historical similarity or global popularity. We aim to design an evaluation metric that examines similarity, popularity, and actual user relevance. To achieve this, we introduce SPADE (Serendipitous Pareto Distance Evaluation). SPADE maps all items into a two-dimensional space to directly calculate a user-specific Pareto frontier of maximally popular and historically similar items. The final serendipity score is then computed by averaging the minimum Euclidean distance from this boundary strictly for the correctly recommended test-set items. Evaluating SPADE across five datasets and five baseline algorithms confirms its effectiveness; our results show that the metric successfully prevents algorithms from exploiting beyond-accuracy measures with irrelevant or non-personalized recommendations, reliably isolating serendipitous discoveries.
Figures & tables
Dataset
# Interactions
# Users
# Items
Avg.#Int. per user
Avg.#Int. per item
Sparsity
CiteULike
200,251
5,550
15,439
36.08
12.97
99.77%
MIND*
1,086,939
10,000
33,220
108.69
32.72
99.67%
ML-1M
574,385
6,038
3,125
95.13
183.80
96.96%
ML-20M*
730,034
10,000
11,361
73.00
64.26
99.36%
Netflix*
1,234,880
10,000
15,093
123.49
81.82
99.18%
Table 1: Statistical information on the datasets used in our experiments. Datasets marked with (*) have been subsampled to 10,000 users to keep within hardware limitations.
Figure 1: The y -axis denotes normalized item similarity to the user’s historical profile, and the x -axis denotes global item popularity. Candidate items are represented by blue dots, EASE baseline predictions in green, and ground truth interactions (test set) by red crosses. The solid line indicates the Pareto front maximizing similarity and popularity. More serendipitous items are in the bottom-left quadrant. These relevant but unexpected items lie furthest from the Pareto front and therefore receive the highest serendipity scores under the SPADE metric.
Figure 2: Performance comparison of baseline recommendation algorithms (EASE, SLIM, ItemKNN, and Popularity) across varying recommendation list sizes ( K∈{1,2,3,5,10,20,30} ). The subplots depict the evaluation trajectories for accuracy (NDCG) and beyond-accuracy metrics (Primitivity, Co-Occurrence Surprise, Novelty, and SPADE) as the cutoff threshold increases.
Dataset
Algorithm
NDCGK
Primitivity
CoOccurence
Novelty
SPADE
Netflix
EASE
0.315685
4.844650
0.103907
1.168139
0.127368
Netflix
SLIM
0.280544
4.544350
0.100814
1.110615
0.117351
Netflix
ItemKNN
0.190461
3.705450
0.085582
0.993255
0.061379
Netflix
Popularity
0.109781
0.000000
0.076752
0.783255
0.029467
Netflix
Random
0.001734
5.498050
0.214283
inf
0.004001
ML-1M
EASE
0.334812
4.396189
0.095179
1.094719
0.101459
Table 2: Cross-domain evaluation of baseline recommendation models (EASE, SLIM, ItemKNN, and Popularity) at a top- K cutoff of 10. The table details ranking accuracy alongside beyond-accuracy metrics to illustrate the trade-off between relevance and serendipitous discovery across the different datasets.
One of the enduring challenges in online platforms is recommendation. Both communities and advertisers face difficulties in analyzing complex user behavior, identifying habits and evolving preferences, and providing timely recommendations that are relevant to user interests. In particular, point of interest recommendation poses the additional challenge of identifying the factors that influence user behavior, capturing user tendencies, and generating accurate, personalized recommendations for each individual. In this work, we propose SOAR, a Social and Opinion Aware Recommendation model that incorporates both the user's friendship network and textual reviews to assess whether a location is suitable for a specific user, with a particular emphasis on the sentiment of shared reviews. Experimental results in real-world datasets demonstrate that incorporating reviews into the recommendation process improves the precision of the recommendation.
Giannis Christoforidis, Apostolos N. Papadopoulos
School of Informatics, Aristotle University of Thessaloniki, AUTH Campus, Thessaloniki, 54124, Greece.
Fairness-aware recommender systems often mitigate bias by increasing exposure to under-represented or long-tail content, commonly through mechanisms that promote novelty and diversity. In practice, the strength of such interventions is typically controlled using global hyperparameters, fixed regularization weights, heuristic caps, or offline tuning strategies. These approaches implicitly assume that a single level of exploration is appropriate across users, contexts, and stages of interaction. In this work, we study exploration saturation as a user-dependent phenomenon arising from fairness- and novelty-driven recommendation strategies. We define exploration saturation as the point at which further increases in exploration no longer improve user utility and may instead reduce engagement or perceived relevance. Rather than proposing a new fairness-aware algorithm or optimizing a specific objective, we empirically analyze how increasing exploration affects users across varied recommendation models. Through longitudinal experiments using MovieLens-1M and Last.fm datasets, our results indicate that fairness-induced exploration exhibits diminishing or non-monotonic returns and varies substantially across users. In particular, users with limited interaction histories tend to reach saturation earlier, suggesting that uniform fairness or novelty pressure can disproportionately disadvantage certain users. These findings reveal a trade-off between fairness and user experience, suggesting that recommendation systems should adapt not only to relevance but also to the amount of fairness-driven exploration applied to individual users.
Enock O. Ayiku, Evelyn Osei, Emebo Onyeka
University of Massachusetts Boston, Boston, MA, USA · Kansas State University, Manhattan, KS, USA · Virginia Tech, Blacksburg, VA, USA
Sequential recommendation has increasingly shifted toward generative recommenders that combine sequential patterns with semantic item information. Yet these methods are often evaluated on a small set of widely used benchmarks, raising a key question: do these benchmarks actually require the advanced modeling capabilities that modern generative recommenders claim to provide? We conduct a benchmark audit with an intentionally simple graph heuristic. Starting from only the last one or two interacted items, it retrieves candidates from a few-hop item-transition graph and ranks them by item-feature similarity. Despite using no sequence encoder, generative objective, or training, this heuristic matches or outperforms many modern baselines, with relative NDCG@10 improvements of 38.10% and 44.18% over the best competing baseline on Amazon Review Sports and CDs. We show that this behavior reflects shortcut solvability rather than an artifact of one heuristic. We identify three shortcut structures that can make next-item prediction easier than expected: low-branching local transitions, feature-smooth transitions, and limited dependence on long user histories. These shortcuts need not appear together; even one or two strong signals can make simple local retrieval highly competitive, while weakening them makes the benefits of more sophisticated models clearer. Across 14 datasets, model rankings vary substantially with dataset properties, yet the heuristic remains competitive on 10 of them. Our findings suggest that strong performance on standard benchmarks does not always demonstrate advanced sequential, semantic, or generative modeling ability. We call for more careful dataset selection and dataset-level diagnostic analysis when using benchmarks to support claims about new recommendation models.