SPADE: Escaping the Popularity-Similarity Frontier to Measure Serendipitous Recommendations

Tobias Vente; Maarten Peirsman; Noah Daniëls; Hannu Toivonen; Bart Goethals

cs.IRSep 25, 2026

SPADE: Escaping the Popularity-Similarity Frontier to Measure Serendipitous Recommendations

Authors: Tobias Vente, Maarten Peirsman, Noah Daniëls, Hannu Toivonen, Bart Goethals

Organizations: ADREM Data Lab, University of Antwerp, Middelheimlaan 1, 2020 Antwerpen, Belgium · Department of Computer Science, PO Box 68, FI-00014 University of Helsinki, Finland · Froomle, Post X, Borsbeeksebrug 34, 2600 Antwerpen, Belgium

Abstract

Recommender systems engineer serendipity to foster active exploration and break predictable consumption cycles. The problem with existing offline beyond-accuracy metrics is that they often either isolate historical similarity or global popularity. We aim to design an evaluation metric that examines similarity, popularity, and actual user relevance. To achieve this, we introduce SPADE (Serendipitous Pareto Distance Evaluation). SPADE maps all items into a two-dimensional space to directly calculate a user-specific Pareto frontier of maximally popular and historically similar items. The final serendipity score is then computed by averaging the minimum Euclidean distance from this boundary strictly for the correctly recommended test-set items. Evaluating SPADE across five datasets and five baseline algorithms confirms its effectiveness; our results show that the metric successfully prevents algorithms from exploiting beyond-accuracy measures with irrelevant or non-personalized recommendations, reliably isolating serendipitous discoveries.

Figures & tables

Dataset	# Interactions	# Users	# Items	Avg.#Int. per user	Avg.#Int. per item	Sparsity
CiteULike	200,251	5,550	15,439	36.08	12.97	99.77%
MIND*	1,086,939	10,000	33,220	108.69	32.72	99.67%
ML-1M	574,385	6,038	3,125	95.13	183.80	96.96%
ML-20M*	730,034	10,000	11,361	73.00	64.26	99.36%
Netflix*	1,234,880	10,000	15,093	123.49	81.82	99.18%

Table 1: Statistical information on the datasets used in our experiments. Datasets marked with (*) have been subsampled to 10,000 users to keep within hardware limitations.

Panel of Figure 1 — Figure 1: The $y$ -axis denotes normalized item similarity to the user’s historical profile, and the $x$ -axis denotes global item popularity. Candidate items are represented by blue dots, EASE baseline predictions in green, and ground truth interactions (test set) by red crosses. The solid line indicates the Pareto front maximizing similarity and popularity. More serendipitous items are in the bottom-left quadrant. These relevant but unexpected items lie furthest from the Pareto front and therefore receive the highest serendipity scores under the SPADE metric.

Figure 2: Performance comparison of baseline recommendation algorithms (EASE, SLIM, ItemKNN, and Popularity) across varying recommendation list sizes ( $K\in\{1,2,3,5,10,20,30\}$ ). The subplots depict the evaluation trajectories for accuracy (NDCG) and beyond-accuracy metrics (Primitivity, Co-Occurrence Surprise, Novelty, and SPADE) as the cutoff threshold increases.

Dataset	Algorithm	NDCGK	Primitivity	CoOccurence	Novelty	SPADE
Netflix	EASE	0.315685	4.844650	0.103907	1.168139	0.127368
Netflix	SLIM	0.280544	4.544350	0.100814	1.110615	0.117351
Netflix	ItemKNN	0.190461	3.705450	0.085582	0.993255	0.061379
Netflix	Popularity	0.109781	0.000000	0.076752	0.783255	0.029467
Netflix	Random	0.001734	5.498050	0.214283	inf	0.004001
ML-1M	EASE	0.334812	4.396189	0.095179	1.094719	0.101459

Table 2: Cross-domain evaluation of baseline recommendation models (EASE, SLIM, ItemKNN, and Popularity) at a top-

K

cutoff of 10. The table details ranking accuracy alongside beyond-accuracy metrics to illustrate the trade-off between relevance and serendipitous discovery across the different datasets.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

SOAR: Social and Opinion-Aware Recommendations
Oct 21, 2026Giannis Christoforidis, Apostolos N. PapadopoulosRecommender Systems
School of Informatics, Aristotle University of Thessaloniki, AUTH Campus, Thessaloniki, 54124, Greece.
Modeling User Exploration Saturation: When Recommender Systems Should Stop Pushing Novelty
Apr 2, 2026Enock O. Ayiku, Evelyn Osei, Emebo OnyekaRecommender Systems Fairness in Recommender Systems
University of Massachusetts Boston, Boston, MA, USA · Kansas State University, Manhattan, KS, USA · Virginia Tech, Blacksburg, VA, USA
An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation
May 8, 2026Haoyu Han, Li Ma, Hanbing Wang +9Sequential Recommendation Benchmark Design
Michigan State University · Airbnb, Inc.

Oct 21, 2026J/K move · Enter open · S save

SOAR: Social and Opinion-Aware Recommendations

Giannis Christoforidis, Apostolos N. Papadopoulos

School of Informatics, Aristotle University of Thessaloniki, AUTH Campus, Thessaloniki, 54124, Greece.

Recommender Systems

Open paper AskPDF arXiv

One of the enduring challenges in online platforms is recommendation. Both communities and advertisers face difficulties in analyzing complex user behavior, identifying habits and evolving preferences, and providing timely recommendations that are relevant to user interests. In particular, point of interest recommendation poses the additional challenge of identifying the factors that influence user behavior, capturing user tendencies, and generating accurate, personalized recommendations for each individual. In this work, we propose SOAR, a Social and Opinion Aware Recommendation model that incorporates both the user's friendship network and textual reviews to assess whether a location is suitable for a specific user, with a particular emphasis on the sentiment of shared reviews. Experimental results in real-world datasets demonstrate that incorporating reviews into the recommendation process improves the precision of the recommendation.

Abstract

Recommender Systems

Explore similar work

SOAR: Social and Opinion-Aware Recommendations

Modeling User Exploration Saturation: When Recommender Systems Should Stop Pushing Novelty

An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation