Why This, Not That? Mining User Profiles for Pair-wise Counterfactuals

Meysam Varasteh; Veronika Bogina; Noam Koenigstein; Robin Burke

cs.IRAug 21, 2026

Why This, Not That? Mining User Profiles for Pair-wise Counterfactuals

Authors: Meysam Varasteh, Veronika Bogina, Noam Koenigstein, Robin Burke

Organizations: Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309, USA · Tel Aviv University, Tel Aviv, Israel · Department of Information Science, University of Colorado Boulder, Boulder, CO 80309, USA

Abstract

The topic of explanation in recommender systems has seen steady research attention since the earliest days of the field. With some exceptions, this work has focused on the explanation of single items in a recommendation list and, especially recently, has emphasized approaches that are decoupled from the logic of the recommendation algorithm itself. Based on findings in the psychology of interpersonal communication, we propose a new task, pairwise interpretation of item rankings, asking the comparative question ``Why is item A ranked higher than item B?''. An effective solution to this task, we argue, is inherently grounded in the operation of the recommendation algorithm. We propose a class of techniques based on counterfactual learning to uncover the items in a user's profile that have contributed to the relative ranking of items. Using multiple datasets, we show that it is possible to identify such items as potential basis for comparative explanation.

Figures & tables

Symbol	Description
$\mathcal{U}$	The set of users.
$\mathcal{V}$	The set of items.
$\mathbf{x}$	The user profile,
	represented as a binary vector over items.
$\mathbf{y}_{t},\mathbf{y}_{c}$	The target and comparative items,
	represented as a one-hot vectors.

Table 1: Table of notations

	ML1M		Yahoo		Pinterest
	HR	CC	HR	CC	HR	CC
MF	0.08	0.13	0.23	0.20	0.04	0.13
VAE	0.19	0.44	0.40	0.23	0.1	0.72

Table 2: Hit rate (HR) and catalog coverage (CC) at 10 for the recommendation algorithms / dataset combinations.

Recommender	Method	ML-1M		Yahoo		Pinterest
		Coverage $\uparrow$	$\text{MPRR}_{\#}$ $\downarrow$	Coverage $\uparrow$	$\text{MPRR}_{\#}$ $\downarrow$	Coverage $\uparrow$	$\text{MPRR}_{\#}$ $\downarrow$
MF	Popularity	56%	50	66%	15	62%	16
	LIME	59%	37	67%	11	60%	12
	ACCENT	68%	28	79%	11	58%	14
	SHAP	32%	52	63%	19	57%	16
	LXR	72%	33	78%	11	74%	11

Table 3: Coverage and MPRR scores for the MF and VAE recommenders on the three datasets. The best result is shown in bold .

Panel of Figure 1 — Figure 1: Coverage versus minimum perturbation size for the different algorithms.

Figure 2: Example of comparative explanation generation. The original profile and recommendations appear at the top, with Alien as the target item and Braveheart as the comparative item. The perturbed profile and new recommendations are shown at the bottom.

Recommender	Explainer	ML-1M		Yahoo
		Coverage $\uparrow$	$\text{MPRR}_{\%}$ $\downarrow$	Coverage $\uparrow$	$\text{MPRR}_{\%}$ $\downarrow$
MF	CLXR-joint	81%	25.3%	87.3%	25%
MF	CLXR-filter	56.3%	25.8%	24%	27%
VAE	CLXR-joint	93.3%	22.3%	80.5%	30%
VAE	CLXR-filter	81.1%	22.5%	26%	24%

Table 4: Comparison of CLXR-joint (as above) and CLXR-filter

Explainable Recommendation

Explainable recommendation develops systems that show users the evidence and preferences behind suggested items, rankings, or decisions.

See all 36 papers

From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems
Apr 21, 2026Quang-Huy Nguyen, Thanh-Hai Nguyen, Khac-Manh Thai +6Counterfactual Evaluation Counterfactual Explanations
VNU University of Engineering and Technology · Delft University of Technology
A Counterfactual Approach for Addressing Individual User Unfairness in Collaborative Recommender System
Feb 24, 2026Nikita Baidya, Bidyut Kr. Patra, Ratnakar DashFairness in Recommender Systems Recommender Systems
Pattern Recognition Laboratory, Department of Computer Science and Engineering, National Institute of Technology Rourkela, Odisha, India. · Department of Computer Science and Engineering, Indian Institute of Technology (BHU) Varanasi, Uttar Pradesh, India.
Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles
Sep 17, 2026Noah Mamié, Laurin van den BerghRecommender Systems Large Language Model-Based Recommendation
University of Zurich Department of Informatics

Apr 21, 2026J/K move · Enter open · S save

From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems

Quang-Huy Nguyen, Thanh-Hai Nguyen, Khac-Manh Thai, Duc-Hoang Pham+5

VNU University of Engineering and Technology · Delft University of Technology

Counterfactual Evaluation Counterfactual Explanations Recommender Systems Explainability Evaluation Explainable Recommendation

Open paper AskPDF arXiv

Counterfactual explanations (CEs) provide an intuitive way to understand recommender systems by identifying minimal modifications to user-item interactions that alter recommendation outcomes. Existing CE methods for recommender systems, however, have been evaluated under heterogeneous protocols, using different datasets, recommenders, metrics, and even explanation formats, which hampers reproducibility and fair comparison. Our paper systematically reproduces, re-implement, and re-evaluate eleven state-of-the-art CE methods for recommender systems, covering both native explainers (e.g., LIME-RS, SHAP, PRINCE, ACCENT, LXR, GREASE) and specific graph-based explainers originally proposed for GNNs. Here, a unified benchmarking framework is proposed to assess explainers along three dimensions: explanation format (implicit vs. explicit), evaluation level (item-level vs. list-level), and perturbation scope (user interaction vectors vs. user-item interaction graphs). Our evaluation protocol includes effectiveness, sparsity, and computational complexity metrics, and extends existing item-level assessments to top-K list-level explanations. Through extensive experiments on three real-world datasets and six representative recommender models, we analyze how well previously reported strengths of CE methods generalize across diverse setups. We observe that the trade-off between effectiveness and sparsity depends strongly on the specific method and evaluation setting, particularly under the explicit format; in addition, explainer performance remains largely consistent across item level and list level evaluations, and several graph-based explainers exhibit notable scalability limitations on large recommender graphs. Our results refine and challenge earlier conclusions about the robustness and practicality of CE generation methods in recommender systems: https://github.com/L2R-UET/CFExpRec.

Abstract

Explainable Recommendation

Explore similar work

From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems

A Counterfactual Approach for Addressing Individual User Unfairness in Collaborative Recommender System

Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles