From Counterfactual Edits to Human Narratives: LLM-Driven Interpretations for Recommender System Explanations
Organizations: Faculty of Information Technology, Czech Technical University in Prague, Prague, Czechia · Department of Computer Science, University of Innsbruck, Innsbruck, Austria
Abstract
Recommender systems increasingly influence what users discover and consume online, yet the decision logic behind their outputs often remains opaque. Counterfactual explanation (CFX) methods provide an effective way to show which changes to a user's interaction history would alter a recommendation. In recommendation settings, however, CFXs often require many edits, producing long, low-level lists that are hard for users to interpret. Moreover, most evaluations emphasize edit counts while largely overlooking whether the resulting CFXs are coherent and actionable for users. We argue that this gap between generating valid CFXs and communicating model behavior stems from the lack of user-centered abstraction and limited qualitative evaluation. To bridge this gap, we investigate large language models as mediators that summarize raw CFX interaction edit lists into concise natural language explanations describing higher-level preference patterns. We optimize these mediators for correctness, informativeness, and linguistic quality through preference optimization, and evaluate the resulting explanations with human-calibrated metrics derived from separate human-labeled datasets. On the MovieLens-1M dataset, across multiple counterfactual generators and language models, the resulting summaries achieve high overall linguistic quality, while the strongest semantic results are obtained with high-success CFX methods such as ACCENT and SPINRec. Direct Preference Optimization improves the semantic metrics in almost all evaluated settings.
Figures & tables
| CFX Generator | CFX Success Rate | CFX Size | Simple CFX Rate | LLM | Correctness | Informativeness | Linguistic Quality | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Vanilla LLM | DPO | Vanilla LLM | DPO | Vanilla LLM | DPO | |||||
| Jaccard | 0.95 | 16.38 11.67 | 0.02 | Ministral 8B | 0.34 0.19 | 0.41 0.20 | 0.14 0.19 | 0.12 0.20 | 0.88 0.33 | 0.75 0.44 |
| Gemma 3 12B | 0.39 0.21 | 0.51 0.18 | 0.13 0.20 | 0.17 0.23 | 0.72 0.45 | 0.84 0.37 | ||||
| Qwen3 8B | 0.34 0.19 | 0.46 0.20 | 0.13 0.19 | 0.18 0.24 | 0.85 0.36 | 0.71 0.46 | ||||
| Cosine | 0.96 | 16.20 11.23 | 0.02 | Ministral 8B | 0.34 0.19 | 0.40 0.21 | 0.16 0.18 | 0.17 0.21 | 0.91 0.29 | 0.78 0.41 |
| Gemma 3 12B | 0.38 0.21 | 0.41 0.22 | 0.13 0.19 | 0.14 0.21 | 0.73 0.44 | 0.78 0.42 | ||||
| Explanation | Correctness | Informativeness | Linguistic Quality | |||
|---|---|---|---|---|---|---|
| LLM judge | Human-cal. | LLM judge | Human-cal. | LLM judge | Human-cal. | |
| (Because you watched) comedy and drama films from 1990s | 0.35 | 0.53 | 0.02 | 0.08 | 0.83 | 1.00 |
| (Because you watched) adventure romances from 1995 | 0.21 | 0.12 | 0.11 | 0.12 | 0.94 | 1.00 |
| (Because you watched) drama films from 1960s to 1990s | 0.83 | 0.50 | 0.44 | 0.40 | 0.89 | 0.00 |
| (Because you watched) mystery and action films from 1970s to 1990s | 0.60 | 0.40 | 0.28 | 0.25 | 1.00 | 1.00 |
| (Because you watched) comedy films from 1980s and 1990s | 0.46 | 0.25 | -0.15 | -0.20 | 0.89 | 1.00 |
Explainable Recommendation
Explainable recommendation develops systems that show users the evidence and preferences behind suggested items, rankings, or decisions.
See all 46 papers