Reading the Mood: Emotion-Guided Book-to-Music Recommendation via CGANs and LLMs

Manousos Linardakis; Georgios Alexandridis

cs.IROct 5, 2026

Reading the Mood: Emotion-Guided Book-to-Music Recommendation via CGANs and LLMs

Authors: Manousos Linardakis, Georgios Alexandridis

Organizations: School of Electrical & Computer Engineering National Technical University of Athens Zografou, 15780, Greece · Department of Digital Industry Technologies National & Kapodistrian University of Athens Evripos Complex, Psachna, 34400, Greece

Abstract

Background music that matches the mood of a text has been shown to make readers feel more immersed and improve their reading experience, motivating recommender systems that pair books with mood-matched music. In this direction, we present Sentiment Aware Generative Adversarial Network for Cross Domain Recommendation (SAGA-CDR), a two-phase cross-domain recommendation framework that personalizes music suggestions and emotionally aligns them with the book being read. In the first phase, transformer-based sentiment embeddings are constructed from user reviews and mapped across domains via a Conditional Generative Adversarial Network, whose mask-conditioned generator handles missing sentiment components and injects stochasticity for richer preference transfer. A compact rating neural network then fuses sentiment-specific interaction scores with a collaborative filtering prior to predict music ratings. In the second phase, large language models classify each book into a valence-arousal emotional quadrant, and candidate tracks are filtered to match that quadrant. Experiments on both the English Amazon and Chinese Douban datasets show that SAGA-CDR achieves the best rating prediction accuracy on Amazon (RMSE 0.98) and the lowest RMSE on Douban (0.91), with ranking performance competitive with the strongest sentiment-aware baseline, even in cross-lingual settings.

Figures & tables

Fig. 1: The architectural diagram of the 1 st phase of SAGA-CDR

Fig. 2: SAGA-CDR training curves (in the Amazon dataset) after hyperparameter selection.

Book	Music	Q	LLM reasoning (summary)	Rating
The Heroin Diaries (Nikki Sixx)	“Sing Sing Death House” – The Distillers	Q2	Addiction, despair, turmoil, and self-destruction.	4.8
Foley Is Good (Mick Foley)	“I Can’t Stop” – Al Green	Q1	Celebratory, energetic, positive, and triumphant mood.	4.75
Franny and Zooey (J.D. Salinger)	“Is This Desire?” – PJ Harvey	Q3	Introspective, somber, emotionally fragile, and existential.	4.7
The Seven Storey Mountain (Thomas Merton)	“Prelude to a Kiss” – Duke Ellington	Q4	Reflective spiritual journey, transformation, and inner peace.	4.6

TABLE I: Sample reading soundtrack recommendations. Q: emotional quadrant.

Method	RMSE $\downarrow$	$p$	MAE $\downarrow$	$p$
SAGA-CDR	0.9806 $\pm$ 0.0078	—	0.6097 $\pm$ 0.0145	—
CDR-SAFM	1.1204 $\pm$ 0.0018	0.0006 †	0.6674 $\pm$ 0.0069	0.0145 †
EMCDR	4.6179 $\pm$ 0.0129	0.0000 †	4.5115 $\pm$ 0.0129	0.0000 †
PTUPCDR	4.6742 $\pm$ 0.0084	0.0000 †	4.5011 $\pm$ 0.0147	0.0000 †

TABLE II: Amazon rating prediction (mean

\pm

std,

n{=}3

seeds)

\dagger

: significance against SAGA-CDR (

\alpha{=}0.05

) – Lower is better

$\mathbf{k}$	Method	MAP $\uparrow$	NDCG $\uparrow$	MRR $\uparrow$	HR $\uparrow$
1	SAGA-CDR	.8907 $\pm$ .0005	.8907 $\pm$ .0005	.8907 $\pm$ .0005	.8907 $\pm$ .0005
	SAFM	.8902 $\pm$ .0019 ( $p{=}.2955$ )	.8902 $\pm$ .0019 ( $p{=}.2955$ )	.8902 $\pm$ .0019 ( $p{=}.2955$ )	.8902 $\pm$ .0019 ( $p{=}.2955$ )
	EMCDR	.8859 $\pm$ .0007 ( $p{=}.0056^{\dagger}$ )	.8859 $\pm$ .0007 ( $p{=}.0056^{\dagger}$ )	.8859 $\pm$ .0007 ( $p{=}.0056^{\dagger}$ )	.8859 $\pm$ .0007 ( $p{=}.0056^{\dagger}$ )
	PTUPCDR	.8861 $\pm$ .0012 ( $p{=}.0075^{\dagger}$ )	.8861 $\pm$ .0012 ( $p{=}.0075^{\dagger}$ )	.8861 $\pm$ .0012 ( $p{=}.0075^{\dagger}$ )	.8861 $\pm$ .0012 ( $p{=}.0075^{\dagger}$ )
3	SAGA-CDR	.8946 $\pm$ .0011	.9026 $\pm$ .0014	.9075 $\pm$ .0012	.9254 $\pm$ .0020
	SAFM	.8947 $\pm$ .0019 ( $p{=}.5243$ )	.9027 $\pm$ .0020 ( $p{=}.5561$ )	.9071 $\pm$ .0020 ( $p{=}.2732$ )	.9255 $\pm$ .0021 ( $p{=}.7887$ )

TABLE III: Amazon ranking metrics (mean

\pm

std,

n{=}3

seeds).

\dagger

: significant. Higher is better.

		Pred. $\downarrow$		$k{=}1$ $\uparrow$				$k{=}3$ $\uparrow$				$k{=}5$ $\uparrow$				$k{=}10$ $\uparrow$
	Method	RMSE	MAE	MAP	NDCG	HR	MRR	MAP	NDCG	HR	MRR	MAP	NDCG	HR	MRR	MAP	NDCG	HR	MRR
pre-tr.	SAFM	0.9169	0.6925	0.9118	0.9118	0.9118	0.9118	0.8848	0.9085	0.9775	0.9433	0.8769	0.9098	0.9839	0.9447	0.8704	0.9121	0.9839	0.9447
	EMCDR	4.0819	3.9763	0.8341	0.8341	0.8341	0.8341	0.7963	0.8371	0.9637	0.8953	0.7850	0.8411	0.9798	0.8990	0.7790	0.8487	0.9833	0.8995
	PTUPCDR	4.0793	3.9735	0.8272	0.8272	0.8272	0.8272	0.7877	0.8306	0.9672	0.8924	0.7785	0.8357	0.9781	0.8948	0.7742	0.8444	0.9833	0.8956
no-pre.	SAFM	0.9151	0.6928	0.9084	0.9084	0.9084	0.9084	0.8866	0.9097	0.9781	0.9417	0.8775	0.9097	0.9833	0.9429	0.8720	0.9131	0.9839	0.9430
	EMCDR	3.5591	3.2499	0.8750	0.8750	0.8750	0.8750	0.8426	0.8758	0.9764	0.9236	0.8346	0.8792	0.9816	0.9248	0.8258	0.8820	0.9833	0.9251

TABLE IV: Douban results (5-fold CV averages). Bold: best; underline: second best.

	Rating $\downarrow$		Ranking $\uparrow$
Variant	RMSE	MAE	@1	MAP@10	NDCG@10	MRR@10
Full model	0.981	0.610	.8907	.9004	.9095	.9078
$-$ noise	0.973	0.636	.8857 †	.8979 †	.9075 †	.9049 †
$-$ mask	0.974	0.621	.8857 †	.8979 †	.9075 †	.9049 †
$-\,\mathcal{L}_{\mathrm{adv}}$	1.002	0.640 †	.8858 †	.8979 †	.9076 †	.9049 †
$-\,\mathcal{L}_{\mathrm{recon}}$	0.973	0.625	.8857 †	.8979 †	.9075 †	.9049 †

TABLE V: CGAN ablation on Amazon (mean over

n

=3 seeds). @1: MAP, NDCG and MRR.

\dagger

: significantly worse than full (

\alpha

=0.05).

Panel of Figure 8 — Fig. 4: SHAP explanations on Amazon dataset. Top: summary plot showing feature impact across all instances (red: high values, blue: low values). Bottom: force plot for record 200; red features increase the prediction, while blue (neu_score) decreases it.

Fig. 5: ICE plots for Amazon dataset (blue: individual curves; red: PDP average).

Explore similar work

Mood-Aware Music Recommendation: Integrating User Affective Signals into Ranking Systems
Jun 11, 2026Terence Zeng, Abhishek K. UmrawalLearning to Rank Recommender Systems
University of Illinois Urbana-Champaign, Urbana, Illinois 61801, USA
Aligning MusicLLM with Emotion using Instruction Tuning and Feedback-Driven Alignment
Jun 23, 2026Takuya Hasumi, Welly NaptaliLLM Alignment AI Alignment
LY Corporation
Multimodal Music Recommendation System using LLMs
May 28, 2026Srikar Prabhas Kandagatla, Sreehitha R. Narayana, Chandana Magapu +6Sequential Recommendation Recommender Systems
University of Massachusetts Amherst, Massachusetts, USA · Dolby Laboratories Atlanta, Georgia, USA · Adobe Research San Jose, California, USA +2

Jun 11, 2026, cs.IRJ/K move · Enter open · S save

Mood-Aware Music Recommendation: Integrating User Affective Signals into Ranking Systems

Terence Zeng, Abhishek K. Umrawal

University of Illinois Urbana-Champaign, Urbana, Illinois 61801, USA

Learning to Rank Recommender Systems Affective Computing

Open paper AskPDF arXiv

Recommendation systems are essential in modern music streaming platforms due to the vast amount of available content. While collaborative filtering is widely used to suggest items based on the preferences of others with similar patterns, it performs poorly in domains where user-item interactions are sparse, such as music. Content-based filtering is an alternative approach that examines the qualities of the items themselves. Genre, instrumentation, and lyrics have been explored; however, relatively little attention has been given to emotion recognition. Since a user's emotional state strongly influences their music choice, incorporating mood signals offers a promising direction for personalization. In this work, we propose a mood-conditioned ranking framework that integrates user affective signals into the recommendation process via softmax-based sampling in the energy-valence space. We evaluate the approach via single-blind experiments in which participants compare recommendations from the proposed system against a baseline. The results indicate improved perceived recommendation quality, providing preliminary evidence for the effectiveness of incorporating mood-based inputs into music recommendations.

Abstract

Explore similar work

Mood-Aware Music Recommendation: Integrating User Affective Signals into Ranking Systems

Aligning MusicLLM with Emotion using Instruction Tuning and Feedback-Driven Alignment

Multimodal Music Recommendation System using LLMs