DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialogues

Joonhyeok Shin; Jaehoon Kang; Yujun Lee; Hannah Lee; Yejin Lee; Yoonji Park; Kyuhong Shim

cs.AIApr 9, 2026

DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialogues

Authors: Joonhyeok Shin, Jaehoon Kang, Yujun Lee, Hannah Lee, Yejin Lee, Yoonji Park, Kyuhong Shim

Organizations: Sungkyunkwan University, Republic of Korea

Abstract

Selecting an appropriate background music (BGM) that supports natural human conversation is a common production step in media and interactive systems. In this paper, we introduce dialogue-conditioned BGM recommendation, where a model should select non-intrusive, fitting music for a multi-turn conversation that often contains no music descriptors. To study this novel problem, we present DialBGM, a benchmark of 1,200 open-domain daily dialogues, each paired with four candidate music clips and annotated with human preference rankings. Rankings are determined by background suitability criteria, including contextual relevance, non-intrusiveness, and consistency. We evaluate a wide range of open-source and proprietary models, including audio-language models and multimodal LLMs, and show that current models fall far short of human judgments; no model exceeds 35% Hit@1 when selecting the top-ranked clip. DialBGM provides a standardized benchmark for developing discourse-aware methods for BGM selection and for evaluating both retrieval-based and generative models.

Figures & tables

Figure 1: Dialogue-conditioned BGM recommendation . Given a multi-turn dialogue and a large-scale music clip database, the system uses the dialogue as a contextual filter to rank candidates and selects the one that best matches the dialogue as background music (BGM).

Figure 2: DialBGM examples. Each dataset instance consists of a multi-turn dialogue paired with four candidate background music (BGM) clips, along with human preference rankings indicating which clip best matches the conversational atmosphere. Each music clip is presented with its corresponding caption and human-annotated rank.

Figure 3: Dataset construction. The DialBGM dataset is constructed through a four-stage pipeline, consisting of (1) source data collection and rule-based BGM suitability filtering, (2) dialogue caption generation via an LLM, (3) embedding score-based candidate selection, and (4) expert human ranking annotation with quality control.

Metric	Formula
Hit@1	$\mathbb{I}(\text{top}(R)=\text{top}(G))$
MRR	$\frac{1}{N}\sum_{i=1}^{N}\frac{1}{\text{rank}_{i}}$
nDCG	$\frac{\text{DCG}}{\text{IDCG}},\quad\text{DCG}=\sum_{i=1}^{4}\frac{rel_{i}}{\log_{2}(i+1)}$
Kendall’s $\tau_{b}$	$\frac{P-Q}{\sqrt{(P+Q+T_{x})(P+Q+T_{y})}}$

Table 1: Evaluation metrics for DialBGM benchmark. We employ tie-aware variants to handle equal scores.

N

denotes the number of samples. For Kendall’s

\tau_{b}

P

and

Q

denote concordant and discordant pairs, and

T_{x}

T_{y}

denote ties.

Dialogue Statistics
Total Dialogues	1,200
Avg. Turns per Dialogue	7.85
Avg. Words per Dialogue	85.98
Audio Statistics
Unique Audio Tracks	1,020
Avg. Duration (sec)	10.0

Table 2: Statistics of the DialBGM.

Figure 4: Tested input-model-output configurations. Overview of the experimental settings (Tables 3–6), illustrating the input compositions (audio track or audio caption, full dialogue or GPT-4o dialogue summary, and optional prompts), the model families (retrieval models, multimodal LLMs, and audio-language LLMs), and the corresponding outputs (embedding-based similarity or scalar scores for each candidate).

		Full Dialogue				One-line Summary
Model Category	Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Multimodal LLMs	Gemini 2.5 Pro	0.3233	0.5851	0.7904	0.1745	0.3474	0.5958	0.7965	0.1808
	Gemini 2.5 Flash	0.3293	0.5872	0.7905	0.1657	0.3228	0.5842	0.7912	0.1765
	GPT-4o-audio †	0.3051	0.5629	0.7822	0.1515	0.3192	0.5732	0.7856	0.1625
	Qwen2.5-Omni	0.3137	0.5742	0.7853	0.1739	0.3153	0.5740	0.7830	0.1674
	Phi-4-Multimodal	0.2506	0.5206	0.7490	-0.0092	0.2622	0.5311	0.7550	0.0241

Table 3: Performance of baseline models on the DialBGM Ranking task. † Accessed via the gpt-audio API.

Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Gemini 2.5 Pro	0.3491	0.5978	0.8000	0.2003
GPT-5-mini	0.3474	0.5945	0.7991	0.2016

Table 4: Text-only performance using full multi-turn dialogue as input.

Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Gemini 2.5 Pro	0.3422	0.5954	0.8003	0.2106
GPT-5-mini	0.3321	0.5866	0.7968	0.1991

Table 5: Text-only performance using the generated summary caption as input.

Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Gemini 2.5 Pro	0.3204	0.5848	0.7924	0.1783
GPT-4o-audio	0.3058	0.5695	0.7849	0.1690

Table 6: Joint reranking results where all four audio candidates are provided simultaneously.

Figure 5: Pairwise Kendall’s $\tau_{b}$ correlation matrix of ranking predictions from different models. Brighter boxes indicate the higher agreement in ranking.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Gemini 2.5 Pro	0.3712	0.6193	0.8103	0.2544
GPT-4o-audio	0.3142	0.5695	0.7837	0.1578

Appendix

Table 7: Results without Chain-of-Thought prompting.

Model	Hit@1	MRR	nDCG	Kendall’s $\tau_{b}$
Gemini 2.5 Pro	0.3443	0.5976	0.7941	0.1758
GPT-4o-audio	0.2949	0.5553	0.7769	0.1365

Appendix

Table 8: Results with few-shot demonstrations and explicit guidelines.

Figure 6: Screenshot of the Gradio-based data collection interface. The tool displays the dialogue and allows annotators to rank four BGM candidates (Candidates A–D) before proceeding to the next turn.

Panel of Figure 7 — Figure 7: The full system prompt used for LLM-based BGM evaluation. The prompt incorporates a weighted scoring mechanism ( $0.2\times\text{Technicality}+0.5\times\text{Fit}+0.3\times\text{Pacing}$ ) and explicit penalty rules for vocal interference.

Music Recommendation

See all 15 papers

Overview and Analysis of the RecSys Challenge 2026: Conversational Music Recommendation
Sep 27, 2026Seungheon Doh, Sergio Oramas, Bruno Sguerra +3Recommender Systems Music Recommendation
Sony Group Corporation Japan · SiriusXM United States · Deezer Research France +3
Reddit2Deezer: A Scalable Dataset for Real-World Grounded Conversational Music Recommendation
May 9, 2026Haven Kim, Julian McAuleyRecommender Systems Music Recommendation
University of California San Diego
ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval
May 5, 2026Honglei Zhang, Yuting Chen, Chenpeng Hu +2Audio Reasoning Audio Understanding
School of Software, Nanjing University Nanjing, China · School of Software, Northwestern Polytechnical University Xi’an, China · College of Computing and Data Science, Nanyang Technological University Singapore, Singapore

Sep 27, 2026J/K move · Enter open · S save

Overview and Analysis of the RecSys Challenge 2026: Conversational Music Recommendation

Seungheon Doh, Sergio Oramas, Bruno Sguerra, Abhinav Bohra+2

Sony Group Corporation Japan · SiriusXM United States · Deezer Research France+3

Recommender Systems Music Recommendation

Open paper AskPDF arXiv

The RecSys Challenge 2026 studies conversational music recommendation as a joint item recommendation and response generation problem: given a multi-turn dialogue, systems must retrieve relevant tracks from a large catalog and produce a grounded natural-language response. This paper presents the challenge task, dataset, evaluation protocol, and official results. Beyond the leaderboard, we analyze the 16 accepted systems through a common retrieve--rerank--generate framework and examine how recommendation performance varies across users, requests, and dialogue contexts. Strong systems commonly combine heterogeneous candidate sources and preserve source-specific evidence for learned reranking. Across the system papers and our organizer-side analysis, robust design also means 1) grounding cold-start retrieval in multi-turn conversation and item signals, 2) using intent detectors, and 3) modeling the full multi-turn context rather than the current query alone. We further identify limitations of the benchmark and evaluation protocol, including single-ground-truth relevance and teacher-forced evaluation of synthetic dialogues. Together, these findings provide practical guidance for future conversational recommender systems and shared evaluation efforts.

Abstract

Music Recommendation

Explore similar work

Overview and Analysis of the RecSys Challenge 2026: Conversational Music Recommendation

Reddit2Deezer: A Scalable Dataset for Real-World Grounded Conversational Music Recommendation

ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval