MQAScore: Fine-Grained Text-Music Alignment Evaluation via Large Audio Language Model

Wei-Jaw Lee; Chun-Ping Wang; Xuanjun Chen; Yi-Hsuan Yang

cs.SDOct 22, 2026

MQAScore: Fine-Grained Text-Music Alignment Evaluation via Large Audio Language Model

Authors: Wei-Jaw Lee, Chun-Ping Wang, Xuanjun Chen, Yi-Hsuan Yang

Organizations: GICE, National Taiwan University · AI-CoRE, National Taiwan University · Taiwan AILabs

Abstract

Text-to-music alignment is commonly evaluated by the global similarity between text and audio embeddings, which collapses all described attributes into a single score and obscures which ones are missing. We instead introduce MQAScore, which verifies each atomic musical concept along five clip-level dimensions (genre, instrument, mood/theme, vocal, and usage) by the probability that a large audio language model (LALM) answers affirmatively to a binary question. To assess evaluator reliability, we construct verification labels from presence/absence annotations and LLM-guided concept substitutions, and propose the Sensitivity-Invariance Ratio, which measures how selectively an evaluator responds to meaning-changing substitutions. Experiments show that concept-level scoring improves both question-answering and similarity-based evaluators, and MQAScore can be a more reliable and interpretable fine-grained complement to global similarity for evaluating text-to-music alignment.

Figures & tables

Figure 1: Global similarity (e.g. CLAP, MuQ-MuLan) yields a single score; MQAScore verifies each musical dimension separately.

Input	Method	Model	Genre		Instrument		Mood/Theme		Vocal		Usage
Input	Method	Model	AUC	WR	AUC	WR	AUC	$R_{S/I}$	AUC	$R_{S/I}$	AUC	$R_{S/I}$
Caption	AQAScore	Qwen3-Omni	–	–	–	–	0.776	31.3	0.870	31.0	0.718	36.9
		AF-Next	–	–	–	–	0.739	20.6	0.727	9.4	0.685	6.9
		Music Flamingo	–	–	–	–	0.684	10.0	0.711	6.8	0.610	5.1
	Similarity	CLAP	–	–	–	–	0.535	1.94	0.629	2.63	0.559	1.47
	Similarity	MuQ-MuLan	–	–	–	–	0.532	2.34	0.500	0.38	0.528	1.55

Table 1: Concept verification results: AUC with WR for genre and instrument, and with

R_{S/I}

for mood/theme, vocal, and usage. Caption-level scores are unavailable for tag-only genre and instrument data.

	H	Q3-O	AF-N	MF	CLAP	MuQ
Genre	92.5	85.0	86.7	75.0	71.7	71.7
Instrument	82.3	79.7	72.9	89.8	74.6	62.7
Mood/Theme	84.6	86.4	88.6	88.6	54.5	43.2
Vocal	95.2	92.5	92.5	85.0	80.0	60.0
Usage	82.5	85.4	78.0	68.3	63.4	61.0

Table 2: Agreement (%) with expert labels on MusicCaps pairs. H: human-human agreement; Q3-O: Qwen3-Omni; AF-N: Audio Flamingo Next; MF: Music Flamingo; MuQ: MuQ-MuLan. All evaluators use concept-level scoring.

Audio-Text Alignment

See all 21 papers

MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent
Oct 7, 2026Zekai Liu, Zhilin Wang, Xuzheng He +2Text-to-Music Generation Music Generation Evaluation
Shandong University · University of Science and Technology of China · Central Conservatory of Music +2
HumMusQA: A Human-written Music Understanding QA Benchmark Dataset
Mar 29, 2026Benno Weck, Pablo Puentes, Andrea Poltronieri +2Music Understanding Audio-Language Model Evaluation
Universitat Pompeu Fabra · Universitat Autònoma de Barcelona
DeRA-MOS: Optimizing Text-to-Music Evaluation via Decoupled Listwise Ranking and Modality Alignment
Jun 8, 2026Chien-Chun Wang, Hung-Shin Lee, Hsin-Min Wang +1Text-to-Music Generation Music Generation Evaluation
E.SUN Financial Holding Co., Ltd., Taipei 10546, Taiwan · United Link Co., Ltd., Taipei 11493, Taiwan · Institute of Information Science, Academia Sinica, Taipei 11529, Taiwan +1

Oct 7, 2026J/K move · Enter open · S save

MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent

Zekai Liu, Zhilin Wang, Xuzheng He, Yu Cheng+1

Shandong University · University of Science and Technology of China · Central Conservatory of Music+2

Text-to-Music Generation Music Generation Evaluation Prompt Optimization

Open paper AskPDF arXiv

Text-to-music systems produce increasingly convincing audio, yet evaluation reveals little about whether the result matches user intent. A global text-audio relevance score can overlook the implicit intent in underspecified prompts and mask failures in specific requirements, such as instrumentation, structure, rhythm, or mood progression. To bridge this gap, we formulate text-to-music intent alignment as satisfying a per-request rubric of independently verifiable items covering both a request's explicit requirements and its implied musical intent. Scoring items individually makes evaluation diagnostic by intent source and musical dimension, rather than a single opaque score. We instantiate this as MuRA-Bench, a benchmark of real-world platform requests curated by music experts. We further propose MIRA (Musical Intent Refinement Agent), a test-time agent that first grounds a request's intent into rubrics, then searches over prompt revisions for a black-box generator under a bounded budget, iteratively generating music, verifying it against the rubrics, and using this feedback to guide a trajectory-aware tree search. Experiments across open-source and commercial backends show that MIRA improves intent alignment, enabling an open-source generator to achieve performance comparable to representative commercial systems (e.g. Suno and Mureka). Project page: https://mirareview.github.io/.

Abstract

Audio-Text Alignment

Explore similar work

MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent

HumMusQA: A Human-written Music Understanding QA Benchmark Dataset

DeRA-MOS: Optimizing Text-to-Music Evaluation via Decoupled Listwise Ranking and Modality Alignment