MixJudge: Learning a Language-Queryable Critic for Vocal Mixing

Kuan-Hsun Ho; Yuan-Chiao Cheng; Yen-Tung Yeh; Yi-Hsuan Yang

cs.SDOct 22, 2026

MixJudge: Learning a Language-Queryable Critic for Vocal Mixing

Authors: Kuan-Hsun Ho, Yuan-Chiao Cheng, Yen-Tung Yeh, Yi-Hsuan Yang

Organizations: Independent Researcher · Music Informatics Group, Georgia Institute of Technology · Music and AI Lab, National Taiwan University · AI-CoRE, National Taiwan University

Abstract

Mixing requires not only applying processing but also assessing the resulting mix. Audio-language models can provide conversa- tional mixing advice, but existing evaluations do not separately test whether they can judge specific mixing issues. Since professional supervision for such judgments is scarce, we construct a curriculum of 11 vocal-mix issues at multiple severity levels and use it to train MixJudge, a language-queryable audio critic. MixJudge shows its ability to retrieve mixing issues, distinguish opposite issues, and order the severity of an issue. Moreover, its judgments transfer to menu selection: given a set of processing candidates, MixJudge often favors versions acoustically closer to a held-out professional version, demonstrating how explicit mixing judgment can support comparison and downstream mixing decisions.

Figures & tables

Figure 1: Same-content training batch construction.

System	The critic’s exam						Menu selection
	Issue retrieval		Opposite issue	Severity ordering
	$T\!\rightarrow\!A$	$A\!\rightarrow\!T$	$A_{\rm opp}$	$\rho_{\rm sev}$	$A_{\rm chain}$	$S_{\rm sel}$	$C$	$\rho_{\rm menu}$	$R_{\rm med}$
Frozen CLAP	.050/.203	.066/.230	.266	.331	.442	.172	.330/.284	.146/.147	$-$ .552/ $-$ .636
Music Flamingo	.097/.255	.061/.212	.250	.235	.189	.119	.364/.318	.168/.056	$-$ .218/ $-$ .303
Audio Flamingo Next	.093/.237	.041/.191	.243	.175	.170	.129	.523/.386	.350/.082	.134/ $-$ .466
Qwen2.5-Omni	.142/.296	.057/.203	.374	.309	.239	.197	.398/.386	.290/.305	$-$ .060/ $-$ .211

Table 1: Results of the critic’s exam and menu selection. Retrieval entries report R@1/MRR.

A_{\rm opp}

is both-poles accuracy;

\rho_{\rm sev}

, severity correlation;

A_{\rm chain}

, full-order accuracy;

S_{\rm sel}

, selectivity;

C

, closer rate;

\rho_{\rm menu}

, whole-menu correlation; and

R_{\rm med}

, median recovery. Slash-separated menu entries correspond to mean/max evidence policies. Green marks the best result; red marks performance below chance.

	$T\!\rightarrow\!A$	$A_{\rm opp}$	$\rho_{\rm sev}$	$S_{\rm sel}$	$C$	$\rho_{\rm menu}$
Objective	The critic’s exam				Menu selection
InfoNCE	.144/.317	.812	.611	.495	.636/.682	.501/.650
$\mathcal{L}_{\rm sig}$	.119/.317	.725	.578	.356	.739/.773	.728/ .728
$+$ $\mathcal{L}_{\rm rank}$	.098/.259	.786	.916	.384	.705/.614	.680/.558
$+$ $\mathcal{L}_{\rm sel,opp}$	.164/.321	.839	.971	.838	.727/.716	.730 /.721

Table 2: Objective ablation.

Figure 2: The critic’s exam. Green marks correct responses; red marks errors.

Figure 3: Menu selection. Gray shows issue evidence; red shows each candidate’s

D

to professional version.

Audio-Language Models

Audio-language models connect audio and natural language for understanding, retrieval, dialogue, speech generation, and music or sound analysis.

See all 300 papers

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings
Oct 9, 2026Mateusz Modrzejewski, Tomasz RadzikowskiAudio-Language Model Evaluation LLM-as-a-Judge
Institute of Computer Science, Warsaw University of Technology
Agentic Music Post-Production
Oct 21, 2026Jaywon Koo, Junghyun Koo, Marco A. Martinez-Ramirez +5Audio Editing
Rice University · Sony Corporate Technology Center America, Inc. · Sony Group Corporation
Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing
Sep 24, 2026Ilpo Viertola, Giulio Cengarle, Gouthaman KV +2Audio Editing Audio Source Separation
Dolby Laboratories · Tampere University

Oct 9, 2026J/K move · Enter open · S save

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

Mateusz Modrzejewski, Tomasz Radzikowski

Institute of Computer Science, Warsaw University of Technology

Audio-Language Model Evaluation LLM-as-a-Judge Music Generation Evaluation

Open paper AskPDF arXiv

Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validated by how well its ratings correlate with expert ratings. The correlation does not show what the judge responds to: the music, the text prompt, or the way the question is asked. We propose four probes that test each of these, three of them read against a control computed from the human ratings: removing the audio, replacing the clip with another model's clip for the same prompt, swapping the order of two clips in pairwise judging, and rewording the instruction. We apply them to five open judges on all 2,748 clips of MusicEval, and the last two to the SongEval songs. Without audio, no judge exceeds what the prompt alone predicts from the human ratings, and for prompt alignment every judge falls below it. With audio, every judge follows the clip it hears when rating prompt alignment, two do when rating musical quality, and the strongest judge's quality score tracks the experts' alignment rating more than their quality rating. The pairwise verdicts of the four smaller judges follow the position of a clip in the pair (its slot) rather than its quality on both rating sets. For musical quality, rewording the instruction moves the correlation of most judges as much as changing the judge. MusicEval's prompt assignment bounds any prompt-only judge at the model level. We recommend reporting the four probes with every music judge.

Abstract

Audio-Language Models

Explore similar work

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

Agentic Music Post-Production

Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing