Understanding Machine Taste: Aesthetic Agreement Across Foundation Models in K-pop

Kaishen Wang; Yucheng Pan; Xuehao Cui; Hanling Tian; Heng Huang

cs.CLOct 13, 2026

Understanding Machine Taste: Aesthetic Agreement Across Foundation Models in K-pop

Authors: Kaishen Wang, Yucheng Pan, Xuehao Cui, Hanling Tian, Heng Huang

Organizations: University of Maryland, College Park

Abstract

As foundation models enter creative industries, understanding their aesthetic preferences becomes an important step toward AI-assisted creative decision making. In this work, we investigate how foundation models evaluate creative content and characterize their ``machine taste'' using K-pop songs and music videos as a testbed. We formulate aesthetic evaluation as a pairwise ranking problem and systematically examine pairwise inter-model agreement, group-level consensus, and cross-modal preference alignment. Our results reveal stronger pairwise inter-model agreement and group-level consensus in visual evaluation than in audio evaluation, while preferences across the two modalities remain weakly aligned. We further examine how visual consensus rankings relate to real-world popularity using historical YouTube viewership data, finding positive but varying correlations across artist groups. These findings provide initial insights into machine aesthetic preferences and lay the groundwork for future AI-assisted creative decision making that considers creative content, artist identity, audience preferences, and market dynamics.

Figures & tables

Figure 1: Overview of our study of machine taste in K-pop. ALMs and VLMs perform pairwise aesthetic evaluations of songs and music videos, respectively. The resulting rankings are analyzed to examine inter-model agreement, group-level consensus, and cross-modal preference alignment.

Group	# Audio	# Video
RED VELVET	30	28
TWICE	30	30
BLACKPINK	30	30
I-DLE	30	25
ITZY	30	26
AESPA	30	23

Table 1: Statistics of the K-pop evaluation dataset across 14 artist groups. The audio collection includes up to 30 songs per group, while the video collection contains the subset with eligible official music videos.

Group	Kendall’s $W$	$p$ -value
RED VELVET	0.334	$<0.001$
TWICE	0.240	0.043
BLACKPINK	0.319	$<0.001$
I-DLE	0.356	$<0.001$
ITZY	0.272	0.010
AESPA	0.263	0.015

Table 2: Group-level aesthetic consensus among ALMs. Kendall’s coefficient of concordance

W

is computed from the rankings of all six model configurations for each artist group in the audio-based evaluation.

Figure 2: Pairwise ranking agreement between ALMs. Each cell reports the mean Spearman rank correlation across 14 artist groups. Inst and Think denote Instruct and Thinking configurations, respectively; Qwen refers to Qwen3-Omni-30B-A3B.

Group	Kendall’s $W$	$p$ -value
RED VELVET	0.656	$<0.001$
TWICE	0.649	$<0.001$
BLACKPINK	0.769	$<0.001$
I-DLE	0.681	$<0.001$
ITZY	0.516	$<0.001$
AESPA	0.584	$<0.001$

Table 3: Group-level aesthetic consensus among VLMs. Kendall’s coefficient of concordance

W

is computed from the rankings of all 28 model configurations for each artist group in the music-video setting.

Figure 3: Pairwise ranking agreement between VLMs. Each cell reports the mean Spearman rank correlation across 14 artist groups in the music-video setting. Models are grouped by family, showing strong within-family agreement and substantial cross-family correlations.

Group	Spearman $\rho$	$p$ -value
Red Velvet	0.023	0.905
TWICE	0.475	0.007
BLACKPINK	0.178	0.344
I-DLE	0.377	0.064
ITZY	0.356	0.074
AESPA	0.006	0.977

Table 4: Cross-modal consensus alignment between audio and visual evaluations. Spearman rank correlation is computed between the audio and visual consensus rankings for each artist group, using only songs shared by both modalities.

Figure 4: Cross-modal ranking agreement between ALMs and VLMs. Each cell reports the mean Spearman rank correlation across 14 artist groups, computed using only songs shared by both evaluation sets.

Artist	# MVs	Spearman $\rho$	$p$ -value
AESPA	19/23	0.661	0.002
IVE	19/23	0.363	0.127
TWICE	26/30	0.406	0.040

Table 5: Association between VLM consensus rankings and early YouTube viewership. Historical view counts are collected approximately seven days after release (

\pm 2

days). We report within-group Spearman correlations and two-sided

p

-values.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	MOSS-4B-Inst	MOSS-4B-Think	MOSS-8B-Inst	MOSS-8B-Think	Qwen-Inst	Qwen-Think
MOSS-4B-Inst	–	0.018	0.091	0.176	0.106	-0.043
MOSS-4B-Think		–	0.276	0.425	0.320	-0.063
MOSS-8B-Inst			–	0.433	0.354	-0.029
MOSS-8B-Think				–	0.526	-0.144
Qwen-Inst					–	-0.086
Qwen-Think						–

Appendix

Table 6: Pairwise ranking agreement between ALMs. Each cell reports the mean Spearman rank correlation across 14 artist groups. Higher values indicate stronger agreement between model rankings.

Abbreviation	Model
G-N / G-T	GLM-4.6V-Flash, non-thinking / thinking
I1-N / I1-T	InternVL3.5-1B, non-thinking / thinking
I2-N / I2-T	InternVL3.5-2B, non-thinking / thinking
I4-N / I4-T	InternVL3.5-4B, non-thinking / thinking
I8-N / I8-T	InternVL3.5-8B, non-thinking / thinking
I14-N / I14-T	InternVL3.5-14B, non-thinking / thinking

Appendix

Table 7: Abbreviations used for VLMs in Table 14 .

Song	M4-I	M4-T	M8-I	M8-T	Q-I	Q-T	Consensus
Savage	10	2	2	1	6	16	1
Welcome To MY World	7	3	1	5	5	24	2
Drama	5	14	8	2	3	23	3
Count On Me	15	10	16	6	15	6	6
Spicy	9	25	10	2	2	24	8
Next Level	27	4	6	9	12	27	13

Appendix

Table 8: Song-level audio rankings of all 30 AESPA songs across six ALM configurations. M4, M8, and Q denote MOSS-Audio-4B, MOSS-Audio-8B, and Qwen3-Omni-30B-A3B, respectively; I and T indicate Instruct and Thinking. Consensus rankings are obtained by averaging the six model rankings and re-ranking the songs. Lower ranks indicate stronger preferences.

Song	Mean Rank	Rank SD	Consensus
HEYA	2.54	1.82	1
I AM	4.50	2.79	2
ELEVEN	4.55	3.62	3
BLACKHOLE	5.39	4.49	4
LOVE DIVE	5.71	2.82	5
BANG BANG	7.46	4.02	6

Appendix

Table 9: Selected IVE songs illustrating visual consensus among 28 VLM configurations over 23 music videos. Mean Rank and Rank SD denote the mean and sample standard deviation of model rankings, respectively. Consensus is obtained by re-ranking the mean ranks. Lower ranks indicate stronger preferences, while smaller SD values indicate greater agreement.

Song	Audio Rank	Visual Rank
Fancy	2	2
Feel Special	5	5
Celebrate	3	6
Heart Shaker	17	17
TT	25	24
Cheer Up	30	29

Appendix

Table 10: Selected TWICE songs illustrating cross-modal preference alignment. Audio and visual consensus rankings are obtained from six ALM and 28 VLM configurations, respectively, using the same 30 songs with official music videos. Lower ranks indicate stronger preferences.

Setting	AESPA	IVE	TWICE
Day 7 ( $\pm 2$ days)	0.661 (19)	0.363 (19)	0.406 (26)
Day 7 ( $\pm 1$ day)	0.704 (15)	0.354 (14)	0.423 (22)
Day 30 ( $\pm 3$ days)	0.782 (11)	0.423 (13)	0.367 (22)

Appendix

Table 11: Robustness analysis of the association between VLM consensus rankings and historical YouTube viewership across different observation windows. Each cell reports Spearman’s

\rho

, with the number of available music videos indicated in parentheses.

Group	Songs
AESPA	Lemonade; Switchblade (Feat. Ty Dolla $ign); Supernova; Kiss N Tell; WDA (Whole Different Animal); Whiplash; Dirty Work; Drama; Live My Life; Armageddon; Rich Man; Spicy; Hot Mess; Better Things; Welcome To My World (Feat. Nævis); Life’s Too Short (English Ver.); Savage; Next Level; Dreams Come True; Girls; Black Mamba; Forever; Hold On Tight; Count On Me; ’Til We Die; Aenergy; Yeppi Yeppi; Can’t Help Myself; Just Another Girl; Prologue.
IVE	Eleven; Love Dive; After Like; Kitsch; I Am; Wave; I Want; Either Way; Off The Record; Baddie; All Night (Feat. Saweetie); Heya; Accendio; Summer Festa; Crush; Rebel Heart; Attitude; XOXZ; Bang Bang; Blackhole; Supernova Love; Lucid Dream; Be Alright; Take It; Royal; Blue Blood; Holy Moly; OTT; Payback; Ice Queen.
LE SSERAFIM	Hot; Crazy; Made My Night; Fearless; Perfect Night; Unforgiven; Smart; Antifragile; Boompala; Impurities; Celebration; Different; Easy; Come Over; Spaghetti; Eve, Psyche & The Bluebeard’s Wife; Kawaii; Sour Grapes; Blue Flame; The Great Mermaid; No Celestial; Good Parts (When The Quality Is Bad But I Am); Fire In The Belly; 1-800-Hot-N-Fun; Pierrot; Crazier; Ash; So Cynical (Badum); Iffy Iffy; Trust Exercise.
NEWJEANS	Attention; Hype Boy; Hurt; Cookie; Ditto; OMG; Zero; New Jeans; Super Shy; Cool With You (Side A); ETA; Bubble Gum; How Sweet; Right Now; Supernatural; ASAP; Get Up; Gods; Beautiful Restriction; Our Night Is More Beautiful Than Your Day.
I-DLE	Nxde; Tomboy; Mono; I Want That; Queencard; Oh My God; Gimme Dat Love; Good Thing; Girlfriend; Crow; Lion; Klaxon; Super Lady; Hwaa; Wife; Latata; Senorita; Uh-Oh; Hann (Alone); Allergy; Dumdi Dumdi; Revenge; I Do; Last Dance; Blow Your Mind; My Bag; Fate; I’m The Trend; Bloom; Nightmare.
TWICE	This Is For; Strategy; What Is Love?; The Feels; Fancy; Feel Special; Talk That Talk; I Got You; Set Me Free; One Spark; I Can’t Stop Me; Alcohol-Free; Me+You; More & More; Scientist; TT; Knock Knock; Yes Or Yes; Dance The Night Away; Likey; Cheer Up; Like Ooh-Ahh; Heart Shaker; Signal; Doughnut; Perfect World; Kura Kura; Celebrate; Hare Hare; Moonlight Sunrise.

Appendix

Table 12: Complete list of audio tracks included in the audio evaluation (373 items across 14 K-pop groups).

Group	Songs
AESPA	Lemonade; Switchblade (Feat. Ty Dolla $ign); Supernova; Kiss N Tell; WDA (Whole Different Animal); Whiplash; Dirty Work; Drama; Live My Life; Armageddon; Rich Man; Spicy; Hot Mess; Better Things; Welcome To My World (Feat. Nævis); Life’s Too Short (English Ver.); Savage; Next Level; Dreams Come True; Girls; Black Mamba; Forever; Hold On Tight.
IVE	Eleven; Love Dive; After Like; Kitsch; I Am; Wave; I Want; Either Way; Off The Record; Baddie; All Night (Feat. Saweetie); Heya; Accendio; Summer Festa; Crush; Rebel Heart; Attitude; XOXZ; Bang Bang; Blackhole; Supernova Love; Lucid Dream; Be Alright.
LE SSERAFIM	Hot; Crazy; Made My Night; Fearless; Perfect Night; Unforgiven; Smart; Antifragile; Boompala; Impurities; Celebration; Different; Easy; Come Over; Spaghetti; Eve, Psyche & The Bluebeard’s Wife; Kawaii.
NEWJEANS	Attention; Hype Boy; Hurt; Cookie; Ditto; OMG; Zero; New Jeans; Super Shy; Cool With You (Side A); ETA; Bubble Gum; How Sweet; Right Now; Supernatural; ASAP.
I-DLE	Nxde; Tomboy; Mono; I Want That; Queencard; Oh My God; Gimme Dat Love; Good Thing; Girlfriend; Crow; Lion; Klaxon; Super Lady; Hwaa; Wife; Latata; Senorita; Uh-Oh; Hann (Alone); Allergy; Dumdi Dumdi; Revenge; I Do; Last Dance; Blow Your Mind.
TWICE	This Is For; Strategy; What Is Love?; The Feels; Fancy; Feel Special; Talk That Talk; I Got You; Set Me Free; One Spark; I Can’t Stop Me; Alcohol-Free; Me+You; More & More; Scientist; TT; Knock Knock; Yes Or Yes; Dance The Night Away; Likey; Cheer Up; Like Ooh-Ahh; Heart Shaker; Signal; Doughnut; Perfect World; Kura Kura; Celebrate; Hare Hare; Moonlight Sunrise.

Appendix

Table 13: Complete list of music videos included in the mv evaluation (278 items across 14 K-pop groups).

Preference Alignment

Preference alignment studies how AI systems are trained or steered to produce outputs that match human or user priorities.

See all 109 papers

APEX: Large-scale Multi-task Aesthetic-Informed Popularity Prediction for AI-Generated Music
May 5, 2026Jaavid Aktar Husain, Dorien HerremansMulti-Task Learning Music Information Retrieval
AMAAI Lab, Singapore University of Technology and Design
Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
May 12, 2026Yichen Feng, Yuetai Li, Chunjiang Liu +14Multimodal Large Language Models Image Quality Assessment
TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design
May 20, 2026Haonan Zhu, Elad Hirsch, Alexandria Minetti +3VLM Evaluation T2I Generation Evaluation
Lica World, San Francisco, USA. · Contra.Work Inc., Brooklyn, NY, USA.

May 5, 2026J/K move · Enter open · S save

APEX: Large-scale Multi-task Aesthetic-Informed Popularity Prediction for AI-Generated Music

Jaavid Aktar Husain, Dorien Herremans

AMAAI Lab, Singapore University of Technology and Design

Multi-Task Learning Music Information Retrieval

Open paper AskPDF arXiv

Music popularity prediction has attracted growing research interest, with relevance to artists, platforms, and recommendation systems. However, the explosive rise of AI-generated music platforms has created an entirely new and largely unexplored landscape, where a surge of songs is produced and consumed daily without the traditional markers of artist reputation or label backing. Key, yet unexplored in this pursuit is aesthetic quality. We propose APEX, the first large-scale multi-task learning framework for AI-generated music, trained on over 211k songs (10k hours of audio) from Suno and Udio, that jointly predicts engagement-based popularity signals - streams and likes scores - alongside five perceptual aesthetic quality dimensions from frozen audio embeddings extracted from MERT, a self-supervised music understanding model. Aesthetic quality and popularity capture complementary aspects of music that together prove valuable: in an out-of-distribution evaluation on the Music Arena dataset, comprising pairwise human preference battles across eleven generative music systems unseen during training, including aesthetic features consistently improves preference prediction, demonstrating strong generalisation of the learned representations across generative architectures.

Abstract

Preference Alignment

Explore similar work

APEX: Large-scale Multi-task Aesthetic-Informed Popularity Prediction for AI-Generated Music

Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?

TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design