TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description

Mingyang Yao; Jingyue Huang; Ke Chen; Taylor Berg-Kirkpatrick; Shlomo Dubnov

cs.SDOct 11, 2026

TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description

Authors: Mingyang Yao, Jingyue Huang, Ke Chen, Taylor Berg-Kirkpatrick, Shlomo Dubnov

Organizations: University of California, San Diego · University of Rochester

Abstract

Text-conditioned symbolic music generation has attracted growing interest with the development of generative models. Yet two questions remain underexplored: what aspects of music the conditioning text should describe, and how the corresponding data should be collected and trained to improve controllability and text-music alignment. We address these challenges with TASTE, a taxonomy-aware framework for symbolic music generation with a main focus on piano pieces. First, we introduce a taxonomy of musical elements that specifies how a piece can be captured through textual description in a musically grounded way. The taxonomy comprises a set of metadata tags (genre, emotion, section plan, etc.) together with a free-form description of music imagery. This structure decomposes textual input to match the generation goal more coherently and precisely. Second, we build a data pipeline for collecting, annotating, and retrieving public symbolic piano resources, yielding a corpus of 432K MIDI files labeled at multiple levels of granularity. Finally, we design a staged training procedure that leverages this multi-granularity data across two model families: an autoregressive model and a non-autoregressive diffusion model. Both objective and subjective evaluations show that our models achieve superior musical-element control, text-description alignment, and overall music quality. We will release the full pipeline, all annotated datasets, and the model checkpoints.

Figures & tables

Figure 1: The TASTE taxonomy. Sources of musical intention (left) map to three condition channels (middle), each with its own evaluation target (right).

Figure 2: Progressive annotation across three tiers. Forward arrows gain scale at the cost of label reliability, while the dotted backward arrow is the restricted consolidation pass of Section 4.3 .

Table 3

Figure 3: TASTE model families. (A) AR reads key, tempo, and the section plan as a token prefix. (B) DiT receives the section plan as per-position embeddings. In both, summed tag embeddings drive adaptive normalization, and imagery enters via cross-attention to a frozen text encoder.

Model	Key	Tempo	Emotion	Genre	Accomp.	Perf.	CLaMP 3	CLAP
Reference	1.000	0.820	0.840	0.830	0.550	0.530	0.132	0.338
MuseCoco	$0.582^{m}$	0.636	$0.575^{q}$	0.386	–	–	–	–
text2midi	0.181	0.456	0.276	0.380	0.205	0.270	0.054	0.273
MIDI-LLM	0.340	0.713	0.285	0.370	0.201	0.226	0.059	0.292
EMO-Disentangler	$0.752^{\dagger}$	–	$\mathbf{0.611^{q}}$	–	–	–	–	–
Piano Transformer	–	–	–	0.458	–	–	–	–

Table 3: Tag agreement and full-prompt alignment (

\uparrow

). Reference: the 100 held-out high-quality MIDIs, excluded from ranking.

m

: mode-only key agreement.

q

: emotion agreement over four quadrants.

\dagger

: agreement with the key declared by the model. Bold and underline mark the best and second best.

System	Boundary F1	Contrast (random)	Pairwise F1
Reference	0.896	2.52 (1.72)	0.769
AR	0.517	1.99 (1.69)	0.608
DiT	0.868	3.18 (1.78)	0.775

Table 4: Temporal structure (

\uparrow

)

Model	MAESTRO V $\mathbf{\downarrow}$	MAESTRO C $\downarrow$	POP909 V $\downarrow$	POP909 C $\downarrow$	PQ $\uparrow$	CE $\uparrow$
Reference	0.497	193.427	0.662	239.368	8.007	7.333
MuseCoco	1.045	372.282	0.964	280.778	7.709	6.879
text2midi	1.067	296.012	1.113	277.089	7.789	6.953
MIDI-LLM	1.425	282.214	2.025	250.544	7.586	6.840
EMO-Disentangler	0.979	356.460	0.560	250.884	7.944	7.448
Piano Transformer	0.745	196.619	1.542	247.119	7.761	7.180

Table 5: Objective music quality. V/C: Fréchet distance in VGGish/CLaMP 3 space. Bold and underline mark the best and second best, excluding the reference row.

System	Quality (1–5) $\uparrow$	Imagery accuracy (%) $\uparrow$
Reference	3.85 $\pm$ 0.85	69.7
text2midi	2.71 $\pm$ 0.91	54.5
MIDI-LLM	2.82 $\pm$ 1.08	51.5
TASTE (AR)	3.26 $\pm$ 0.79	80.3
TASTE (DiT)	2.73 $\pm$ 1.00	75.8

Table 6: Listening study.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Classes and operational definition
Key	24 classes, 12 major and 12 minor tonics: one global estimate per piece.
Tempo	3 classes from the estimated bpm: slow $<75$ , moderate $\in[75,115)$ , fast $\geq 115$ .
Emotion	happy brightness, playfulness, or triumph. sad sorrow, longing, or grief. calm serenity or reflection without persistent distress. angry confrontation or defiance. anxious tension, restlessness, or foreboding.
Genre	classical Baroque through modern and neoclassical. jazz ragtime, stride, swing, blues, jazz ballads. pop pop-song covers, pop ballads, New Age and easy-listening styles. Hybrids take the strongest overall identity.
Accompaniment	chordal block or voiced chords. ostinato a repeated driving cell. arpeggiated broken or rolling chords. sustained held tones, pedal points, or sparse long harmony. contrapuntal independently moving voices.
Performance	Two binary axes crossed into four classes: timing steady (strict, even, motoric) versus flexible (rubato, elastic, breathing), and articulation connected (legato, singing, smooth) versus detached (staccato, crisp, percussive).

Appendix

Table 7: Tag vocabularies and annotation scope.

Source	Pieces
Aria-MIDI named subset ( Bradshaw and Colton, 2025 )	32,522
PIAST-YT ( Bang et al., 2024 )	8,669
Pop2Piano cover index ( Choi and Lee, 2023 )	5,866
PiJAMA ( Edwards et al., 2023 )	2,777
POP909 ( Wang et al., 2020 )	909
EMOPIA-derived pieces ( Hung et al., 2021 )	387

Appendix

Table 8: Source composition of the curated tier.

	Emotion	Genre	Accomp.	Timing	Articulation	Performance
Unpruned	0.5353	0.9086	0.4445	0.6527	0.7256	0.5119
Pruned	0.5629	0.9130	0.4621	0.6523	0.7405	0.5027
Kept	55.3%	93.1%	50.8%	65.4%	81.0%	–

Appendix

Table 9: Tag-probe macro-F1 before and after pruning. Kept: fraction of training labels retained.

Field	$\theta^{\mathrm{prec}}$	$\theta^{\mathrm{floor}}$	Applied	Val. prec.	Coverage	Binding rule
Emotion	0.7319	0.8807	0.7319	0.600	86.6%	precision
Genre	0.3968	1.0000	0.3968	0.936	100.0%	precision
Accompaniment	0.9873	0.8097	0.8097	0.538	75.0%	coverage floor
Performance	0.9319	0.9083	0.9083	0.588	75.0%	coverage floor

Appendix

Table 10: Confidence thresholds, validation precision, and large-tier coverage. Genre coverage includes source metadata.

Figure 4: Tokenization example. The prefix holds key, tempo, and the section plan, where Sec_A, Plan_1 means that section A lasts one five-second window. <T> anchors each window and is followed by the active section identity, and <S> and <E> mark the start and end of the performance.

Parameter	Value
Layers / hidden width / context	26 / 1,536 / 4,096
Query heads / key–value heads / head width	24 / 6 / 64
Feed-forward width (SwiGLU) / embedding table	6,144 / 2,176
Normalization / rotary base / embeddings	RMSNorm $+$ QK-norm / 100,000 / tied
Backbone parameters	892,881,664
Tag tables (emotion, genre, accomp., perf.)	$(6,4,6,5)\times 1{,}536$

Appendix

Table 11: Autoregressive architecture. Parameter counts exclude the frozen text encoder.

	Pretrain	Large public	Curated	High-quality
Distinct pieces per pass	816,682	369,553	50,130	9,849
Passes / training rows	5 / 4,083,410	6 / 2,217,318	3 / 150,390	5 / 49,245
Validation rows	1,518	1,500	1,000	100
Training tokens	11.44B	6.60B	429.72M	155.97M
Optimizer updates	43,708	12,616	1,644	607
Peak learning rate	$2\!\times\!10^{-4}$	$1.5\!\times\!10^{-4}$	$5\!\times\!10^{-5}$	$2.5\!\times\!10^{-5}$

Appendix

Table 12: Autoregressive training configuration by stage.

Parameter	Value
Hidden width / layers / heads	1,024 / 16 / 16
Feed-forward ratio (GELU) / latent width	$4\times$ / 128
Canvas positions / conditioning-code width	384 / 512
AdaLN-Zero head output width per block	$9\times 1{,}024$
Cross-attention context width	2,560
Normalization	LayerNorm, no affine; QK-RMSNorm

Appendix

Table 13: Latent diffusion architecture with a 384-position canvas.

	Pretrain	Large	Curated		High-
	Pretrain	public	(a)	(b)	quality
Steps	120,000	120,000	24,000	36,000	9,000
Peak learning rate ( $\times 10^{-5}$ )	10	10	5	3	2
Warm-up steps	3,000	2,000	500	500	300
Timestep mean $\mu$	$-0.5$	$-0.25$	$-0.25$	$-0.25$	$-0.25$
Passes visited	2	3	3	3	3

Appendix

Table 14: Diffusion training configuration. Curated (b) continues (a) with a restarted schedule. Pretraining rows per pass are listed as clean / augmented.

System	Received conditions and adaptation
MuseCoco ( Lu et al., 2023 )	Attribute-to-music stage: piano, emotion quadrant, genre, key mode, tempo class, 4/4 meter, and a length request over 60 s. Native top- $k$ 15, temperature 1.0, and 512–2,560 tokens. Its tempo cutoffs (76/120 bpm) differ from ours (75/115).
text2midi ( Bhandari et al., 2025 ) , MIDI-LLM ( Wu et al., 2026 )	All six tags and imagery in a caption. Tempo classes become Adagio 65, Andante 95, or Allegro 135 bpm. Decoding masks non-piano programs and drum notes.
EMO-Disentangler ( Huang et al., 2024 )	Emotion quadrant, decomposed into valence and arousal across its two stages. The first stage samples and declares its own key.
Piano Transformer ( Lehmkuhl et al., 2025 )	Genre prefix only.
Aria ( Bradshaw et al., 2025 ) , Compose & Embellish ( Wu and Yang, 2023 )	Unconditional piano generation for quality evaluation.

Appendix

Table 15: Baseline input mappings. Emotion quadrants map happy to Q1, angry/anxious to Q2, sad to Q3, and calm to Q4.

Model / stage	Key	Tempo	E4	E	G0	G	A	P	CLaMP 3	CLAP
Reference	1.000	0.820	0.630	0.840	0.500	0.830	0.550	0.530	0.132	0.338
MuseCoco	$0.582^{m}$	0.636	0.280	$0.575^{q}$	0.315	0.386	–	–	–	–
text2midi	0.181	0.456	0.268	0.276	0.330	0.380	0.205	0.270	0.054	0.273
MIDI-LLM	0.340	0.713	0.281	0.285	0.391	0.370	0.201	0.226	0.059	0.292
EMO-Disentangler	$0.752^{\dagger}$	–	0.477	$\mathbf{0.611^{q}}$	–	–	–	–	–	–
Piano Transformer	–	–	–	–	0.375	0.458	–	–	–	–

Appendix

Table 16: Tag agreement with external diagnostics (

\uparrow

). E/G/A/P: emotion/genre/accompaniment/performance. E4: external emotion. G0: zero-shot genre. Ranking and the marks

m

q

, and

\dagger

follow Table 3 .

AR
Model / stage	Key	Tempo	E4	E	G0	G	A	P	CLaMP 3	CLAP
Large public	0.642	0.731	0.440	0.395	0.388	0.451	0.295	0.281	0.083	0.339
$+\,$ Curated	0.559	0.682	0.406	0.370	0.434	0.615	0.236	0.300	0.090	0.332
$+\,$ High-quality	0.767	0.730	0.514	0.585	0.444	0.509	0.296	0.273	0.141	0.335
w/o large public	0.781	0.754	0.516	0.566	0.458	0.481	0.252	0.299	0.131	0.333
DiT

Appendix

Table 17: Control and alignment across training stages (

\uparrow

), with 800 samples per row.

+

denotes continued training. w/o large public : AR trained on the curated and high-quality tiers only, not ranked. Notation follows Table 16 .

AR
	MAESTRO $\downarrow$		POP909 $\downarrow$		PQ $\uparrow$	CE $\uparrow$
Model / stage	V	C	V	C
Large public	0.210	172.422	0.998	286.869	7.775	7.252
$+\,$ Curated	0.319	175.372	1.177	226.742	7.781	7.168
$+\,$ High-quality	0.426	170.697	0.605	201.652	7.946	7.410
w/o large public	0.445	165.549	0.592	206.130	7.947	7.416

Appendix

Table 18: Quality across training stages. Metrics follow Table 5 , and rows follow Table 17 .

AR
	Boundary @5 s $\uparrow$			Contrast $\uparrow$	Pairwise
Model / stage	P	R	F1	(random)	F1 $\uparrow$
Reference	0.876	0.918	0.896	2.52 (1.72)	0.769
Large public	0.384	0.301	0.338	1.88 (1.66)	0.600
$+\,$ Curated	0.362	0.275	0.313	1.82 (1.65)	0.623
$+\,$ High-quality	0.567	0.475	0.517	1.99 (1.69)	0.608

Appendix

Table 19: Temporal structure across training stages (

\uparrow

). P/R: boundary precision/recall. Parentheses give random-position contrast. Rows follow Table 17 .

	Quality (1–5)		Imagery accuracy (%)
System	$<$ 2 years	$\geq$ 2 years	$<$ 2 years	$\geq$ 2 years
Reference	4.00	3.72	70.0	69.4
text2midi	2.90	2.56	56.7	52.8
MIDI-LLM	2.93	2.72	50.0	52.8
TASTE (AR)	3.37	3.17	76.7	83.3
TASTE (DiT)	2.97	2.53	80.0	72.2

Appendix

Table 20: Listening results by years of musical training: 10 participants with less than two years and 12 with two or more. Imagery accuracy is in percent; chance is 50%.

Text-to-Music Generation

Text-to-music generation turns natural-language descriptions into audio or symbolic scores, with control over style, instruments, mood, lyrics, and structure.

See all 47 papers

Text2Score: Generating Sheet Music From Textual Prompts
May 13, 2026Keshav Bhandari, Sungkyun Chang, Abhinaba Roy +4Text-to-Music Generation Music Generation Evaluation
Queen Mary University of London, UK · EmotionWave, Korea · Singapore University of Technology and Design, Singapore +1
MusPyExpress: Extending MusPy with Enhanced Expression Text Support
Aug 21, 2026Phillip Long, Hao-Wen Dong, Julian McAuley +1Symbolic Music Generation Controllable Music Generation
Computer Science & Engineering Department, University of California, San Diego · Department of Performing Arts Technology, University of Michigan, Ann Arbor
Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts
Sep 26, 2026Chao Peter Yang, Cynthia Rudin, Yue Jiang +2Controllable Music Generation Symbolic Music Generation
Yale University New Haven, CT · Duke University Durham, NC · Chinese University of Hong Kong, Shenzhen Shenzhen, Guangdong

May 13, 2026J/K move · Enter open · S save

Text2Score: Generating Sheet Music From Textual Prompts

Keshav Bhandari, Sungkyun Chang, Abhinaba Roy, Francesca Ronchini+3

Queen Mary University of London, UK · EmotionWave, Korea · Singapore University of Technology and Design, Singapore+1

Text-to-Music Generation Music Generation Evaluation Controllable Music Generation Symbolic Music Generation

Open paper AskPDF arXiv

Developing text-driven symbolic music generation models remains challenging due to the scarcity of aligned text-music datasets and the unreliability of automated captioning pipelines. While most efforts have focused on MIDI, sheet music representations are largely underexplored in text-driven generation. We present Text2Score, a two-stage framework comprising a planning stage and an execution stage for generating sheet music from natural language prompts. By deriving supervision signals directly from symbolic XML data, we propose an alternative training paradigm that bypasses noisy or scarce text-music pairs. In the planning stage, an LLM orchestrator translates a natural language prompt into a structured measure-wise plan defining musical attributes such as instruments, key, time signatures, harmony, etc. This plan is then consumed by a generative model in the execution stage to produce interleaved ABC notation conditioned on the plan's structural constraints. To assess output quality, we introduce an evaluation framework covering playability, readability, instrument utilization, structural complexity, and prompt adherence, validated by expert musicians. Text2Score consistently outperforms both a pure LLM-based agentic framework and three end-to-end baselines across objective and subjective dimensions. We open-source the dataset, code, evaluation set and LLM prompts used in this work; a demo is available on our project page (https://keshavbhandari.github.io/portfolio/text2score).

Abstract

Text-to-Music Generation

Explore similar work

Text2Score: Generating Sheet Music From Textual Prompts

MusPyExpress: Extending MusPy with Enhanced Expression Text Support

Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts