Music production typically involves multiple editing operations that each require specialized tools and expertise. Existing text-guided music editing systems support few operations or instruments, operate on short clips, or handle only one operation per instruction, unlike real workflows where several changes are requested at once. Here, we present ProCraft, a lightweight 0.5B-parameter autoregressive codec language model that edits 30s music excerpts end-to-end from free-form instructions, including compound requests. ProCraft supports seven operations across eight instruments, from tasks with a unique target (e.g., track removal) to creative ones (e.g., accompaniment generation). We align intermediate representations with pretrained music embeddings and pitch and instrument targets to preserve content outside the edit, and gated cross-attention in every decoder layer to keep generation tied to the instruction. To train on compound requests, we introduce On-the-fly Editing-pair Synthesis, which generates paired instructions and audio combining track-level operations with effect changes. ProCraft performs competitively against task-specific systems on single-task editing, and outperforms four free-text editors on compound instructions for track removal, extraction and denoising on real (MoisesDB) and synthetic (Slakh) recordings. In a user study (16 listeners), ProCraft was rated highest on removal and denoising and on-par with a 16-times larger model on extraction and instrument transfer. Ablations further show that representation alignment and cross-attention reduce leakage from unwanted tracks and raise editing success rates, especially on out-of-domain data.
Figures & tables
Model
Free phrasing
Remove
Extract
Accomp.
Cover
Instr. transfer
Denoise
Effects
ProCraft (ours)
∙
∙
∙
∙
∙
∙
∙
∙
Instruct-MusicGen ( Zhang et al., 2025b )
×
∙
∙
∙
×
×
×
×
Audio-Omni ( Tian et al., 2026 )
∙
×
×
∙
∙
∙
×
×
AudioSep ( Liu et al., 2025b )
▲
▲
∙
×
×
×
▲
×
SD-Codec ( Bie et al., 2025 )
×
×
▲
×
×
×
∙
×
MSDM ( Mariani et al., 2024 )
×
×
×
∙
×
×
×
×
Table 1: Supported music editing operations per system. ∙ = supported, ▲ = partial or indirect, × = not supported; judgments argued in Appendix H .
Figure 1: Architecture of ProCraft. The instruction, source codes, and K learnable queries precede the target codes in a single causal sequence. The final hidden states of the queries form the edit head hedit , which is injected into every backbone layer through gated cross-attention. During training, four MLP projectors align layer 4 with MERT, pitch, instrument, and StemFX targets.
Figure 2: On-the-fly editing-pair synthesis. For each training item, a track-level task and/or audio effects are applied to the stems of a multitrack recording. A tool-calling language model writes the instruction and specifies audio effects as executable function calls, which are rendered on the stems to produce the target mix.
Figure 3: Model performance on single-task music editing from free-form text, tested on synthetic recordings in Slakh (left) and real recordings in MoisesDB (right) over diverse metrics and model baselines. We multiply a metric by -1 if the raw metric value is better when lower so larger radii always indicate better results. See full results in Appendix B .
Figure 4: Fraction of items whose edit strength exceeds a threshold τ , for ProCraft and ProCraft-plain (150 items per panel). The dotted line marks the 6 dB threshold used for edit rate.
System
Removal
Extraction
Denoising
ProCraft (ours)
−7.39
−15.73
−4.74
ZETA
−46.18
−38.03
−35.42
DDIM inv.
−44.83
−41.26
−32.18
SteerMusic
−25.60
−27.20
−29.90
Audio-Omni
−24.88
−23.98
−26.02
Table 2: SI-SDR (dB, higher is better) on compound instructions in Slakh.
Figure 5: Model performance on music editing from compound instructions on accompaniment, cover, and instrument transfer on Slakh. Larger radii indicate better performance; isolation strength shares the same range across panels. See Appendix C for full results including MoisesDB.
Figure 6: MUSHRA-style user study on perceived audio from music editing with compound instructions. Each triangle shows a task’s mean Editing, Preservation, and Audio-quality ratings (0 to 5) from sixteen raters; See Table 9 and Appendix J for results, statistical inference, and protocol details.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Accompany
CLAP-T ↑
CLAP-A ↑
KL ↓
FAD ↓
KAD ↓
PaSST E ↑
APA P ↑
StructDist P ↓
ProCraft (ours)
0.200
0.590
0.406
0.476
1.959
0.827
0.614
216.624
ProCraft-plain
0.203
0.581
0.376
0.472
1.937
0.853
0.573
211.236
IMG (released)
0.264 ∗
0.604
0.374
0.492
2.005
—
0.565
230.702 ∗
IMG (retrained)
0.134 ∗
0.558 ∗
1.600 ∗
0.549
2.271
0.240 ∗
0.183
262.266 ∗
MSDM
0.143 ∗
0.614 ∘
2.179 ∗
0.456
1.761
0.493 ∗
0.595
282.190 ∗
MusicGen-Stem
0.183 ∗
0.568 ∘
0.558 ∘
0.475
1.924
0.913 ∘
0.623
220.629
Appendix
Table 3: Full metric set, Slakh (synthetic) domain, N=150 per task. E=editing, P=preservation, E+P=both. Our metrics are defined in Appendix E ; a dash marks an undefined metric. Bold = best in column, shaded worst to best; circle and star mark paired tests against ProCraft at p<0.05 and p<0.01 . IMG is Instruct-MusicGen.
Accompany
CLAP-T ↑
CLAP-A ↑
KL ↓
FAD ↓
KAD ↓
PaSST E ↑
APA P ↑
StructDist P ↓
ProCraft (ours)
0.165
0.518
0.472
0.264
0.939
0.840
0.366
209.256
ProCraft-plain
0.166
0.526
0.469
0.272
0.995
0.840
0.369
216.648 ∘
IMG (released)
0.257 ∗
0.535 ∘
0.442
0.264
0.933
—
0.409
217.349 ∘
IMG (retrained)
0.121 ∗
0.513
1.654 ∗
0.323
1.160
0.227 ∗
0.000
262.031 ∗
MSDM
0.119 ∗
0.629 ∗
1.584 ∗
0.315
1.121
0.560 ∗
0.321
279.847 ∗
MusicGen-Stem
0.134 ∗
0.520
0.622 ∘
0.286
1.022
0.947 ∗
0.524
219.187 ∗
Appendix
Table 4: Full metric set, real audio domain, N=150 per task. E=editing, P=preservation, E+P=both. Our metrics are defined in Appendix E ; a dash marks an undefined metric. Bold = best in column, shaded worst to best; circle and star mark paired tests against ProCraft at p<0.05 and p<0.01 . IMG is Instruct-MusicGen.
Accompany
CLAP-T ↑
CLAP-A in-context ↑
KL ↓
FAD ↓
KAD ↓
PaSST E ↑
APA P ↑
StructDist P ↓
Audio Feat. E ↓
isolation E ↑
ProCraft (ours)
0.170
0.565
0.681
0.446
1.782
0.747
0.621
208.784
0.852
7.471
ProCraft-plain
0.323 ∗
0.553
0.723
0.422
1.677
0.747
0.598
211.537
0.809
8.880
IMG (retrained)
0.139 ∗
0.550
1.677 ∗
0.579
2.389
0.193 ∗
0.170
106.944 ∗
0.945 ∘
-11.929 ∗
ZETA
0.277 ∗
0.605 ∗
1.247 ∗
0.561
2.334
0.617 ∘
0.559
299.294 ∗
0.862
-2.330 ∗
DDIM inv.
0.322 ∗
0.571
1.185 ∗
0.579
2.409
0.638 ∘
0.583
275.083 ∗
0.850
8.607
SteerMusic
0.304 ∗
0.655 ∗
1.536 ∗
0.534
2.153
0.247 ∗
0.546
117.297 ∗
0.712 ∗
-6.677 ∗
Appendix
Table 5: Combined edit and audio effects benchmark, Slakh (synthetic) domain, 150 samples per task; the target is the edited and processed audio. E=editing, P=preservation, E+P=both. Bold = best in column, shaded worst to best.
Accompany
CLAP-T ↑
CLAP-A in-context ↑
KL ↓
FAD ↓
KAD ↓
PaSST E ↑
APA P ↑
StructDist P ↓
Audio Feat. E ↓
isolation E ↑
ProCraft (ours)
0.303
0.483
0.959
0.256
0.852
0.693
0.366
123.701
0.846
3.873
ProCraft-plain
0.316
0.500 ∘
0.841
0.260
0.884
0.773
0.461
123.593
0.839
8.704 ∗
IMG (retrained)
0.106 ∗
0.491
1.879 ∗
0.404
1.450
0.180 ∗
0.000
113.529 ∗
0.991 ∗
-11.355 ∗
ZETA
0.113 ∗
0.594 ∗
4.138 ∗
0.387
1.104
0.287 ∗
0.417
96.880 ∗
1.089 ∗
-13.347 ∗
DDIM inv.
0.055 ∗
0.435 ∗
3.426 ∗
0.464
1.590
0.200 ∗
0.237
146.692 ∗
1.002 ∗
-17.029 ∗
SteerMusic
0.234 ∗
0.630 ∗
1.781 ∗
0.348
1.159
0.153 ∗
0.270
83.552 ∗
0.764
-13.333 ∗
Appendix
Table 6: Combined edit and audio effects benchmark, real audio domain, 150 samples per task; the target is the edited and processed audio. E=editing, P=preservation, E+P=both. Bold = best in column, shaded worst to best.
90s grunge
folk back-porch
reggae dub
ambient and dreamy
gospel warmth
shoegaze wall-of-sound
arena rock
gritty electronic
smoky jazz club
boom-bap hip-hop
industrial cold
spaghetti-western twang
bossa nova café
lo-fi bedroom
surf rock
chillwave sunset
modern radio pop
synthwave neon
cinematic and widescreen
motown groove
trap night-drive
Appendix
Table 7: The 30 mixing motivations, in alphabetical order.
Task
Instruction
Single operations
Remove
Cut the bass from the mix
Extract
Extract only the guitar track.
Accompany
Generate a piano part that complements the track
Cover
Arrange entire track as solo piano performance
Instrument transfer
Replace guitar with bass using same notes.
Appendix
Table 8: Example free-form instructions from the demo page.
Task
Attribute
ProCraft (ours)
Audio-Omni
SteerMusic
ZETA
DDIM inv.
Remove
Editing
2.50
1.69
2.00
0.81
0.81
Preservation
4.06
1.38
2.88
0.44
0.50
Audio quality
3.56
0.69
2.12
1.25
1.38
Extract
Editing
3.12
3.25
2.38
0.56
0.06
Preservation
3.44
3.69
3.12
0.31
0.12
Audio quality
3.06
3.69
2.75
1.31
0.31
Appendix
Table 9: Listening test, mean rating (0 to 5) per task, attribute and system over 16 raters. Bold = highest sample mean; underline = second-highest sample mean; significance is reported in Table 11 .
Task
Effect
F (df)
p
ηp2
Remove
system
19.31 (4, 60)
< 0.001
0.56
attribute
4.39 (2, 30)
0.021
0.23
system × attribute
6.25 (8, 120)
< 0.001
0.29
Extract
system
62.83 (4, 60)
< 0.001
0.81
attribute
3.06 (2, 30)
0.062
0.17
system × attribute
2.66 (8, 120)
0.010
0.15
Appendix
Table 10: Repeated-measures ANOVA on the listening ratings, one per task, with system (5 levels) and attribute (3 levels) as within-rater factors over 16 raters. ηp2 = partial eta squared.
Task
Attribute
Audio-Omni
SteerMusic
ZETA
DDIM inv.
Remove
Editing
+0.81 (0.341)
+0.50 (0.376)
+1.69 (0.047)
+1.69 (0.047)
Preservation
+2.69 ( < 0.001)
+1.19 (0.014)
+3.62 ( < 0.001)
+3.56 ( < 0.001)
Audio quality
+2.88 ( < 0.001)
+1.44 (0.005)
+2.31 (0.004)
+2.19 (0.007)
Extract
Editing
-0.12 (0.791)
+0.75 (0.208)
+2.56 ( < 0.001)
+3.06 ( < 0.001)
Preservation
-0.25 (0.656)
+0.31 (0.656)
+3.12 ( < 0.001)
+3.31 ( < 0.001)
Audio quality
-0.62 (0.328)
+0.31 (0.656)
+1.75 (0.019)
+2.75 ( < 0.001)
Appendix
Table 11: All ProCraft–baseline follow-up comparisons after the task-wise ANOVAs. Each cell gives the mean rating difference (ProCraft minus baseline) and Holm-adjusted p ; correction covers 12 comparisons per task.