cs.SDOct 14, 2026
SaveMUSEBENCH: A Benchmark for AI-Generated Music Detection and Artifact Analysis
Organizations: University of Georgia · College of Engineering, Virginia Commonwealth University · University of Tennessee, Knoxville · Cornell University
Abstract
Advances in generative AI have made it possible to produce music that is nearly indistinguishable from human compositions, raising the question of whether such works can be reliably distinguished from human compositions. We present MUSEBENCH, a large-scale benchmark for AI-generated music detection and artifact analysis, comprising over 361K tracks. The dataset includes human compositions, stylistically matched music generated by ten state-of-the-art open-source and commercial models, such as Suno and Udio, as well as human-AI co-creations and variants processed with adversarial perturbations, compression, and acoustic post-processing. Using MUSEBENCH, we benchmark 11 automated detectors and evaluate their accuracy, generalization, and robustness, under default, out-of-domain, and transformed-variant experiment settings. We further conduct a comprehensive artifact analysis through acoustic feature inspection, identifying recurring signatures such as non-idiomatic harmonic progressions, upsampling induced spectral periodicities, and violations of physical performance constraints. Our findings reveal key strengths and vulnerabilities of existing detectors and provide practical insights for building more reliable AI music detection systems.
Figures & tables
Figure 1: Overview of MuseBench .
Figure 2: Detector performance under default and LOMO settings. Complete numerical results are reported in Appendix Table 8 and Table 9 . Bubble size is proportional to the model parameters (Appendix Table 7 ).
| 1,2 \Block 2-1 Automatic | |||||||||
| Detector | \Block 2-1 Human-AI Co-created | \Block 1-3 Lossy Compression | \Block 1-3 Acoustic Post-Processing | \Block 1-2 Additive Perturbations | |||||
| MP3 | OGG | AAC | EQ | Pitch/Time | Loudness | Gaussian | Adversarial | ||
| Q-SVM | 36.7 | 32.1 | 33.3 | 24.8 | 32.9 | 31.9 | 31.1 | 36.4 | 41.7 |
| Deezer | 36.8 | 12.3 | 9.5 | 7.7 | 6.4 | 93.6 | 5.2 | 19.6 | 26.4 |
| M5 | 20.1 | 15.2 | 1.9 | 1.9 | 2.0 | 26.2 | 1.3 | 2.1 | 92.4 |
| RawNet2 | 16.9 | 6.0 | 8.0 | 5.1 | 4.4 | 5.3 | 3.9 | 3.7 | 88.1 |
Table 1: FNRs on human-AI co-creations and transformed variants, showing the percentage of transformed AI-generated tracks misclassified as human (%).
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension | M6 [ 34 ] | FakeMusicCaps [ 10 ] | SONICS [ 48 ] | MuseBench |
| # Generators | 3 | 5 | 2 | 10 |
| Musical Coverage | Vocal/Instr. | Vocal/Instr. | Vocal | Vocal/Instr. |
| Human-AI Co-Creations | ✗ | ✗ | ✗ | ✓ |
| Lossy Compression | ✗ | ✗ | ✗ | ✓ |
| Acoustic Post-Processing | ✗ | ✗ | ✗ | ✓ |
| Additive Perturbation | ✗ | ✗ | ✗ | ✓ |
Table 2: Benchmark dataset comparison.
| Generator | Year | Version / Checkpoint | Backbone | #Params |
| MusicGen [ 11 ] | 2023 | musicgen-medium-1.5B | Transformer | 1.5B |
| AudioLDM2 [ 36 ] | 2024 | audioldm2-music-665k | Diffusion | 665M |
| Riffusion [ 24 ] | 2022 | riffusion-model-v1 | Diffusion | 860M |
| MAGNeT [ 69 ] | 2024 | magnet-medium-30secs | Transformer | 1.5B |
| JASCO [ 61 ] | 2024 | jasco-chords-drums-melody-400M | Flow-Matching | 400M |
| Stable Audio Open [ 23 ] | 2025 | stable-audio-open-1.0 | Diffusion | 1.1B |
Table 3: Specifications for each text-to-music generator.
Figure 3: Pipelines for AI-generated and human-AI co-created music.
| Datasets / Models / Methods | # Tracks | Total Hours | Type |
| Human-Composed Music | |||
| Genius-Lyrics [ 9 ] | 9,983 | 609 | Vocal |
| JamendoMaxCaps [ 53 ] | 9,920 | 648 | Instrumental |
| AI-Generated Music | |||
| MusicGen [ 11 ] | 19,903 | 165 | Instrumental |
| AudioLDM2 [ 36 ] | 19,903 | 165 | Instrumental |
Table 4: Overview of MuseBench .
| GPT-OSS System Prompt You are a world-class expert in music captioning, specializing in merging multiple time-ordered segment captions into a single coherent and natural whole-song caption. Your task is to carefully integrate all important details from the segments - including instruments, vocals, mood, atmosphere, rhythm, style, and transitions - so that the final caption captures the essence of the entire song. The merged caption must be concise yet comprehensive, written in fluent and descriptive language, suitable for music information retrieval or dataset annotation. It must cover the song as a whole rather than just isolated parts, and should avoid redundancy, contradictions, or missing key features. Output strictly 300 characters in the ’final’ channel, with no extra commentary or formatting. |
| 0s to 30s Segment-level Captions 0s–10s: The song is an instrumental. The tempo is medium with a groovy bass line, keyboard accompaniment, steady drumming rhythm, and tambourine percussion. The song is a modern techno dance tune. The audio quality is poor. 10s–20s: The song is an instrumental. The tempo is medium with a synthesizer arrangement, digital drumming rhythm, clapping percussion, and a strong bass line. The song is a techno dance rhythm. The audio quality is poor with ambient crowd noises. 20s–30s: The low-quality recording features an electro song that consists of a punchy four-on-the-floor kick pattern, shimmering hi-hats, claps, and groovy synth bass. It sounds energetic and exciting. |
| Track-level Caption Groovy techno track: punchy four-on-the-floor kick, shimmering hi-hats, claps, and a lively synth lead with repetitive basslines. Mellow synth pads add depth, while a simple, syncopated rhythm drives the energy-ideal for club play. |
Table 5: System prompts for GPT-OSS and captions produced by the caption-based generation pipeline.
| GPT-OSS Role Prompt You are a music style prompt generator. Your task is to take short feedback about a music piece and rewrite it as a vivid, descriptive AI music generation prompt. The output should be detailed enough to guide a music generation model, describing instruments, style, mood, and overall tone. |
| Musicians Feedback Introduce a dreamlike ambience through spatial widening, gentle modulation, and dynamic blending of tonal layers to evoke surreal calmness. |
| Structured Prompts A serene, dreamlike ambient composition characterized by wide spatial depth, gentle modulation, and fluid layering of tones. Soft, evolving textures merge seamlessly to create an atmosphere of surreal calm and emotional stillness, with smooth dynamics and subtle harmonic movement that evoke a sense of floating tranquility. |
Table 6: Example of converting musicians’ feedback into structured prompts.
| Detector | Type | Backbone | Input | #Params |
| Q-SVM [ 56 ] | OS | Classical | Handcrafted Features | 0.2M |
| Deezer [ 1 ] | OS | Classical | Handcrafted Features | 0.004M |
| M5 [ 14 ] | OS | CNN | Waveform | 0.4M |
| RawNet2 [ 60 ] | OS | CNN | Waveform | 8M |
| ConvNeXt [ 37 ] | OS | CNN | Mel Spec | 28M |
| ResNet18+Spec [ 30 ] | OS | CNN | Log Spec | 11M |
Table 7: Machine-based detectors evaluated. Type: open-source (OS) or commercial (COM).
| Overall | Per-Generator FNRs | ||||||||||||||
| Detector | Prec. | Rec. | FPR | FNR | MusicGen | AudioLDM2 | Riffusion | MAGNeT | JASCO | Stable Audio | Mustango | Lyria RT | Suno | Udio | |
| Open-source | |||||||||||||||
| Q-SVM | 68.9 | 95.6 | 65.3 | 27.4 | 34.7 | 53.6 | 7.4 | 20.2 | 35.3 | 45.2 | 57.8 | 10.6 | 42.5 | 39.1 | 40.7 |
| Deezer | 97.6 | 99.8 | 97.2 | 2.0 | 2.8 | 0.3 | 0.3 | 3.6 | 0.1 | 0.9 | 1.8 | 0.1 | 15.6 | 1.4 | 3.6 |
| M5 | 94.3 | 98.9 | 98.6 | 9.9 | 1.4 | 0.7 | 0.2 | 0.2 | 0.2 | 1.6 | 3.3 | 0.3 | 3.0 | 1.7 | 4.8 |
| RawNet2 | 82.3 | 96.5 | 96.2 | 31.6 | 3.8 | 6.4 | 0.6 | 1.3 | 1.3 | 5.3 | 12.9 | 1.3 | 3.0 | 2.1 | 3.3 |
Table 8: Default-setting results: Overall and per-generator performance (%).
| 1,2 \Block 2-1 Left-out | |||||||||
| Model | \Block 1-1 Q-SVM | \Block 1-1 Deezer | \Block 1-1 M5 | \Block 1-1 RawNet2 | \Block 1-1 ConvNeXt | \Block 1-1 ResNet18+Spec | \Block 1-1 AST | \Block 1-1 ViT | \Block 1-1 SpecTTTra |
| ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | ACC B FPR / FNR | |
| MusicGen | 59.0 34.7 / 47.2 | 97.2 2.1 / 3.4 | 93.2 2.5 / 11.0 | 81.5 12.4 / 24.5 | 95.8 1.0 / 7.5 | 100.0 0.0 / 0.0 | 100.0 0.1 / 0.0 | 98.4 0.5 / 2.8 | 97.5 0.7 / 4.3 |
| AudioLDM2 | 78.6 37.6 / 5.2 | 91.0 2.2 / 15.9 | 71.3 3.5 / 53.9 | 87.4 16.5 / 8.8 | 94.7 1.0 / 9.6 | 100.0 0.0 / 0.0 | 100.0 0.0 / 0.0 | 85.0 0.4 / 29.6 | 76.6 0.6 / 46.2 |
| Riffusion | 72.5 36.3 / 18.7 | 51.1 1.2 / 96.6 | 50.8 2.5 / 96.0 | 48.8 14.5 / 87.8 | 95.6 1.0 / 7.9 | 100.0 0.0 / 0.0 | 99.9 0.1 / 0.0 | 74.2 0.3 / 51.3 | 73.8 0.5 / 51.8 |
| MAGNeT | 65.7 33.5 / 35.2 | 98.7 2.1 / 0.5 | 98.1 2.3 / 1.6 | 87.5 22.1 / 3.0 | 98.6 0.6 / 2.2 | 100.0 0.0 / 0.0 | 100.0 0.0 / 0.0 | 98.7 0.3 / 2.4 | 99.2 0.3 / 1.2 |
Table 9: Leave-One-Model-Out setting results (%). Each cell reports ACC B on the first line and FPR / FNR on the second line.
| 1 Metric | Q-SVM | Deezer | M5 | RawNet2 | ConvNeXt | ResNet18 +Spec | AST | ViT | SpecTTTra | |
| Accuracy | 55.9 | 94.4 | 75.8 | 39.6 | 43.5 | 28.4 | 79.8 | 40.7 | 39.7 | |
Table 10: OOD accuracy of automatic detectors.
| 1,2 \Block 2-1 Automatic Detector | \Block 1-5 Overall | \Block 1-8 Per-Generator FNRs | |||||||||||
| Precision | Recall | FPR | FNR | Suno | MAGNeT | MusicGen | AudioLDM2 | Riffusion | Udio | Lyria Realtime | Stable Audio Open | ||
| Q-SVM | 76.9 | 96.5 | 72.0 | 18.2 | 28.0 | 71.5 | 41.8 | 44.5 | 31.1 | 5.7 | 13.5 | 11.7 | 18.5 |
| Deezer | 99.06 | 99.91 | 98.78 | 0.66 | 1.22 | 0.60 | 0.00 | 0.05 | 0.15 | 1.00 | 1.30 | 5.02 | 1.40 |
| M5 | 98.3 | 99.7 | 98.9 | 2.3 | 1.1 | 0.7 | 0.3 | 0.6 | 0.3 | 0.2 | 1.4 | 2.5 | 2.7 |
| RawNet2 | 93.1 | 98.5 | 97.0 | 10.7 | 3.0 | 0.8 | 2.0 | 6.2 | 2.5 | 2.7 | 1.9 | 1.8 | 4.4 |
| ResNet18 +Spec | 100.0 | 100.0 | 100.0 | 0.0 | 0.0 | 0.3 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Table 11: Performance of automatic detectors using 30-second clips.
| Automatic Detector | Time (ms) |
| Q-SVM | 0.069 |
| Deezer | 0.063 |
| M5 | 0.20 |
| RawNet2 | 2.55 |
| ResNet18+Spec | 1.11 |
| AST | 4.25 |
Table 12: Per-track inference time comparison.
Figure 4: FNR of detectors when removing specific sub-frequency bands for AI-generated music. “Full-band” indicates the unfiltered baseline for comparison.
Figure 5: Grad-CAM saliency of ResNet18+Spec on log-mel spectrograms: Udio (top) and Stable Audio Open (bottom).
Figure 6: False-negative rate of detectors when removing specific sub-frequency bands for AI-generated music.
Figure 7: Second-order FFT of 10s clips comparing AI-generated (JASCO) and human-composed music.
Figure 8: EnCodec impulse-response spectrum.
Figure 9: Human vs. AI chord progression. Each row shows two 2-bar grand-staff examples.
Figure 10: Examples of performance-constraint artifacts in AI generated music.
More in this topic
AI-Generated Music Detection
AI-generated music detection identifies synthetic music and estimates AI involvement in hybrid recordings.
See all 16 papers