How Close to the Theme? Similarity-Label Conditioning for Game Music Generation
Organizations: Centre for Digital Music, Queen Mary University of London, UK
Abstract
Composers develop music from an original idea, controlling how similar each new section is to it, from near repetition to new material. Game music needs this often, as its material repeats over long play. Music models, however, rarely learn to control similarity from real pieces. We further train a pretrained MIDI model on section pairs from Super Mario pieces. A label states each pair's similarity, and a context vector summarises the given section, or context. For 157 test contexts, the model generates six outputs per label. Twelve measures score the outputs; one main measure each covers melody, pitch content, rhythm and note length. On average, outputs requested as similar are more similar to their context than those requested as dissimilar, and not to other pieces. All twelve measures favour them; each main measure stays significant after correction for multiple tests. On melody, the gap comes mainly from more frequent restatement of the context's melody. Against an otherwise identical model without a label, only the request for high similarity makes outputs more similar. In a listening test, ratings followed melodic overlap with the context most closely of all measures.
Figures & tables
| Measure | Computation | Annotation | Listeners |
|---|---|---|---|
| Melody | Jaccard overlap of the three-interval sequences of the two melodies | 0.92 | 0.62 |
| Chroma | Cosine similarity of pitch-class histograms, each note counted once, at the best of 12 transpositions | 0.90 | 0.27 |
| Rhythm | One minus the Mallows distance between inter-onset intervals in sixteenth notes, capped at one bar, over the cap | 0.75 | 0.23 |
| Duration | Overlap of note-length histograms with eight fixed bins, from a thirty-second to a double whole note | 0.77 | 0.14 |
| CS (ours) | The label measure | 0.83 | 0.30 |
| Measure | Generated Gap | 95% Interval | Real Gap | Ratio |
|---|---|---|---|---|
| Melody | 0.0080 | 0.0045 to 0.0121 | 0.249 | 3.2% |
| Chroma | 0.0137 | 0.0048 to 0.0227 | 0.065 | 21.1% |
| Rhythm | 0.0050 | 0.0019 to 0.0082 | 0.051 | 9.8% |
| Duration | 0.0154 | 0.0023 to 0.0294 | 0.168 | 9.2% |
| CS (ours) | 0.0132 | 0.0030 to 0.0231 | 0.355 | 3.7% |
| Measure | High > No Label | No Label > Low | No Label > Mid | Wrong Label: High > Low | No Label > Unconditioned |
|---|---|---|---|---|---|
| Melody | 66% (0.003) | 49% (1.000) | 54% (0.993) | 36% (0.020) | 57% (0.453) |
| Chroma | 58% (0.024) | 54% (0.922) | 59% (0.016) | 45% (0.044) | 68% (0.003) |
| Rhythm | 69% (0.003) | 49% (1.000) | 53% (0.993) | 52% (1.000) | 57% (0.196) |
| Duration | 53% (0.067) | 54% (0.923) | 65% (0.005) | 46% (0.231) | 62% (0.010) |
| CS (ours) | 60% (0.004) | 44% (1.000) | 49% (0.993) | 50% (1.000) | 50% (0.214) |
| Measure | Rise per SD | 95% Interval | p | Holm p |
|---|---|---|---|---|
| Melody | +6.7 | 4.6 to 8.9 | < 0.001 | < 0.001 |
| Chroma | +2.9 | 0.2 to 5.5 | 0.035 | 0.21 |
| Rhythm | +2.7 | -0.04 to 5.4 | 0.054 | 0.22 |
| Duration | +1.5 | -1.2 to 4.3 | 0.28 | 0.44 |
| CS (ours) | +3.4 | 0.9 to 6.0 | 0.009 | 0.060 |
| Table S1. Formulas of the Label Measure’s Components and of the Seven Additional Measures | |
|---|---|
| Name | Computation |
| CS-Chroma | Highest cosine similarity of the duration-weighted pitch-class histograms over the 12 transpositions of |
| CS-Duration | Cosine similarity of note-length histograms with 16 logarithmic bins between each section’s own shortest and longest note, so the two sections’ bins differ |
| CS-Register | , with the mean pitch in semitones |
| CS-Density | , with the notes per unit time |
| CS | for the four components in this order, the weights renormalised when a component cannot be computed |
| Table S2. Details Needed to Reproduce the Results | |
|---|---|
| Item | Value |
| Scores and data | NinSheetMusic MusicXML, converted to ABC for annotation and to note arrays for the model; third-party transcriptions, not redistributed. The annotations, pair lists, listening-test ratings and code are public (see Code and Data Availability) |
| Annotation | Gemini 3.1 Pro, prompt version 1, thinking level high, default sampling, JSON output; the prompt and output schema are part of the code release |
| Pair labels | Thresholds 0.60 and 0.78 applied: the tertiles of CS over all 3,304 pairs, test pieces included (0.6033 and 0.7779), rounded; a pair at or above a threshold takes the higher label; fixed once, they favour no label |
| Split | 236 / 49 / 49 pieces, stratified by section count, with 2,398 / 478 / 428 pairs; arrangements of one tune in one set |
| Sequence | One token per note, with its six fields; context and target each truncated to 509 notes, so that both and five special tokens fit 1,024; one CTX token per target bar comes on top |
| Table S3. All Twelve Measures Place High-Similarity Outputs Closest to the Context | |||||
|---|---|---|---|---|---|
| Measure | Comparison | Share | Mean Gap (95% Interval) | Holm p | After the Filter: Mean Gap; Holm p |
| Melody | high > low | 62.7% | +0.0080 (+0.0045 to +0.0121) | 0.006 | +0.0094 (+0.0050 to +0.0142); 0.006 |
| high > mid | 70.4% | +0.0119 (+0.0081 to +0.0163) | 0.006 | +0.0132 (+0.0086 to +0.0184); 0.006 | |
| mid > low | 45.6% | -0.0038 (-0.0064 to -0.0012) | 1.000 | -0.0038 (-0.0076 to +0.0000); 1.000 | |
| Chroma | high > low | 55.4% | +0.0137 (+0.0048 to +0.0227) | 0.006 | +0.0121 (+0.0015 to +0.0228); 0.091 |
| high > mid | 66.9% | +0.0231 (+0.0157 to +0.0301) | 0.006 | +0.0167 (+0.0063 to +0.0271); 0.007 | |
| Measure | High > No Label | No Label > Low | No Label > Mid | Wrong Label: High > Low | No Label > Unconditioned |
|---|---|---|---|---|---|
| Melody | 62% (0.003) | 51% (1.000) | 56% (1.000) | 40% (0.040) | 50% (0.948) |
| Chroma | 56% (0.020) | 50% (1.000) | 55% (1.000) | 47% (1.000) | 62% (0.003) |
| Rhythm | 68% (0.003) | 45% (1.000) | 47% (1.000) | 52% (1.000) | 55% (0.248) |
| Duration | 57% (0.020) | 50% (1.000) | 64% (0.005) | 50% (1.000) | 53% (0.339) |
| CS (ours) | 62% (0.003) | 52% (1.000) | 54% (1.000) | 49% (1.000) | 52% (0.948) |
| Measure | Gap, Own Context | Gap, Foreign Contexts | Difference (95% Interval) | Holm p |
|---|---|---|---|---|
| Melody | 0.0080 | -0.0001 | 0.0081 (0.0041 to 0.0126) | 0.003 |
| Chroma | 0.0137 | 0.0057 | 0.0080 (0.0010 to 0.0149) | 0.030 |
| Rhythm | 0.0050 | 0.0012 | 0.0038 (0.0009 to 0.0069) | 0.030 |
| Duration | 0.0154 | 0.0051 | 0.0103 (-0.0036 to 0.0248) | 0.069 |
| CS (ours) | 0.0132 | -0.0003 | 0.0136 (0.0033 to 0.0236) | 0.030 |
| Table S6. The Three Strongest Predictors of Listener Ratings All Follow the Melody | |||||
|---|---|---|---|---|---|
| Measure | Rise per SD | 95% Interval | p | Holm p | r over 60 Clips |
| Melody | +6.7 | 4.60 to 8.86 | < 0.001 | < 0.001 | 0.62 |
| Chroma | +2.9 | 0.20 to 5.51 | 0.035 | 0.211 | 0.27 |
| Rhythm | +2.7 | -0.04 to 5.42 | 0.054 | 0.224 | 0.23 |
| Duration | +1.5 | -1.25 to 4.30 | 0.281 | 0.442 | 0.14 |
| CS (ours) | +3.4 | 0.87 to 6.01 | 0.009 | 0.060 | 0.30 |
Music Generation
Music generation studies computational methods for composing, synthesizing, and editing music from symbolic, audio, textual, and multimodal inputs.
See all 110 papers