cs.SDOct 11, 2026

Harmonic Representations in Symbolic Music Transformers are Localized and Controllable

Authors: Adi Silberschein, Megan Wei, Tamar Rott Shaham

Organizations: Weizmann Institute of Science · Brown University · MIT CSAIL

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Transformer models trained on symbolic music have become remarkably capable. However, it is yet unclear how they represent music structure, and whether understanding their internal representations can be used to unlock methods for model control. We examine this through a case study of harmonic analysis: when a music transformer navigates a cadence, what does it internally "know" about its harmonic context, and can that knowledge be used to affect the generation? Using Aria, a LLaMA-based symbolic music transformer evaluated on Bach chorales, we find that harmonic decisions are causally localized progressively from middle layers onward. Linear probes show that mode becomes decodable in the same layer range where patching identifies growing causal effect, while key identity emerges later. We then confirm these representations are actionable for music generation control: steering via latent directions corresponding to harmonic changes reliably produces a deceptive or minor resolution, while preserving structural coherence and generation quality. This demonstrates that, at inference time, the model's internal harmonic knowledge can be directly manipulated to control the harmonic output.

Figures & tables

Figure 1: Local Intervention. A V →\rightarrow I cadence is identified in a Bach chorale (left). The chorale is truncated at the end of the V chord, and the chordal third of V is lowered by a semitone, turning V into v (right), yielding an original-counterfactual pair that differs in exactly one note.
Figure 2: Intervention on a single pitch shifts cadence resolution. Modifying a single pitch in a V →\rightarrow I or V →\rightarrow i cadence substantially redistributes the model’s resolution distribution in both major and minor source chorales.
Figure 3: Causal Effect of Harmonic Information. Patching residual stream representations from minorized into the original sequences at each layer causes an increase in IIA beginning in the middle layers, indicating that harmonic computation develops progressively through the network. Shaded regions denote 95% confidence intervals.
Figure 4: Linear probe accuracy across layers. Mode (chance = 50%) becomes decodable in the middle layers (from layer 4), while key identity (chance = 5.6%) emerges later in the network (from layer 10). Note that classes might be imbalanced.
Figure 5: Key and Mode Steering. Applying steering on major chorales with steering directions corresponding to relative and parallel minor keys substantially increases the frequency of alternative resolutions compared to the unsteered baseline, which usually resolves to the major tonic (I). Relative minor steering shifts mass toward the submediant (vi), as it is the tonic of the relative minor, while parallel minor steering produces the most decisive shift toward the parallel minor tonic (i). Mode steering also reduces I resolutions but produces no structured redistribution, without concentrating on a specific harmonic target. Numbers above bars are counts per category. org. & rel. diatonic : the chords diatonic to the original key and its relative minor; par. diatonic : chords of the parallel minor only; other : all remaining resolutions.
 Sequence-LevelNote-LevelQualityHarmony
 Struct.PitchAvg. gen.Out ofNote dur.PitchFMDRNTop-1
 errorrepeatdurationrangeavgvar. entropydom.
 (%) ↓\downarrow(%) ↓\downarrow(ms)(%) ↓\downarrow(ms) ↓\downarrow(bits) ↑\uparrow(%) ↓\downarrow
Aria (no steering)0.00.037400.07356080.271.2659.0
Relative minor steering2.90.038000.07285585.421.4055.3
Parallel minor steering0.00.037830.074155106.651.5049.6
Table 1: Key Steering Evaluation. Results confirm that generation quality is well-preserved across key steering conditions: structural errors remain below 3% and out-of-range notes near 0%, FMD stays close to the unsteered baseline (FMD calculated against original Bach chorales; for reference, the FMD between Bach chorales and Bach fugues is 195.97, far larger than any steering-induced change), and RN entropy varies by at most 0.24 bits while top-1 dominance remains near 50% — showing the steered model generates harmonically varied continuations rather than collapsing onto a single chord. For ↓\downarrow lower is better, for ↑\uparrow higher is better, when no indication similarity to the baseline is preferred.
Figure 6: Relative vs. Parallel probing. Grouping keys by their shared key signature (relative-grouped) yields substantially higher peak accuracy (83.4%) than the baseline 18-class key probe (65.5%) or grouping by parallel keys (65.3%). The probe thus separates key classes more easily when each class is defined by a shared pitch collection (relative grouping) than by a shared root (parallel grouping).
Figure 7: Relative keys are more confusable than parallel keys. Confusion matrices at probe peak accuracy layer. (a) The full probe shows systematic confusions between relative key pairs (e.g., F major with D minor, Bb major with G minor), indicating that the model might place relative keys close together in representation space. (b) Grouping by shared key signature (relative-grouped) resolves these confusions, yielding a near-diagonal matrix. (c) Grouping by shared tonal center (parallel-grouped) retains more off-diagonal mass, confirming that parallel keys are more distinct.
More in this topic

Transformer Interpretability

Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.

See all 334 papers

Explore similar work

CardsList

  1. Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation

    May 29, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Disentangled Representation LearningControllable Music Generation

    Athens University of Economics and Business Orfium Athens, Greece · Innovation Lab Orfium Athens, Greece · Department of Music Technology and Acoustics Hellenic Mediterranean University Rethymno, Greece +2

  2. Equivariant Music Transformer

    Aug 4, 2026Zixun Guo, Simon DixonRepresentation LearningTransformer

    Centre for Digital Music, Queen Mary University of London, United Kingdom

  3. Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts

    Sep 26, 2026Chao Peter Yang, Cynthia Rudin, Yue Jiang +2Controllable Music GenerationSymbolic Music Generation

    Yale University New Haven, CT · Duke University Durham, NC · Chinese University of Hong Kong, Shenzhen Shenzhen, Guangdong