Audio-language models remain unreliable at extracting basic musical properties from audio, motivating pipelines that place a music information retrieval (MIR) tool between the audio and the model. The tool reports measurements such as keys, chords, and structural similarity, and the model is expected to derive the answer. We test whether it can. Holding the musical information fixed, we vary derivation depth: the number of reasoning steps between the tool output and the answer. When the report states the conclusion, all five models score near 100%, and they follow it even when it is wrong. One step away, accuracy drops. On modulation, open models fall to 57-81%, although they answer that single step correctly when asked directly, and reach 0% at greater depths. Frontier models degrade less, but still degrade. Adding the audio does not recover the loss. The bottleneck in tool-augmented music analysis is not access to musical information but reasoning from it, so tool design must account for how much derivation it leaves to the model.
Figures & tables
Figure 1: Setup and failure modes. (a) The model answers a musical question from the audio and an MIR tool’s output. (b) It follows a stated conclusion even when wrong (top), and fails to derive the conclusion from intermediate output such as the home key and secondary dominant (bottom).
Style
Example tool output
F1
Verdict
“The piece modulates at bar 9 to a new MAJOR key (G major).”
F1
Verdict (flip)
“The piece modulates at bar 9 to a new MINOR key (A minor).”
F2
Key–value
bars9_16_key: G major
F3
Prose
“From bar 9 the harmony re-centers on G major, a major key (pivot chord D7).”
Table 1: One modulation item rendered in each tool-output format. Colors group formats: verdict ( green ), value or prose ( blue ), raw tool output ( rose ).
Model
Verdict
Meas.
Δ no audio
Δ+ aud.
F1
Qwen2.5-Omni
100
36
64 [56,72]
48
0
Music Flamingo
100
35
65 [48,83]
67
0
Audio Flamingo 3
100
35
65 [48,83]
58
0
Gemini 2.5 Flash
100
85
15 [9,21]
21
0
Gemini 3.6 Flash
100
90
10 [4,17]
15
0
Table 2: Format profile, pair accuracy (%), structure and modulation pooled ( n=60 pairs, 12 compositions). Verdict = F1/F3, Measurement = F2/F4/F5/F6. Δ is verdict minus measurement, no audio [95% CI] and with audio. F1 is the flipped verdict (pair accuracy; 0 means every pair followed the wrong statement).
Figure 2: Pair accuracy against derivation depth on three tasks, no audio, n=72 independent items each. Depth zero states the conclusion; each further step withholds one inference. Open models in color, frontier models in gray; dotted line is chance ( 25% ).
Audio,
Ladder, without/with audio
Knowledge, asked alone
Model
no report
d0
d1
d2
d3
d1
d2
d3
Qwen
17
100
76/89
26/25
0/0
100
19/59
0
AF3
0
100
81/8
0/0
0/0
100
17/100
12
MF
0
100
57/81
6/3
0/1
100
85/59
19
G2.5F
17
100
100
15/43
21/32
100
83/100
18
G3.6F
13
100
100
86/81
82/47
100
85/100
23
Table 3: Modulation: derivation ladder beside the knowledge each depth needs. Color marks the reasoning required: none , one step , two or more . Audio, no report : pair accuracy from the clip alone (chance 25 ). Ladder : pair accuracy (%) without / with audio ( n=72 items; one number when both agree). Knowledge : the fact each depth needs, asked alone ( n=48 per question for the open models, 37 for the scale-degree question; n=144 for Gemini): d1, is a named key major or minor; d2, which key a dominant seventh resolves to / whether a scale degree is major or minor; d3, which chord a set of pitches spells. G2.5F/G3.6F: Gemini 2.5/3.6 Flash.
Audio, no report
Δ verdict − meas.
Wrong verdict followed (%)
Model
Synth.
Guitar
Templ.
Librosa
F1
Librosa
Guitar
Qwen
21
55
64/48
60/57
100
100
63
MF
0
62
65/67
73/52
100
100
100
AF3
0
50
65/58
54/61
100
100
33
G2.5F
17
8
15/21
33/23
100
100
100
G3.6F
13
15
10/15
12/22
100
100
100
Table 4: Beyond the template report. Audio, no report : accuracy from the recording alone (synthetic: pair accuracy, chance 25 ; GuitarSet: open-ended key identification on 60 recordings, chance about 4 ). Δ : verdict minus measurement formats, without / with audio, for the template report and for a running librosa analyzer on the same synthetic audio (analyzer correct on 56 of 60 pairs in the open-model runs, 54 in the Gemini runs). Wrong verdict followed : how often a wrong tool conclusion was adopted (synthetic F1 ; the analyzer’s wrong pairs; a wrong key annotation on GuitarSet).
Figure 3: Logit lens on Qwen2.5-Omni: log-probability margin between the correct and incorrect answer words decoded from each layer. With derivation required the margin never leaves zero for modulation (d2, n=72 items) but rises to +0.9 at layer 25 for structure (F4, n=30 ) before falling at the output.
d1
d2
d3
Mod.
76→79
26→0
0→0
Cad.
83→83
17→42
17→8
Struct.
100→62
90→38
40→43
Table 5: Two-call mitigation, Qwen2.5-Omni: pair accuracy (%), single → two-call, n=72 . Column color: reasoning depth; cell color: direction of change.