Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We identify this error pattern as the evidence bottleneck: state-of-the-art systems show larger deficits in acoustic evidence extraction than in downstream reasoning, suggesting that upstream perception is often the limiting factor. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that enhances acoustic evidence preservation through hierarchical aggregation and non-compressive, time-aligned fusion. We also build EvA-Perception, a large-scale training set with about 54K event-ordered captions and 500K evidence-grounded QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception results on MMAU, MMAR, and MMSU, with the largest gains on perception-heavy splits. Human evaluation on open-ended captioning further shows improved fine-grained acoustic coverage and caption quality. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning. Project can be found at https://satsuki2486441738.github.io/EvA/.
Figures & tables
Figure 1: Perception–reasoning performance gap comparison between models and humans. Performance is averaged over Qwen2.5–Omni and Kimi–Audio.
Figure 2: Model architecture. The left half of the left panel shows the Kimi-Audio backbone, while the right half illustrates the additional EvA Path modules. Audio is encoded into four frequency-band features by the CED Encoder, pooled across frequencies, fused via cross-attention, temporally aligned with Whisper, and integrated through gated additive fusion.
Name
# of Audio/QA
Avg. Caps Len
Visual
Music
Speech
Integration
Temporal Info
AudioCaps ( Kim et al., 2019 )
46k/46k
9.03
✗
✗
✗
✗
✗
Clotho ( Drossos et al., 2020 )
5k/5k
11.00
✗
✗
✗
✗
✗
LAION-Audio-630K ( Wu et al., 2023 )
630k/630k
7.30
✗
✗
✗
✗
✗
WavCaps ( Mei et al., 2024 )
403k/403k
7.80
✗
✗
✗
✗
✗
AudioSetCaps ( Bai et al., 2025a )
1.9M/1.9M
28.00
✗
✗
✗
✗
✗
Auto-ACD ( Sun et al., 2024 )
1.5M/1.5M
18.10
✓
✗
✗
✓
✗
Table 1: Comparison of open-source audio caption datasets.
Model
MMAU-Clean
MMAR
MMSU
CochlScene
Perc.
Reas.
Perc.
Reas.
Perc.
Reas.
Qwen2.5-Omni-7B
72.06
67.36
52.80
61.43
39.96
69.91
80.38
Audio-Flamingo-3
73.90
71.50
59.23
61.64
45.63
77.86
75.57
Step-Audio-2-mini
66.54
68.58
54.58
61.01
44.36
78.32
83.24
Audio-Reasoner
65.44
63.39
44.34
38.23
40.73
68.38
–
R1-AQA
72.06
62.87
48.75
50.19
41.68
71.94
76.30
Table 2: Main results under our unified zero-shot setting. For MMAU, we report the decontaminated split, denoted as MMAU-Clean, to reduce potential source-audio overlap with AudioSet-Strong. Green numbers denote absolute gains over Kimi-Audio-7B-Instruct. Definitions of perception and reasoning are given in Sec. 5.1 ; the full MMAU decontamination protocol and original-vs-cleaned results are provided in Appendix A.14 .
Figure 3: Qualitative comparison of captions generated by different models on AudioCaps examples.
Stage
Setting
Trainables
Start Ckpt
AudioCaps (CLAP)
Cos
R@1
R@5
S0
Base Model
N/A
N/A
14.61
9.27
24.97
S1(1)
w/o CED path
Adapter
S0
35.40
18.50
43.60
S1(2)
w/o frequency pooling
Adapter & CED Agg.
S0
35.54
21.24
49.61
S1(3)
w/o crossing fusion
Adapter & CED Agg.
S0
28.63
11.82
29.74
S1(4)
Q-former
Adapter & CED Q-former
S0
36.24
20.08
47.36
Table 3: Ablations of the EvA fusion path. On AudioCaps , we report CLAP-based cosine similarity ( Cos ) and retrieval; on MMAU/MMAR/MMSU, we report Perception accuracy. Adapter denotes the Whisper adapter, and LLM denotes LoRA on the backbone. In Stage 2, w/o CED path is trained and evaluated without the CED branch.
Model
Label Recall
# Obj. Mentions
Obj. Acc.
Arena
Step-Audio-2-mini
58.6
191
67.5
43.2
Qwen2.5-Omni
57.9
166
78.3
55.1
Audio-Reasoner
51.9
128
85.9
64.2
Audio-Flamingo-3
50.4
125
83.2
39.7
R1-AQA
47.4
123
84.6
39.1
Kimi-Audio-7B
31.6
80
71.3
25.3
Table 4: Human evaluation summary on AudioSet-Strong captioning. Green numbers denote gains over Kimi-Audio-7B. Full results are in Appendix A.12 .
Benchmark
Subset
Base Arch.
EvA
MMAU-mini
Perception
75.23
76.16 +0.93
Reasoning
67.80
69.73 +1.93
MMAR
Perception
53.24
56.82 +3.58
Reasoning
61.61
65.08 +3.47
MMSU
Perception
42.02
49.21 +7.19
Reasoning
69.95
79.47 +9.52
Table 5: The performance of Qwen2.5-Omni architecture.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Stage 1 (Alignment)
Stage 2 (SFT, LoRA)
Trainable modules
CED Aggregator
CED Aggregator, Whisper adapter; LLM via LoRA
Dataset
EvA-Alignment
EvA-Perception
Epochs
5
2
Per-device batch
2
2
Grad. accumulation
8
16
Global batch (8 × A100)
128
256
Appendix
Table 6: Training hyperparameters. Encoders (Whisper, CED-Base) are frozen throughout.
Item
Result
Sample size
500 captions
Sampling seed
42
Audit input
Generated caption only
Flagged captions
7 / 500
Estimated leakage rate
1.4%
Appendix
Table 7: Summary of the visual-leakage audit.
Model
Label Recall
# Obj. Mentions
Obj. Acc.
Arena Shown
W/L/T
Arena
Step-Audio-2-mini
78 / 133 (58.6)
191
129 / 191 (67.5)
154
50 / 71 / 33
43.2
Qwen2.5-Omni
77 / 133 (57.9)
166
130 / 166 (78.3)
138
63 / 49 / 26
55.1
Audio-Reasoner
69 / 133 (51.9)
128
110 / 128 (85.9)
169
94 / 46 / 29
64.2
Audio-Flamingo-3
67 / 133 (50.4)
125
104 / 125 (83.2)
165
46 / 80 / 39
39.7
R1-AQA
63 / 133 (47.4)
123
104 / 123 (84.6)
137
36 / 66 / 35
39.1
Kimi-Audio-7B
42 / 133 (31.6)
80
57 / 80 (71.3)
162
23 / 103 / 36
25.3
Appendix
Table 8: Detailed human evaluation on AudioSet-Strong captioning. Values in parentheses are percentages; Arena treats ties as half a win.
MMAU split
Original
Removed
Cleaned
Perception
323
51
272
Reasoning
677
98
579
Total
1,000
149
851
Appendix
Table 9: Statistics of the decontaminated MMAU-Clean split.
Model
Original
MMAU-Clean
Δ
EvA
78.33
77.57
−0.75
Audio-Flamingo-3
76.78
73.90
−2.88
Qwen2.5-Omni-7B
73.68
72.06
−1.63
R1-AQA
73.99
72.06
−1.93
Step-Audio-2-mini
70.28
66.54
−3.73
Kimi-Audio-7B-Instruct
67.18
66.18
−1.01
Appendix
Table 10: Original and decontaminated results on the MMAU Perception subset.
Model
Original
MMAU-Clean
Δ
Audio-Flamingo-3
74.15
71.50
−2.65
Step-Audio-2-mini
71.79
68.58
−3.21
Qwen2.5-Omni-7B
69.72
67.36
−2.36
EvA
67.35
64.17
−3.19
R1-AQA
65.88
62.87
−3.01
Audio-Reasoner
65.73
63.39
−2.35
Appendix
Table 11: Original and decontaminated results on the MMAU Reasoning subset.
Figure 4: Instruction for Caption Generation.
Figure 5: Instruction for QA Generation.
Dataset
Constituent Sources
Modality
Quantity
EvA-Alignment
EvA-Captions
Sound, Speech, Music
53,934
AudioTime ( Xie et al., 2025a )
Sound
5,000
CommonVoice ( Ardila et al., 2019 )
Speech
20,000
MusicBench ( Melechovsky et al., 2023 )
Music
30,000
MusicCaps ( Agostinelli et al., 2023 )
Music
4,852
Total
113,786
Appendix
Table 12: Composition of EvA-Perception and EvA-Alignment datasets.
Figure 6: Example from EvA-Captions and EvA-QA (Case 1).
Figure 7: Example from EvA-Captions and EvA-QA (Case 2).
Model
Sound
Speech
Music
Avg.
Qwen2-Audio
58.86
47.75
44.31
50.30
Qwen2.5-Omni
73.87
65.47
67.96
69.10
Kimi-Audio
74.77
62.35
64.24
67.19
Audio-Reasoner
65.77
66.07
66.77
65.00
R1-AQA
74.47
65.17
66.77
68.80
EvA(Ours)
80.78
68.47
74.65
74.63
Appendix
Table 13: Main results on MMAU-mini-test.
Model
Single Modality
Mixed Modality
Avg.
Sound
Speech
Music
S-M
S-S
M-S
S-M-S
Qwen2-Audio
52.73
42.86
34.95
36.36
50.46
45.12
50.00
44.80
Qwen2.5-Omni
59.39
61.22
48.06
54.55
61.01
64.63
58.33
58.30
Kimi-Audio
55.76
59.86
45.15
36.36
61.01
54.88
45.83
55.40
Audio-Reasoner
50.30
49.66
38.35
36.36
56.42
48.78
50.00
48.70
R1-AQA
60.00
51.36
42.23
54.55
57.80
52.44
45.83
52.30
Appendix
Table 14: Main results on MMAR.
Setting
MMAU
MMAR
MMSU
CochlScene
Mask 0–2 kHz
72.20
55.53
63.72
60.67
Mask 2–4 kHz
72.30
55.63
63.72
60.91
Mask 4–6 kHz
72.30
56.24
63.82
60.61
Mask 6–8 kHz
72.40
55.33
63.88
61.00
Mask 0–4 kHz
72.30
54.93
63.11
60.67
Mask 4–8 kHz
71.90
55.43
63.98
60.78
Appendix
Table 15: Main results on various audio benchmarks. All numbers are average scores.
Figure 8: Comparison of audio token sequence length after feature fusion. SALMONN’s Q-Former compresses audio tokens into a shorter latent sequence, while EvA preserves sequence length via inject-and-add fusion.
Large language models (LLMs) have been widely used as knowledge backbones of Large Audio Language Models (LALMs), yet how much auditory knowledge they encode through text-only pre-training and how this affects downstream performance remains unclear. We study this gap by comparing different LLMs under two text-only and one audio-grounded setting: (1) direct probing on AKB-2000, a curated benchmark testing the breadth and depth of auditory knowledge; (2) cascade evaluation, where LLMs reason over text descriptions from an audio captioner; and (3) audio-grounded evaluation, where each LLM is fine-tuned into a Large Audio Language Model (LALM) with an audio encoder. Our findings reveal that auditory knowledge varies substantially across families, and text-only results are strongly correlated with audio performance. Our work provides empirical grounding for a comprehensive understanding of LLMs in audio research.
Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang +13
National Taiwan University, Taiwan · NVIDIA · Academia Sinica, Taiwan
Multimodal large language models can exhibit text dominance, over-relying on linguistic priors instead of grounding predictions in non-text inputs. One example is large audio-language models (LALMs) where decisive audio evidence can be under-utilized even when it contains important information. To address this issue we use mechanistic interpretability to identify a small set of audio-specialist attention heads whose audio attention yields a ``listening'' signal. We show that this signal increases when audio evidence affects the model's output, providing an indicator of audio engagement under standard prompting. Leveraging this localization, we construct an audio--silence steering direction and apply an inference-time activation intervention to the final representation, amplifying the model's audio effect. To demonstrate the utility of this intervention, we show on MMAU that this improves accuracy by up to +8.0 percentage points on two Qwen-based LALMs, without any parameter updates.
Neta Glazer, Lenny Aharon, Ethan Fetaya
Bar-Ilan University, Ramat Gan, Israel · Columbia University, New York, NY, USA
Large audio language models (LALMs) can answer questions about speech, music, and environmental sounds, yet their internal reasoning is largely opaque and difficult to validate. We describe TalTech's solution to the Agent Track of the Interspeech 2026 Audio Reasoning Challenge, in which systems are evaluated on reasoning process quality, specifically the factual accuracy, logical soundness, and completeness of their reasoning chains. Our multi-source ensemble pipeline uses two LALMs that generate independent observations, while a separate text-only reasoning model cross-checks these against outputs from 25 acoustic tools organized into reliability tiers. By grounding every inference step in explicit, reliability-tagged evidence, the system produces dense, verifiable reasoning chains. Our system ranked first in the challenge, outperforming all competing systems by a wide margin in challenge's reasoning quality metric.