Multi-Source Evidence Fusion for Audio Question Answering

Aivo Olev; Tanel Alumäe

eess.ASMar 18, 2026

Multi-Source Evidence Fusion for Audio Question Answering

Authors: Aivo Olev, Tanel Alumäe

Organizations: Tallinn University of Technology, Estonia

Abstract

Large audio language models (LALMs) can answer questions about speech, music, and environmental sounds, yet their internal reasoning is largely opaque and difficult to validate. We describe TalTech's solution to the Agent Track of the Interspeech 2026 Audio Reasoning Challenge, in which systems are evaluated on reasoning process quality, specifically the factual accuracy, logical soundness, and completeness of their reasoning chains. Our multi-source ensemble pipeline uses two LALMs that generate independent observations, while a separate text-only reasoning model cross-checks these against outputs from 25 acoustic tools organized into reliability tiers. By grounding every inference step in explicit, reliability-tagged evidence, the system produces dense, verifiable reasoning chains. Our system ranked first in the challenge, outperforming all competing systems by a wide margin in challenge's reasoning quality metric.

Figures & tables

Panel of Figure 1 — Figure 1: Multi-source ensemble pipeline. Two speech LLMs analyze audio across four segments each; unified analysis identifies corroborations and disagreements. Step 1 gathers evidence from reliability-tiered tools (up to 3 rounds), three-stage contradiction detection generates verification hypotheses, and Step 2 performs targeted segment-level validation. Bracket counts indicate additional music-specific tools.

Tier	Representative tools	Cap	Weight
Analytic	Beat detection, energy dynamics, spectral features	0.90	1.0
Probabilistic	Whisper ASR, diarization, source separation, instruments	0.75	0.75
Heuristic	Chord analysis, environment detection, scene context	0.60	0.50
LALMs	StepAudioR1, Qwen3-Omni	0.70	0.40

Table 1: Tool reliability tiers with representative tools, default confidence caps, and evidence weights.

Agent Track
Rank	Team / Model	Rubrics	Acc.
1	TalTech (ours)	69.83	76.9%
2	Team B	66.23	77.4%
3	Team C	66.09	75.1%
4	Team D	64.61	72.2%
5	Team E	63.00	71.0%

Table 2: Agent Track leaderboard (top 10) and single-model baselines from the MMAR benchmark [ 7 ] . Rubrics = MMAR-Rubrics composite score (ranking criterion); Acc. = raw accuracy. Single-model baselines do not have Rubrics scores.

Agreement	N	Correct	Accuracy
Unanimous	128	121	94.5%
Majority	565	470	83.2%
Conflicting	307	178	58.0%
Overall	1000	769	76.9%

Table 3: Accuracy by LALM agreement level.

Layer	Sub-category	N	Acc.	Reas.	%Max
Signal	Acoustic Quality	18	88.9	84.1	94.6
	Anomaly Detection	17	70.6	65.5	92.8
	Audio Difference	8	37.5	25.8	68.9
Perception	Correlation	50	84.0	75.2	89.5
	Counting & Stats	99	62.6	54.0	86.2
	Environ. Perc.	149	82.6	75.5	91.4

Table 4: Performance by MMAR reasoning layer and sub-category. %Max = reasoning quality given correct answers.

Tool	Model / Toolkit	Mus.	Int.	Samples	Avg	Direct	Confirm	Deny	Resolve	Irrel.	% Irrel.
Speech LLM query (Qwen3)	Qwen3-Omni			926	3.51	303	342	78	81	121	13.1%
Transcription	Canary [ 27 ] / Whisper lg-v3			602	3.46	233	181	61	4	122	20.3%
Diarization + transcription	Whisper lg-v3 + ECAPA-TDNN [ 30 ]			572	3.33	177	225	20	15	134	23.5%
Speech LLM query (StepAudio)	StepAudioR1			132	2.87	15	67	11	13	27	20.3%
Melody transcription	CREPE [ 32 ] + librosa	Y	Y	74	2.74	9	30	7	2	26	35.1%
Instrument detection	PANNs [ 33 ] (CNN14)	Y	Y	213	2.35	8	92	24	4	85	39.9%

Table 5: Per-tool statistics across 1,000 evaluation samples (25 enabled tools, 21 observed), scored by an LLM judge. Sorted by average usefulness score (1–5). Contribution roles: Direct = direct answer, Confirm = confirm hypothesis, Deny = deny hypothesis, Resolve = resolve conflict, Irrel. = not useful. Mus. = music-specific tool; Int. = LLM-interpreted output.

Configuration	Acc.	$\Delta$	$N_{d}$	$p$
Baseline replay	76.6%	—	—	—
Step-Audio-R1 only	72.3%	$-4.3$	109	$<$ .001
Qwen3-Omni only	73.4%	$-3.2$	110	.003

Table 6: Ablation results (argumentation-level replay).

\Delta

= accuracy change vs. baseline in pp;

N_{d}

= discordant pairs;

p

= p -value of McNemar’ test.

Audio QA

Audio question answering uses spoken, musical, and environmental sounds to answer questions grounded in acoustic evidence.

See all 45 papers

EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning
Jun 13, 2026Siyuan Zhang, Jian Zong, Junyu Wang +7Audio QA Tool-Augmented Language Model Agents
School of Artificial Intelligence, Tianjin University, Tianjin, China
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio-Language Model Evaluation Audio Reasoning
Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Oct 7, 2026Pooneh Mousavi, Mirco Ravanelli, Cem SubakanAudio-Language Models Audio Reasoning
Concordia University · Mila – Quebec AI Institute · Laval University

Jun 13, 2026J/K move · Enter open · S save

EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning

Siyuan Zhang, Jian Zong, Junyu Wang, Peiyuan Jiang+6

School of Artificial Intelligence, Tianjin University, Tianjin, China

Audio QA Tool-Augmented Language Model Agents Audio Reasoning Audio-Language Models LLM Agents

Open paper AskPDF arXiv

While LALMs show promise on audio question answering, they fail to focus on question-relevant segments of audio and provide a clear, checkable reasoning process when dealing with complex audio reasoning. Reinforcement learning and tool-augmented prompting can help models better relate questions to audio but lack a reliable way to understand, integrate, and self-verify audio segments. To address this gap, we present EChO-Agent, a modular agent framework that reformulates complex audio QA as a planning, tool execution, evidence integration, and answer verification workflow. Experiments on MMAR benchmark show EChO-Agent improves both accuracy and rubric scores over baseline and ablation studies show evidence integration is the key factor.

Abstract

Audio QA

Explore similar work

EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs