Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

Mohammad Khalooei; Mohammad Sabokrou

cs.AIOct 9, 2026

Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

Authors: Mohammad Khalooei, Mohammad Sabokrou

Organizations: Sharif University of Technology, Iran · Amirkabir University of Technology, Iran · New Uzbekistan University, Uzbekistan · Okinawa Institute of Science and Technology, Japan

Abstract

Large audio-language models (LALMs) frequently exhibit object hallucination, affirming absent sounds. The dominant mitigation, Audio-Aware Decoding (AAD), contrasts logits under real audio against an all-zero "blank" waveform, flagged by prior review as out-of-distribution. We introduce a black-box causal test that isolates the claimed evidence itself: localize a claim's evidence region via CLAP similarity, remove it by spectral gating, and compare against a random-window control across a 2x2x2 grid on three LALMs. On Qwen2-Audio-7B, localized ablation's raw AUROC trails blank's (0.613 vs. 0.721); this is a duration confound, not a localization failure, since blank removes the entire clip versus the grid's 2-4 s. Holding duration fixed, localization beats random removal at every cell on both models. Raw detection value remains model-dependent, especially on Audio Flamingo 3. The same evidence corrects decoding: contrastive decoding combining blank and language-prior contrasts, gated and ensembled, raises Qwen2-Audio accuracy from 57.1% to 71.9%, beating AAD alone. A unified causal-grounding profile does not transfer between models, indicating different grounding regimes. The source code and a detailed report of our findings, including identified limitations, are available at https://github.com/khalooei/causa.

Figures & tables

Method	AUROC
CAUSA causal grounding ensemble (5-feature, § 3.6 )	0.743
AAD blank-audio ablation (baseline) [ 10 ]	0.721
CAUSA localized, 4 s / $-100$ dB (best single cell)	0.613
CAUSA random-window control, 4 s / $-100$ dB	0.577
CAUSA localized, 2 s / $-40$ dB (original config)	0.563
CAUSA random-window control, 2 s / $-40$ dB	0.550

Table 1: Hallucination-detection AUROC,

n=4355

(1867 hallucinated, 2488 correct).

	Oracle	CLAP	Random
Flip rate	16.8%	10.5%	3.1%
Mean $P(\text{yes})$ drop	0.126	0.079	0.016

Table 2: Oracle control, synthetic Qwen2-Audio positives with known evidence location (

n{=}608

-100

dB, matched size).

Figure 1: Qwen2-Audio contrastive-decoding accuracy vs.

\alpha

: combining AAD+LangPrior (ours) sits above every single contrast across the whole range, and confidence-gating it (dotted line, CV) higher still (Audio Flamingo 3’s sweep is flat at a

93.0\%

ceiling for every condition and is omitted; see text).

Model (raw acc.)	Contrast	Acc.	$\Delta$	Best
Qwen2-Audio (57.1%)	LangPrior-CD	68.2%	$+11.1$	68.8%
	AAD-CD	64.7%	$+7.5$	68.9%
	CAUSA-CD (loc. only)	59.6%	$+2.5$	62.8%
	AAD+LangPrior	70.0%	$+12.9$	71.3%
	+ gated	71.4%	$+14.3$	—
	+ ensemble prior (ours)	71.9%	+14.8	—

Table 3: Contrastive-decoding accuracy at

\alpha{=}1

, on each model’s own affirmative claims. Best is swept over the literature-standard

\alpha\in[0,3]

(Section 3.4 ).

Model	Setting	Localized	Random	Gap
Qwen2-Audio	4 s/ $-100$ dB	3.3%	2.2%	$+1.2$
Qwen2.5-Omni	4 s/ $-100$ dB	7.4%	3.9%	$+3.5$
Audio Flamingo 3	4 s/ $-100$ dB	62.5%	41.5%	$\mathbf{+21.0}$
Qwen2-Audio	2 s/ $-40$ dB	1.0%	0.8%	$+0.2$
Qwen2.5-Omni	2 s/ $-40$ dB	1.5%	0.8%	$+0.7$
Audio Flamingo 3	2 s/ $-40$ dB	26.0%	18.8%	$\mathbf{+7.1}$

Table 4: Necessity flip rate, localized vs. random, by model (

n{=}510

1022

1109

Qwen2-Audio/Qwen2.5-Omni/AF3 true positives).

Audio-Language Models

Audio-language models connect audio and natural language for understanding, retrieval, dialogue, speech generation, and music or sound analysis.

See all 300 papers

TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models
Sep 7, 2026Heyu Chang, Nianwen Si, Hao Zhang +2Audio QA Audio-Language Model Hallucination Detection
Information Engineering University, China
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models
Sep 27, 2026Zhi Wen Soi, Giulio Segalini, Jian-Jia Chen +1Audio-Language Model Evaluation Audio-Language Model Hallucination Detection
University of Neuchâtel · TU Dortmund University · RWTH Aachen University +1
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
Apr 21, 2026Feiyu Zhao, Yiming Chen, Wenhuan Lu +3Audio-Language Model Evaluation Audio-Language Model Hallucination Detection
College of Intelligence and Computing, Tianjin University, China · ASUS Intelligent Cloud Services, Singapore

Sep 7, 2026J/K move · Enter open · S save

TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models

Heyu Chang, Nianwen Si, Hao Zhang, Wenlin Zhang+1

Information Engineering University, China

Audio QA Audio-Language Model Hallucination Detection Audio-Language Models Contrastive Decoding

Open paper AskPDF arXiv

Large audio-language models (LALMs) can hallucinate audio objects, answering "yes" to absent sound events, thus undermining reliability in audio question answering. We propose Token-Adaptive Decoding (TAD), a training-free strategy for hallucination mitigation that grounds the initial yes/no decision by contrasting logits under real audio with a matched silent reference. TAD introduces a token-adaptive, confidence-guided gate that is decision-critical at the first decoding step and class-conditional on affirmative tokens, using the audio-silent margin to avoid overcorrection when evidence is weak or already sufficient. Experiments on AudioCaps-Hallucination show that, relative to Audio-Aware Decoding (AAD), a contrastive baseline with fixed contrast strength, TAD improves F1 for Qwen2 by 0.059 to 0.117 across Popular, Adversarial, and Random splits, and for Gemma by 0.025 to 0.064, while on Clotho-AQA it raises F1 from 0.810 to 0.816 on Qwen2 and remains comparable to AAD on Gemma.

Abstract

Audio-Language Models

Explore similar work

TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models

MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models

HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models