Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence

Yunuo Wang; Maoyuan Chen; Yuanjin Huang; Xing Liu

cs.SDOct 27, 2026

Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence

Authors: Yunuo Wang, Maoyuan Chen, Yuanjin Huang, Xing Liu

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · School of Computer Science, Wuhan University, Wuhan, China · National University of Singapore, Singapore · Northeastern University, Toronto, Canada

Abstract

For audio-language models, an answer change after a local sound-event edit can reflect a response to the event's class, sensitivity to waveform rewriting, or both. We compare matched same-class and different-class replacements of a queried event. Both use the same interval, level estimate, donor fitting, and crossfade, while the same-class replacement preserves the correct answer. Across 600 synthetic soundscapes and three models from two architecture families, different-class replacements produce 49.7 to 60.9 percentage points more answer changes than same-class replacements. Same-class accuracy remains within 5.2 percentage points of original accuracy. Among trials with correct original and same-class answers, soundscape-weighted selective accuracy for the inserted class is 73.7 to 81.9%, averaging eligible trials within each soundscape and then weighting eligible soundscapes equally. This protocol tests how model answers respond to local changes in event class.

Figures & tables

Figure 1: Changing the queried event’s class elicits more answer change than rewriting the same interval with another recording of that class. Nuisance and sham edits produce smaller responses. The rightmost group shows the paired gap

\Delta

and its 95% interval. The bootstrap resamples soundscapes with both option orders and all conditions kept together.

Model	Different	Same	Nuisance	Background-	Silence	Sham	$\Delta$ (95% CI)
	class	class	removal	window
Qwen2.5-Omni-7B	73.17	12.25	5.50	1.33	44.42	0.58	60.92 [57.25, 64.58]
Qwen2.5-Omni-3B	70.25	12.50	7.08	1.50	42.75	0.92	57.75 [54.00, 61.67]
Fun-Audio-Chat-8B	65.33	15.67	7.50	1.08	46.00	0.17	49.67 [45.33, 53.92]

Table 1: Answer changes relative to the original answer (%) across 1,200 trials per model. Each of 600 soundscapes contributes two option orders and six interventions.

\Delta

is the different-class minus same-class answer-change gap. The 95% bootstrap intervals resample paired soundscapes, keeping both option orders and intervention conditions together.

Figure 2: Class-level original accuracy versus the different-class minus same-class answer-change gap. Each point represents one of 33 query classes for one model. Both quantities use the same evaluation trials.

Model	Original	Same-class	Different-class	Selective	Eligible	Eligible
	accuracy	accuracy	accuracy	accuracy	trials	soundscapes
Qwen2.5-Omni-7B [ 1 ]	.882	.859	.845	.819	971	542
Qwen2.5-Omni-3B [ 1 ]	.853	.841	.838	.801	941	544
Fun-Audio-Chat-8B [ 2 ]	.847	.795	.800	.737	891	481

Table 2: Correct identification before and after replacement. Selective accuracy averages eligible trials within each soundscape and then weights eligible soundscapes equally. Eligibility requires correct original and same-class answers. The first three accuracy columns use all 1,200 trials. Eligible trial and soundscape counts summarize coverage.

Figure 3: Empirical cumulative distributions of class-level answer-change gaps. Each curve weights the 33 query classes equally and shows the fraction with a gap at or below each threshold. Gaps are measured in percentage points.

Audio Understanding

Audio understanding enables machines to recognize speech, sounds, music, speakers, and events, then answer questions grounded in what they hear.

See all 216 papers

Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models
Jun 3, 2026Yichen Gao, Yiqun Zhang, Zijing Wang +7Audio-Language Model Evaluation Audio Understanding
Northeastern University, China · Shanghai Artificial Intelligence Laboratory, China
Learning from Audio-Dependency Errors: Data Curation Strategies Based on Model Confusion Patterns in Audio Question Answering
Jun 20, 2026Hyeonuk NamAudio-Language Model Evaluation Modality Reliance in Multimodal Models
Independent Researcher · Seoul, South Korea
Tracing Audio Grounding and Answer Selection in Audio LLMs
Sep 4, 2026Hyebin Cho, Suho Yoo, Jihoo Jung +1Audio-Language Models Audio QA
Korea Advanced Institute of Science and Technology, South Korea

Jun 3, 2026J/K move · Enter open · S save

Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models

Yichen Gao, Yiqun Zhang, Zijing Wang, Yujia Li+6

Northeastern University, China · Shanghai Artificial Intelligence Laboratory, China

Audio-Language Model Evaluation Audio Understanding Audio-Language Models Contrastive Decoding Causal Interventions in Language Models

Open paper AskPDF arXiv

Audio-language models (ALMs) often follow text that conflicts with audio, even when the audio evidence is clear. This raises a basic question: is the audio-supported answer unavailable, or is it represented but overridden by the conflicting text? We examine this question using a same-audio counterfactual that keeps the audio fixed, removes only the conflicting text, and measures the resulting shift in model preference. Across five ALMs and four conflict tasks, 64.1% of conflict samples show a sign flip: the same-audio branch prefers the audio-supported answer, whereas the joint branch prefers the text-supported answer. This pattern suggests that the relevant audio evidence is encoded but loses in arbitration. Activation patching further localizes the reversal to answer-position computation, and patching effects closely track output candidate-score differences (Spearman rho=0.93). Using this diagnostic, we propose Gated Audio Counterfactual Logit Correction (GACL), a training-free decoding rule that interpolates between joint and same-audio scores. Under a strict 5 pp faithfulness-drop budget, GACL improves nAUC by 17.8 points over the best contrastive baseline and transfers without retuning to vision-text arbitration (up to +40.5 pp).

Abstract

Audio Understanding

Explore similar work

Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models

Learning from Audio-Dependency Errors: Data Curation Strategies Based on Model Confusion Patterns in Audio Question Answering

Tracing Audio Grounding and Answer Selection in Audio LLMs