Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models

Yu Zhang; Pingrui Zhang; Xuefeng Bai; Pengfei Zhang; Yang Xiang; Kehai Chen

cs.CLSep 29, 2026

Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models

Authors: Yu Zhang, Pingrui Zhang, Xuefeng Bai, Pengfei Zhang, Yang Xiang, Kehai Chen

Organizations: Harbin Institute of Technology, Shenzhen, China

Abstract

Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, recent studies show that AVLLMs face a critical challenge: $\textbf{source-confused grounding hallucination}$ , where cues from the unused modality induce responses that the required modality does not support, undermining reliability in real-world applications. Existing methods have made progress in mitigating this failure, yet how it arises from internal cross-modal interactions remains insufficiently understood. To address this gap, we conduct path-intervention and representation analyses, revealing a $\textbf{question-relay}$ mechanism: question states carry interfering cues alongside required-source evidence, undermining grounding in required-modality evidence. Cutting pathways from interfering modality to question states yields greater correct-answer logit recovery than cutting those to the generation position. Motivated by these findings, we propose $\textbf{SECRET}$ ( $\textbf{S}$ ourc $\textbf{E}$ - $\textbf{C}$ onditioned $\textbf{RE}$ lay s $\textbf{T}$ eering), a training-free method that mitigates cross-modal interference at the question relay. Using contrasting question representations elicited through different modality-pathway interventions, SECRET steers the original question states toward required-source evidence. Experiments on two widely adopted benchmarks CMM and AVHBench across three AVLLMs show that SECRET consistently outperforms prior training-free methods, substantially mitigating source-confused grounding hallucinations (e.g., up to +18.0 and +7.1 percentage points over base models). Modality-specific captioning further demonstrates its generalizability to open-ended generation.

Figures & tables

Figure 1: (a) Source-confused grounding hallucination in the audio-required setting. The audio contains human speech but no piano music, yet the visible piano leads the model to hallucinate piano music. (b) Question-relay mechanism. Question states relay interfering cues alongside required-source evidence, allowing non-required information to influence source-specific answers.

Figure 2: Layer-wise effects of cutting attention pathways from the tokens of required modality

X_{r}

to the tokens of interfering modality

X_{\bar{r}}

, question tokens

X_{Q}

and the generation position

X_{G}

for source-faithful prediction. Cutting the attention pathway from the tokens of required-modality to question tokens produces the largest reduction, suggesting that the pathway is the most important for source-faithful prediction.

Panel of Figure 3 — Figure 3: Interfering cues in question states and their effects on hallucination predictions. (a) Original is the unmodified run; Intervened cuts pathways from interfering-modality into question states. Left: layer-wise fraction of examples with lower target-object scores in Intervened than in Original , exceeding 90% at every tested layer. Right: mean layer-wise target-object scores across all samples, showing lower scores after intervention. (b) Question Cut and Generation Cut cut pathways from interfering-modality to question states and the generation position, respectively. For each intervention, $\Delta$ Logit is the correct-answer output logit at $X_{G}$ after intervention minus that in Original . Question Cut yields greater recovery over most tested layers.

Figure 4: Overview of the proposed Secret . Bottom: The AVLLM predicts the required modality from the question alone to construct $\mathbf{M}^{+}$ and $\mathbf{M}^{-}$ , which cut attention from the interfering and required modalities to question tokens, respectively. Top: Three parallel branches with shared parameters process the same input through the first $L_{1}$ layers, yielding positive, original, and negative question states. Each token-wise positive–negative difference is scaled to the original state’s L2 norm and added to it. Updated question states and original audio-visual states pass through the remaining $L_{2}$ layers for generation.

Method	CMM			AVHBench
	Visual Dom.	Audio Dom.	Overall Acc.	Video-Driven	Audio-Driven	Overall Acc.
	Visual Dom.	Audio Dom.	Overall Acc.	Audio Hall.	Video Hall.	Overall Acc.
VideoLLaMA2-AV-7B	71.8	80.0	75.9	75.7	79.0	77.4
+ VCD (CVPR’24)	71.3	83.3	77.3	66.0	74.8	70.4
+ AVCD (NeurIPS’25)	71.8	84.0	77.9	78.3	80.3	79.3
+ MAD (CVPR’26)	82.3	84.3	83.3	79.7	79.1	79.4

Table 1: Main results on CMM and AVHBench in accuracy (%). Overall Acc. denotes the mean of the two subset accuracies for each benchmark. Values in parentheses indicate absolute gains in percentage points over the corresponding base model.

Panel of Figure 5 — Figure 5: Ablation and representation analysis on CMM. (a) Overall accuracy of Secret , its three intervention variants, and the original AVLLMs. (b) Overall accuracy across relay-intervention depths $L_{1}$ , with question steering applied after the first $L_{1}$ layers. (c) Cosine similarity between the positive and negative question representations at the corresponding depths.

Method	Audio-target Caption			Video-target Caption
Method	T-CIDEr $\uparrow$	D-CIDEr $\downarrow$	LLM-score $\uparrow$	T-CIDEr $\uparrow$	D-CIDEr $\downarrow$	LLM-score $\uparrow$
Qwen2.5-Omni-7B	17.7	5.17	3.12	29.3	3.70	3.57
+Gen-Interv.	17.2	5.10	3.05	30.2	3.10	3.79
+Removal-Interv.	16.5	5.43	3.55	31.1	2.67	3.49
+ Secret	17.4	4.59	3.76	31.8	2.66	4.01
VideoLLaMA2-AV-7B	13.9	20.2	2.93	32.9	5.43	3.27

Table 2: Source-grounding evaluation in audio-target and video-target captioning. T-CIDEr

\uparrow

measures target-reference agreement; D-CIDEr

\downarrow

measures distractor-reference overlap; LLM-score

\uparrow

jointly assesses target fidelity and distractor leakage.

Panel of Figure 6 — Figure 6: Fine-grained evaluation on CMM. (a) Answer-margin distributions, where answer-margin is the correct-answer logit minus the incorrect-answer logit. Diamonds and orange lines denote means and medians; whiskers span min–max. (b) Accuracy across the reported video-duration groups. The results show that Secret maintains substantial gains on longer clips.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Robustness to attention-cutting window size in Qwen2.5-Omni-7B. Layer-wise changes in target-answer probability for source-faithful examples in Qwen2.5-Omni-7B. The horizontal axis indicates the window center. Across both settings and window sizes, cutting $X_{r}\!\rightarrow\!X_{Q}$ produces the most pronounced reduction in the middle layers.

Panel of Figure 8 — Figure 8: Robustness to attention-cutting window size for VideoLLaMA-AV-7B. Layer-wise changes in target-answer probability for source-faithful examples in VideoLLaMA-AV-7B. The horizontal axis indicates the window center. Across both settings and window sizes, cutting $X_{r}\!\rightarrow\!X_{Q}$ produces the most pronounced reduction in the middle layers.

Figure 9: PCA visualization of question representations in Qwen2.5-Omni-7B across layers 21–25 (from left to right). Blue circles and orange stars denote positive and negative representations, respectively; lines connect paired representations.

Figure 10: PCA visualization of question representations in VideoLLaMA2-AV-7B across layers 21–25 (from left to right). Blue circles and orange stars denote positive and negative representations, respectively; lines connect paired representations.

Explore similar work

Probing Cross-modal Information Hubs in Audio-Visual LLMs
May 11, 2026Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1Large Audio Language Models Cross-Modal
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea
Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
May 11, 2026Xuanchen Li, Yuheng Lu, Chenrui Cui +6Audio-Visual Reasoning LLM Reasoning Strategies
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin, China · Tencent, China · Huiyan Technology Company, Ltd., Tianjin, China +1
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Large Language Models Modality-Specific Encoders
Integrated Vision Language Lab, KAIST, South Korea

May 11, 2026, cs.AIJ/K move · Enter open · S save

Probing Cross-modal Information Hubs in Audio-Visual LLMs

Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim, Joon Son Chung

Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea

Large Audio Language Models Cross-Modal

Open paper AskPDF arXiv

Audio-visual large language models (AVLLMs) have recently emerged as a powerful architecture capable of jointly reasoning over audio, visual, and textual modalities. In AVLLMs, the bidirectional interaction between audio and video modalities introduces intricate processing dynamics, necessitating a deeper understanding of their internal mechanisms. However, unlike extensively studied text-only or large vision language models, the internal workings of AVLLMs remain largely unexplored. In this paper, we focus on cross-modal information flow between audio and visual modalities in AVLLMs, investigating where information derived from one modality is encoded within the token representations of the other modality. Through an analysis of multiple recent AVLLMs, we uncover two common findings. First, AVLLMs primarily encode integrated audio-visual information in sink tokens. Second, sink tokens do not uniformly hold cross-modal information. Instead, a distinct subset of sink tokens, which we term cross-modal sink tokens, specializes in storing such information. Based on these findings, we further propose a simple training-free hallucination mitigation method by encouraging reliance on integrated cross-modal information within cross-modal sink tokens. Our code is available at https://github.com/kaistmm/crossmodal-hub.

Abstract

Explore similar work

Probing Cross-modal Information Hubs in Audio-Visual LLMs

Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought

MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models