Hallucination in large language models (LLMs) continues to be a significant issue, particularly in tasks like question answering, where models often generate plausible yet incorrect or irrelevant information. Although various methods have been proposed to mitigate hallucinations, the relationship between attention patterns and hallucinations has not been fully explored. In this paper, we analyze the distribution of attention scores across each layer and attention head of LLMs, revealing a common and intriguing phenomenon: shallow layers of LLMs primarily rely on uniform attention patterns, where the model distributes its attention evenly across the entire sequence. This uniform attention pattern can lead to hallucinations, as the model fails to focus on the most relevant information. To mitigate this issue, we propose a training-free method called Attention Replacement Technique (ART), which replaces these uniform attention patterns in the shallow layers with local attention patterns. This change directs the model to focus more on the relevant contexts, thus reducing hallucinations. Through extensive experiments, ART demonstrates significant reductions in hallucinations across multiple LLM architectures, proving its effectiveness and generalizability without requiring fine-tuning or additional training data.
Figures & tables
Figure 1: A demonstration overview of how ART works during the decoding process, with Nh=8 and k=1 .
Figure 2: Visualization on some of attention weights of Layer 0 in Qwen2.5-7B-Instruct model when encoding a sample from GSM8K.Attention weights are divided into 3 categories: Uniform Attention , Local Attention , and Scattered Attention .
Figure 3: The detailed demonstration of applying ART-max to the second layer of Llama2-7B-Chat .
Figure 4: The structure diagram of ART replaces the uniform attention head in each attention head that predicts the next token with a local attention head. Baseline: Llama2-7B-Chat processes a sample from TruthfulQA.
Figure 5: Accuracy curves of Qwen2.5-7B-Instruct evaluated on Dablation of TruthfulQA and GSM8K subsets after masking different amounts of uniform , scattered , and local attention.
Dataset Performance (%)
Model
Method
TruthfulQA
LogiQA
CommonsenseQA
OpenBookQA
GSM8K
Avg.
Llama2-7B-Chat
Vanilla
19.0
28.4
50.8
45.9
27.2
34.3
ART-max
21.7 ( +2.7 )
29.7 ( +1.3 )
49.5 ( -1.3 )
44.9 ( -1.0 )
28.3 ( +1.1 )
34.8 ( +0.5 )
ART-mean
21.5 ( +2.5 )
30.1 ( +1.7 )
49.6 ( -1.2 )
45.2 ( -0.7 )
28.0 ( +0.8 )
34.9 ( +0.6 )
Llama3.1-8B-Instruct
Vanilla
46.5
38.8
75.5
84.4
88.1
66.7
ART-max
46.3 ( -0.2 )
39.6 ( +0.8 )
76.4 ( +0.9 )
85.4 ( +1.0 )
89.2 ( +1.1 )
67.4 ( +0.7 )
Table 1: Extended performance comparison between ART and vanilla decoding across six reasoning benchmarks. Experiments are conducted over Dtest .
Figure 6: Accuracy curves of Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct evaluated on Dablation over different k values.
LLM
Method
TruthfulQA
LogiQA
GSM8K
Qwen2.5-7B-Inst
ART
52.5
51.0
94.0
Vanilla
51.2
50.7
93.2
ART-inverse
0.0
2.0
7.5
ART-scattered
53.2
51.7
93.5
Llama3.1-8B-Inst
ART
46.0
40.5
87.0
Vanilla
46.5
38.8
88.1
Table 2: Performance comparison between ART and ART-inverse. Tests are conducted on Dablation .
Method
Method
TruthfulQA
LogiQA
GSM8K
Llama2-7B
Vanilla
19.0
28.4
27.2
Beam Search
21.4
29.2
29.5
ITI
19.9
28.2
28.1
ACT
18.7
30.3
26.7
DoLa
21.5
27.8
28.7
ART-greedy
21.5
30.1
28.5
Table 3: Performance comparison between ART and other inference intervention methods, by evaluating Llama2-7B-Chat and Llama3-8B-Instruct on Dtest . Bold and underline denote the best and second best.
Large Language Models (LLMs) frequently exhibit hallucinations, presenting a major barrier to reliability in complex reasoning tasks. While traditional detection methods rely on output-based confidence metrics, these logits are often miscalibrated by modern alignment techniques. In this paper, we investigate the temporal volatility of internal attention mechanisms as an alternative diagnostic signal for hallucination that does not depend on output calibration. By introducing an unsupervised metric for attention dispersion, we show that epistemic uncertainty leaves a measurable trace within intermediate layers, where spikes in attention entropy are associated with reasoning breakdowns. We evaluate our approach on mathematical reasoning benchmarks (GSM8K and MATH-500) using the Qwen2.5 model family (1.5B and 3B parameters), finding statistically significant AUC improvements of up to +0.076 over output-based baselines across all tested conditions. These findings suggest that attention dispersion is a promising complement to traditional hallucination detection methods, requiring further investigation across broader model families and task domains.
Shardul P. More, Tanuja S. Pawar
Rajarambapu Institute of Technology Ishwarpur, Maharashtra, India
In this work, we examine the topology of information flow patterns within attention graphs to effectively distinguish hallucinated from non-hallucinated responses. We analyze the Forman-Ricci curvature to identify structural patterns indicating information bottlenecks in attention graphs. We then introduce a method that captures both semi-local and global information-flow characteristics of attention heads associated with hallucinated responses. We evaluate our approach extensively across several LLMs and established benchmarks. Empirical results demonstrate that our proposed single-pass approach provides consistent improvements over existing attention-based and multi-response baselines across two hallucination-detection benchmarks, while achieving competitive performance across diverse LLM architectures. Further analysis reveals that impaired context sharing among tokens during causal generation is strongly associated with hallucination occurrences in LLMs. In particular, hallucinated responses are consistently characterized by an over-reliance on self-attention, diffused context retrieval from earlier tokens, or information over-squashing, especially in the final transformer layer.
Amir Jalilifard, Anderson Rocha, Eric Wong +1
Institute of Computing, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brazil · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA
Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view of grounding behavior. However, existing approaches typically rely on coarse summaries that fail to capture fine-grained instabilities in attention. Inspired by signal processing, we introduce a frequency-aware perspective on attention by analyzing its variation during generation. We model attention distributions as discrete signals and extract high-frequency components that reflect rapid local changes in attention. Our analysis reveals that hallucinated tokens are associated with high-frequency attention energy, reflecting fragmented and unstable grounding behavior. Based on this insight, we develop a lightweight hallucination detector using high-frequency attention features. Experiments on the RAGTruth and HalluRAG benchmarks show that our approach achieves performance gains over verification-based, internal-representation-based, and attention-based methods across models and tasks.
Siya Qi, Yudong Chen, Runcong Zhao +6
Department of Informatics, King’s College London, UK · Department of Statistics, University of Warwick, UK · The Alan Turing Institute, UK +1