Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing

Amir Jalilifard; Anderson Rocha; Eric Wong; Marcos Medeiros Raimundo

cs.AISep 17, 2026

Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing

Authors: Amir Jalilifard, Anderson Rocha, Eric Wong, Marcos Medeiros Raimundo

Organizations: Institute of Computing, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brazil · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA

Abstract

In this work, we examine the topology of information flow patterns within attention graphs to effectively distinguish hallucinated from non-hallucinated responses. We analyze the Forman-Ricci curvature to identify structural patterns indicating information bottlenecks in attention graphs. We then introduce a method that captures both semi-local and global information-flow characteristics of attention heads associated with hallucinated responses. We evaluate our approach extensively across several LLMs and established benchmarks. Empirical results demonstrate that our proposed single-pass approach provides consistent improvements over existing attention-based and multi-response baselines across two hallucination-detection benchmarks, while achieving competitive performance across diverse LLM architectures. Further analysis reveals that impaired context sharing among tokens during causal generation is strongly associated with hallucination occurrences in LLMs. In particular, hallucinated responses are consistently characterized by an over-reliance on self-attention, diffused context retrieval from earlier tokens, or information over-squashing, especially in the final transformer layer.

Figures & tables

Fig. 1 : Information over-squashing occurs when information from many tokens is funneled through only a single token that bears the burden of representing all that context. However, this central token and its predecessors receive insufficient attention from subsequent tokens, preventing effective context sharing. In this example, all the information passes through token T4 , but later, due to high self-attention on T5 , a significant amount of information from earlier tokens is lost. The line thickness and its color show the amount of information flow from previous tokens to the subsequent ones.

Fig. 2 : The difference between the mean values of Forman-Ricci curvatures per head for hallucinated and not hallucinated responses for LlaMA. For the majority of the attention heads, hallucinated responses have lower curvature values, indicating a less fluid information flow.

Panel of Figure 3 — Fig. 3 : P-values for the per-head comparisons of the 10th percentile outgoingness scores and attention entropy across all layers for LLaMA 3.1 8B Instruct.

Panel of Figure 4 — Fig. 4 : Feature evolution across layers and comparison between hallucinated and non- hallucinated response for LLaMA 3.1 8B Instruct.

Fig. 5 : Mean per-head self-attention differences between hallucinated and non-hallucinated responses for Phi-4.

Temp	Benchmark	Model	AUC
			LapEigvals	EigenScore	Ours - setting (a)
0.1	TruthfulQA	LLaMA 3.1 8B Instruct	80.52	83.79	85.34
0.1	TruthfulQA	Phi 4	78.43	81.62	81.75
0.1	TruthfulQA	Mistral 7B Instruct v0.3	78.80	81.69	80.00
0.1	TruthfulQA	Qwen3-32B	77.36	72.17	80.17
1.0	TruthfulQA	LLaMA 3.1 8B Instruct	78.26	80.22	80.73

TABLE I : AUC Comparison Between Laplacian eigenvalues, EigenScore, and our feature set classification results. Here we show the results for setting (a) which includes our base features. The best results however, were attained using setting (c).

Benchmark	Temp	Model	q10	+ q50	+ q90	+ att_entropy
TruthfulQA	0.1	LLaMA 3.1 8B Instruct	83.10	84.05	84.15	85.34
		Phi 4	79.30	80.20	80.10	81.75
		Mistral 7B Instruct v0.3	77.50	78.20	78.35	80.00
		Qwen3-32B	78.00	78.85	78.90	80.17
	1.0	LLaMA 3.1 8B Instruct	78.50	79.60	79.50	80.73
		Phi 4	82.10	82.70	82.85	84.00

TABLE II : Cumulative Feature Ablation (AUC %) across TruthfulQA and NQOPEN datasets. Features are added sequentially.

Fig. 6 : Cumulative feature set effect for Llama 3.1 8B Instruct accross NQOPEN dataset.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Fig. 7 : The difference between the mean values of Forman-Ricci curvatures per head for hallucinated and not hallucinated responses. For the majority of the attention heads, hallucinated responses have a lower curvature values which is an indication of a less fluid information flow. This pattern is more evident accross all layers in LLaMA and Mistral. For Phi and Qwen, the last layer presents less fluid context sharing for hallucinated responses.

Panel of Figure 10 — Fig. 8 : The p-values of the differences between the mean values of Forman-Ricci curvatures per head for hallucinated and not hallucinated responses. The heads with statistical signiuficance are highlited with red lines.

Panel of Figure 11 — Fig. 9 : P-values for the per-head comparisons of the 10th-percentile outgoingness scores and attention entropy across all layers. LLaMA 3.1 8B Instruct exhibits the strongest separation across the majority of attention heads.

Panel of Figure 12 — Fig. 10 : Feature average evolution across layers and comparison between hallucinated and non-hallucinated responses. Both LLaMA 3.1 Instruct and Mistral 7B Instruct shows diffused attention whereas for Phi-4 shows little to no noticeable difference in outgoingness or attention distribution between the two response types. Also, for LLaMA 3.1 Instruct and Mistral 7B Instruct, the average of 10th percentile of the outgoingness score over attention heads is higher for hallucinated response in the terminal layers. The same pattern was observed for 50th and 90th percentiles.

Panel of Figure 13 — Fig. 11 : Mean per-head self-attention differences between hallucinated and non-hallucinated responses across the evaluated models. Unlike LLaMA 3.1 8B Instruct and Mistral 7B Instruct, Phi-4 and Qwen3-32B exhibit higher self-attention for hallucinated responses across a large proportion of layers and attention heads.

Model	Accuracy	Gwet’s AC1
LLaMA 3.1 8B Instruct	0.92	0.86
Mistral 7B Instruct v0.3	0.88	0.76
Qwen3-32B	0.87	0.82
Phi-4	0.91	0.85

Appendix

TABLE IV : The performance of the LLM-as-judge across different models’ responses.

LLM Hallucination Detection

Large language model hallucination detection identifies unsupported, incorrect, or fabricated claims in model outputs and traces.

See all 198 papers

Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention
Feb 20, 2026Siya Qi, Yudong Chen, Runcong Zhao +6Hallucination Detection LLM Hallucination Detection
Department of Informatics, King’s College London, UK · Department of Statistics, University of Warwick, UK · The Alan Turing Institute, UK +1
Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
Apr 12, 2026Jakub Binkowski, Kamil Adamczewski, Tomasz KajdanowiczLLM Hallucination Detection Attention Sinks
Department of Artificial Intelligence, Wroclaw University of Science and Technology
Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
Sep 16, 2026Shardul P. More, Tanuja S. PawarHallucination Detection LLM Hallucination Detection
Rajarambapu Institute of Technology Ishwarpur, Maharashtra, India

Feb 20, 2026J/K move · Enter open · S save

Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention

Siya Qi, Yudong Chen, Runcong Zhao, Qinglin Zhu+5

Department of Informatics, King’s College London, UK · Department of Statistics, University of Warwick, UK · The Alan Turing Institute, UK+1

Hallucination Detection LLM Hallucination Detection

Open paper AskPDF arXiv

Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view of grounding behavior. However, existing approaches typically rely on coarse summaries that fail to capture fine-grained instabilities in attention. Inspired by signal processing, we introduce a frequency-aware perspective on attention by analyzing its variation during generation. We model attention distributions as discrete signals and extract high-frequency components that reflect rapid local changes in attention. Our analysis reveals that hallucinated tokens are associated with high-frequency attention energy, reflecting fragmented and unstable grounding behavior. Based on this insight, we develop a lightweight hallucination detector using high-frequency attention features. Experiments on the RAGTruth and HalluRAG benchmarks show that our approach achieves performance gains over verification-based, internal-representation-based, and attention-based methods across models and tasks.

Abstract

LLM Hallucination Detection

Explore similar work

Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models

Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models