CHORD: Cross-Model Hallucination Detection via Relational Graph Discrimination

Yongxin Deng; Zhen Fang; Guansong Pang; Sharon Li; Ling Chen

cs.AIOct 28, 2026

CHORD: Cross-Model Hallucination Detection via Relational Graph Discrimination

Authors: Yongxin Deng, Zhen Fang, Guansong Pang, Sharon Li, Ling Chen

Organizations: University of Technology Sydney · Singapore Management University · University of Wisconsin-Madison

Abstract

Hallucination detection is critical for deploying large language models (LLMs) in real-world applications. Due to strong empirical performance, internal representation-based methods have emerged as the prevailing direction for detecting hallucinations, yet they remain largely model-specific and often fail to generalize to unseen LLMs. In this paper, we study an important yet underexplored problem, termed cross-model hallucination detection (CMHD), which aims to train hallucination detectors on source LLMs while ensuring performance on unseen target LLMs. The core challenge of CMHD lies in cross-model representation heterogeneity: hidden states from different LLMs exhibit severe semantic inconsistency, which limits the transferability of classical representation-based detectors. Through analysis and empirical validation, we show that relational structures over layers, tokens, and features capture transferable detection signals despite semantic inconsistency. Based on this observation, we propose cross-model hallucination detection via heterogeneity-oriented relational discrimination (CHORD), a relational graph discrimination framework that represents hidden states as joint relational graphs over layers, tokens, and features. It feeds these graphs into relation-aware attention to obtain structure-aware features, and uses meta-learning to shape these features toward transferable detection signals. Extensive experiments show that CHORD outperforms representative detection methods in cross-model generalization. Code is available at https://github.com/MiracleworkerD/CHORD.

Figures & tables

Figure 1: Overview of Chord . Chord maps hidden states into a shared space, constructs a layer–token–block graph , and uses relation-aware attention with graph pooling for hallucination prediction. Feature-Critic meta-learning encourages the detector to learn transferable signals.

Figure 2: Cross-model alignment and hallucination detection transfer. Raw hidden states show weak local neighborhood agreement and near-chance transfer, while relational constructions improve both mutual

k

NN overlap and source-to-target AUROC. Experimental details are offered in Appendix A.8 .

Figure 3: Detector score distributions for Llama 8B to Qwen 8B. Token-level relations already separate factual and hallucinated examples better than PPL, while stronger relational constructions approach the Semantic Entropy baseline. Experimental details are provided in Appendix A.8 .

Training-free methods
Method	Target Model
	Llama3.2-3B								Llama3.1-8B
	Mean AUROC $\uparrow$								Mean AUROC $\uparrow$
	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average
Perplexity	0.6329	0.6464	0.6448	0.6296	0.6536	0.6188	0.6621	0.6412	0.6546	0.6452	0.6545	0.6395	0.6523	0.6024	0.6857	0.6477
Semantic Entropy	0.6862	0.7095	0.7006	0.6934	0.7112	0.6571	0.7223	0.6972	0.6991	0.7113	0.7065	0.6919	0.7258	0.6636	0.7438	0.7060

Table 1: Cross-model performance (AUROC). To ensure fair comparison, we use a leave-one-family-out protocol: training-based methods are trained on models from two source families and evaluated on models from the held-out target family, while training-free methods are directly applied to the same targets ( Appendix G ). Best results are bold , and second-best results are underlined .

Method	Target Model
	Llama3.2-3B					Llama3.1-8B					Llama3.3-70B
	Mean AUROC $\uparrow$					Mean AUROC $\uparrow$					Mean AUROC $\uparrow$
	QA Avg.	GSM8K	MBPP	xLAM	All Avg.	QA Avg.	GSM8K	MBPP	xLAM	All Avg.	QA Avg.	GSM8K	MBPP	xLAM	All Avg.
Best training-free	0.7031	0.7177	0.6915	0.7253	0.7067	0.7119	0.7293	0.6865	0.7469	0.7158	0.7372	0.7473	0.7181	0.7477	0.7374
Best training-based	0.5209	0.5300	0.5035	0.5397	0.5208	0.5225	0.5382	0.4876	0.5395	0.5212	0.5278	0.5375	0.4905	0.5386	0.5232
CHORD w/o RelAttn	0.6901	0.7241	0.6458	0.7259	0.6937	0.7023	0.7403	0.6678	0.7916	0.7156	0.7373	0.7646	0.7017	0.7821	0.7425

Table 2: Ablation study of Chord (AUROC). The comparison protocol follows Table 1 . In the Best training-free and Best training-based rows, each benchmark column reports the strongest baseline of that group on that benchmark, so different columns may come from different baselines; the QA Avg. and All Avg. columns report the averages of the single strongest baseline of the group, and are therefore not the row-wise means of the columns beside them. Full results are provided in Appendix G . Best results are bold , and second-best results are underlined .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Pilot analysis of cross-model transfer failure in hallucination detection. (a-c) Transfer AUROC matrices for representative training-based methods (MM [ 56 ] , SEP [ 6 ] , SAPLMA [ 57 ] ). While all methods achieve strong in-model performance (diagonal), they suffer severe degradation when across models (off-diagonal), dropping to near-random accuracy. See Appendix A.1 for details.

Figure 5: Token and layer count sweeps for the observational diagnostic and the full Chord method. The diagnostic setting benefits from a moderately larger window, reaching its strongest region around

T=7

and

L=5

. Chord recovers most of its performance with a compact

T=3,L=3

setting, after which larger windows provide only limited additional gain.

Figure 6: Token and layer position sweeps. Late generated tokens perform best across both the observational diagnostic and Chord . Middle layers are also consistently strong, supporting the use of late-token and middle-layer windows in the main experiments.

Figure 7: Effect of feature-block count

B

on the observational joint-relation diagnostic and the full Chord method. Hidden states are padded to

d_{\max}=8192

and partitioned into

B

contiguous blocks. Very small

B

produces overly coarse blocks, while very large

B

makes each node too low-dimensional and greatly increases graph size. The default

B=64

provides a strong trade-off between transfer performance and graph complexity.

Figure 8: Full mutual $k$ nearest neighbor alignment matrices across all model pairs. For each feature view, we compute the average overlap between the $k$ nearest neighbor sets induced by two models over the same evaluation examples. Since this metric is symmetric, we show the upper triangular matrix and set the diagonal to one. Across the full model suite, relational views consistently yield stronger local-neighborhood agreement than raw hidden states, supporting the representative trends shown in the main text.

Figure 9: Full directional cross-model hallucination transfer matrices. Rows denote source models used for training the detector, and columns denote target models used for evaluation. Unlike representational alignment, transfer AUROC is directional, so we show the full matrix. Raw hidden states remain close to chance for most source-to-target pairs, whereas relational views produce substantially stronger transfer across model families and scales.

Figure 10: Representative detector score distributions across source-to-target model pairs. Each row corresponds to one representative transfer pair, and each column corresponds to one feature view. Curves show the score distributions of factual and hallucinated examples on the target model, and the annotated AUROC is recomputed from the plotted sample. The main text KDE plot is one representative case; here we show that the same qualitative pattern holds across the selected same-family, cross-family, scale-matched, and scale-mismatched settings.

Figure 11: Additional diagnostics for the observation analysis. (A) Rank correlations between feature-view orderings induced by mutual

k

nearest neighbor alignment and by transfer AUROC. The moderate correlations indicate that alignment and downstream utility are related but not equivalent. Because the rank correlations are computed over five feature views, the possible Spearman and Kendall values are discrete. (B) Cross-scale transfer is mildly asymmetric: detectors trained on larger source models transfer slightly better to smaller target models than the reverse direction. (C) Raw hidden states exhibit higher seed variance in transfer AUROC, while relational views are more stable across random seeds.

Figure 12: Alignment–utility decoupling. Each point denotes one relational construction on a representative source-to-target pair. Token-level relations often achieve high mutual

k

NN alignment, whereas joint relational structures yield stronger transfer AUROC. The most stable relational structure is not necessarily the most useful for hallucination detection.

Figure 13: Hand-designed relation edges improve transfer, but their gains saturate. Simple relation edges are compared with refined handcrafted variants for token, layer, and intra-block relations. Across representative model pairs, refined edges improve both local neighborhood alignment and source-to-target AUROC. The improvements are smaller than the gain from moving from raw hidden states to relational views and are not uniform across relation types. Fixed graphs are useful diagnostics, but their instance-independent structural bias motivates learned relation aggregation. Here P1 to P5 denote Llama-3B

\rightarrow

Llama-8B, Qwen-14B

\rightarrow

Qwen-8B, Llama-8B

\rightarrow

Qwen-8B, Llama-70B

\rightarrow

Qwen-35B-A3B, and Llama-3B

\rightarrow

Ministral-8B.

Figure 14: Channel attention diagnostics. A lightweight channel probe is trained on source-model relational features and evaluated on target-model examples, with all relational features frozen. The four channel weights sum to one for each example. Panels (A) and (B) show label-conditioned mean profiles and per-example channel-weight variation. Panel (C) shows pair-specific deviations after subtracting each pair’s mean channel weight, so colors emphasize relative channel preference within each transfer pair. Panels (D) and (E) report normalized entropy and maximum channel weight on separate axes. Panel (F) shows the dominant channel frequency. Together, these diagnostics indicate that the useful relation mixture is not well described by a single fixed channel weighting.

Training-free methods
Method	Target Model
	Llama3.2-3B								Llama3.1-8B
	Mean AUROC $\uparrow$								Mean AUROC $\uparrow$
	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average
Perplexity	0.6329	0.6464	0.6448	0.6296	0.6536	0.6188	0.6621	0.6412	0.6546	0.6452	0.6545	0.6395	0.6523	0.6024	0.6857	0.6477
Semantic Entropy	0.6862	0.7095	0.7006	0.6934	0.7112	0.6571	0.7223	0.6972	0.6991	0.7113	0.7065	0.6919	0.7258	0.6636	0.7438	0.7060

Appendix

Table 4: Full ablation results under the cross-family evaluation protocol. Values are AUROC. Best results are bold and second-best results are underlined .

Training-free methods
Method	Scale-Group Transfer
	Small $\rightarrow$ Medium								Medium $\rightarrow$ Small
	Mean AUROC $\uparrow$								Mean AUROC $\uparrow$
	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average
Perplexity	0.6371	0.6580	0.6457	0.6438	0.6444	0.6097	0.6743	0.6447	0.6416	0.6411	0.6588	0.6407	0.6560	0.6175	0.6747	0.6472
Semantic Entropy	0.6846	0.6946	0.6986	0.6944	0.7148	0.6486	0.7239	0.6942	0.6880	0.6952	0.7087	0.6928	0.7140	0.6533	0.7307	0.6975

Appendix

Table 5: Cross-size transfer results. Models are grouped by parameter scale; training-based methods are trained on source-size-group models and evaluated on target-size-group models. Values are AUROC averaged over target models in the target group. Best results are bold and second-best results are underlined .

Training-free methods
Method	Target Model
	Llama3.2-3B								Llama3.1-8B
	Mean AUROC $\uparrow$								Mean AUROC $\uparrow$
	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average
Perplexity	0.6329	0.6464	0.6448	0.6296	0.6536	0.6188	0.6621	0.6412	0.6546	0.6452	0.6545	0.6395	0.6523	0.6024	0.6857	0.6477
Semantic Entropy	0.6862	0.7095	0.7006	0.6934	0.7112	0.6571	0.7223	0.6972	0.6991	0.7113	0.7065	0.6919	0.7258	0.6636	0.7438	0.7060

Appendix

Table 6: Full edge-ablation results under the cross-family evaluation protocol. Values are AUROC. Best results are bold and second-best results are underlined .

Training-free methods
Method	Target Model
	Llama3.2-3B								Llama3.1-8B
	Mean AUROC $\uparrow$								Mean AUROC $\uparrow$
	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average	TruthfulQA	HaluEval	TriviaQA	CoQA	GSM8K	MBPP	xLAM	Average
Perplexity	0.6329	0.6464	0.6448	0.6296	0.6536	0.6188	0.6621	0.6412	0.6546	0.6452	0.6545	0.6395	0.6523	0.6024	0.6857	0.6477
Semantic Entropy	0.6862	0.7095	0.7006	0.6934	0.7112	0.6571	0.7223	0.6972	0.6991	0.7113	0.7065	0.6919	0.7258	0.6636	0.7438	0.7060

Appendix

Table 7: Cross-dataset hallucination detection performance under leave-one-benchmark-out evaluation. Training-based methods are trained on six source benchmarks and evaluated on the held-out target benchmark for each target model, while training-free methods are directly applied to the same targets. The table is split into four blocks only due to space constraints. We report AUROC on each held-out benchmark and the average across benchmarks for each target model. Best results are bold , second-best are underlined .

Hallucination Detection

Hallucination detection identifies unsupported, fabricated, or evidence-inconsistent content in language and multimodal model outputs.

See all 79 papers

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?
Jul 4, 2026Aisha Alansari, Malak Alkhorasani, Hamzah LuqmanMultilingual Language Model Evaluation LLM Hallucination Detection
King Fahd University of Petroleum and Minerals · Imam Abdulrahman bin Faisal University
External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
Oct 1, 2026Kingshuk Gupta, Davide BuscaldiHallucination Detection LLM Hallucination Detection
LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord
Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
Sep 4, 2025Malavika Suresh, Rahaf Aljundi, Ikechukwu Nkisi-Orji +1LLM Hallucination Detection LLM Reliability
Robert Gordon University, Aberdeen, United Kingdom · Toyota Motor Europe, Brussels, Belgium

Jul 4, 2026J/K move · Enter open · S save

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

Aisha Alansari, Malak Alkhorasani, Hamzah Luqman

King Fahd University of Petroleum and Minerals · Imam Abdulrahman bin Faisal University

Multilingual Language Model Evaluation LLM Hallucination Detection Language Model Probing

Open paper AskPDF arXiv

Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal-state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection using internal representations from six LLMs on the generative question-answering task. We conduct a systematic Arabic <-> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross-lingual transfer, cross-domain transfer, and combined cross-lingual and cross-domain transfer. The results reveal that internal-state hallucination signals in LLMs transfer across languages and domains for most models, with cross-lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross-domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.

Abstract

Hallucination Detection

Explore similar work

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

Cross-Layer Attention Probing for Fine-Grained Hallucination Detection