Hallucination detection is critical for deploying large language models (LLMs) in real-world applications. Due to strong empirical performance, internal representation-based methods have emerged as the prevailing direction for detecting hallucinations, yet they remain largely model-specific and often fail to generalize to unseen LLMs. In this paper, we study an important yet underexplored problem, termed cross-model hallucination detection (CMHD), which aims to train hallucination detectors on source LLMs while ensuring performance on unseen target LLMs. The core challenge of CMHD lies in cross-model representation heterogeneity: hidden states from different LLMs exhibit severe semantic inconsistency, which limits the transferability of classical representation-based detectors. Through analysis and empirical validation, we show that relational structures over layers, tokens, and features capture transferable detection signals despite semantic inconsistency. Based on this observation, we propose cross-model hallucination detection via heterogeneity-oriented relational discrimination (CHORD), a relational graph discrimination framework that represents hidden states as joint relational graphs over layers, tokens, and features. It feeds these graphs into relation-aware attention to obtain structure-aware features, and uses meta-learning to shape these features toward transferable detection signals. Extensive experiments show that CHORD outperforms representative detection methods in cross-model generalization. Code is available at https://github.com/MiracleworkerD/CHORD.
Figures & tables
Figure 1: Overview of Chord . Chord maps hidden states into a shared space, constructs a layer–token–block graph , and uses relation-aware attention with graph pooling for hallucination prediction. Feature-Critic meta-learning encourages the detector to learn transferable signals.
Figure 2: Cross-model alignment and hallucination detection transfer. Raw hidden states show weak local neighborhood agreement and near-chance transfer, while relational constructions improve both mutual k NN overlap and source-to-target AUROC. Experimental details are offered in Appendix A.8 .
Figure 3: Detector score distributions for Llama 8B to Qwen 8B. Token-level relations already separate factual and hallucinated examples better than PPL, while stronger relational constructions approach the Semantic Entropy baseline. Experimental details are provided in Appendix A.8 .
Method
Target Model
Llama3.2-3B
Llama3.1-8B
Mean AUROC ↑
Mean AUROC ↑
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
Training-free methods
Perplexity
0.6329
0.6464
0.6448
0.6296
0.6536
0.6188
0.6621
0.6412
0.6546
0.6452
0.6545
0.6395
0.6523
0.6024
0.6857
0.6477
Semantic Entropy
0.6862
0.7095
0.7006
0.6934
0.7112
0.6571
0.7223
0.6972
0.6991
0.7113
0.7065
0.6919
0.7258
0.6636
0.7438
0.7060
Table 1: Cross-model performance (AUROC). To ensure fair comparison, we use a leave-one-family-out protocol: training-based methods are trained on models from two source families and evaluated on models from the held-out target family, while training-free methods are directly applied to the same targets ( Appendix G ). Best results are bold , and second-best results are underlined .
Method
Target Model
Llama3.2-3B
Llama3.1-8B
Llama3.3-70B
Mean AUROC ↑
Mean AUROC ↑
Mean AUROC ↑
QA Avg.
GSM8K
MBPP
xLAM
All Avg.
QA Avg.
GSM8K
MBPP
xLAM
All Avg.
QA Avg.
GSM8K
MBPP
xLAM
All Avg.
Best training-free
0.7031
0.7177
0.6915
0.7253
0.7067
0.7119
0.7293
0.6865
0.7469
0.7158
0.7372
0.7473
0.7181
0.7477
0.7374
Best training-based
0.5209
0.5300
0.5035
0.5397
0.5208
0.5225
0.5382
0.4876
0.5395
0.5212
0.5278
0.5375
0.4905
0.5386
0.5232
CHORD w/o RelAttn
0.6901
0.7241
0.6458
0.7259
0.6937
0.7023
0.7403
0.6678
0.7916
0.7156
0.7373
0.7646
0.7017
0.7821
0.7425
Table 2: Ablation study of Chord (AUROC). The comparison protocol follows Table 1 . In the Best training-free and Best training-based rows, each benchmark column reports the strongest baseline of that group on that benchmark, so different columns may come from different baselines; the QA Avg. and All Avg. columns report the averages of the single strongest baseline of the group, and are therefore not the row-wise means of the columns beside them. Full results are provided in Appendix G . Best results are bold , and second-best results are underlined .
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Pilot analysis of cross-model transfer failure in hallucination detection. (a-c) Transfer AUROC matrices for representative training-based methods (MM [ 56 ] , SEP [ 6 ] , SAPLMA [ 57 ] ). While all methods achieve strong in-model performance (diagonal), they suffer severe degradation when across models (off-diagonal), dropping to near-random accuracy. See Appendix A.1 for details.
Figure 5: Token and layer count sweeps for the observational diagnostic and the full Chord method. The diagnostic setting benefits from a moderately larger window, reaching its strongest region around T=7 and L=5 . Chord recovers most of its performance with a compact T=3,L=3 setting, after which larger windows provide only limited additional gain.
Figure 6: Token and layer position sweeps. Late generated tokens perform best across both the observational diagnostic and Chord . Middle layers are also consistently strong, supporting the use of late-token and middle-layer windows in the main experiments.
Figure 7: Effect of feature-block count B on the observational joint-relation diagnostic and the full Chord method. Hidden states are padded to dmax=8192 and partitioned into B contiguous blocks. Very small B produces overly coarse blocks, while very large B makes each node too low-dimensional and greatly increases graph size. The default B=64 provides a strong trade-off between transfer performance and graph complexity.
Figure 8: Full mutual k nearest neighbor alignment matrices across all model pairs. For each feature view, we compute the average overlap between the k nearest neighbor sets induced by two models over the same evaluation examples. Since this metric is symmetric, we show the upper triangular matrix and set the diagonal to one. Across the full model suite, relational views consistently yield stronger local-neighborhood agreement than raw hidden states, supporting the representative trends shown in the main text.
Figure 9: Full directional cross-model hallucination transfer matrices. Rows denote source models used for training the detector, and columns denote target models used for evaluation. Unlike representational alignment, transfer AUROC is directional, so we show the full matrix. Raw hidden states remain close to chance for most source-to-target pairs, whereas relational views produce substantially stronger transfer across model families and scales.
Figure 10: Representative detector score distributions across source-to-target model pairs. Each row corresponds to one representative transfer pair, and each column corresponds to one feature view. Curves show the score distributions of factual and hallucinated examples on the target model, and the annotated AUROC is recomputed from the plotted sample. The main text KDE plot is one representative case; here we show that the same qualitative pattern holds across the selected same-family, cross-family, scale-matched, and scale-mismatched settings.
Figure 11: Additional diagnostics for the observation analysis. (A) Rank correlations between feature-view orderings induced by mutual k nearest neighbor alignment and by transfer AUROC. The moderate correlations indicate that alignment and downstream utility are related but not equivalent. Because the rank correlations are computed over five feature views, the possible Spearman and Kendall values are discrete. (B) Cross-scale transfer is mildly asymmetric: detectors trained on larger source models transfer slightly better to smaller target models than the reverse direction. (C) Raw hidden states exhibit higher seed variance in transfer AUROC, while relational views are more stable across random seeds.
Figure 12: Alignment–utility decoupling. Each point denotes one relational construction on a representative source-to-target pair. Token-level relations often achieve high mutual k NN alignment, whereas joint relational structures yield stronger transfer AUROC. The most stable relational structure is not necessarily the most useful for hallucination detection.
Figure 13: Hand-designed relation edges improve transfer, but their gains saturate. Simple relation edges are compared with refined handcrafted variants for token, layer, and intra-block relations. Across representative model pairs, refined edges improve both local neighborhood alignment and source-to-target AUROC. The improvements are smaller than the gain from moving from raw hidden states to relational views and are not uniform across relation types. Fixed graphs are useful diagnostics, but their instance-independent structural bias motivates learned relation aggregation. Here P1 to P5 denote Llama-3B → Llama-8B, Qwen-14B → Qwen-8B, Llama-8B → Qwen-8B, Llama-70B → Qwen-35B-A3B, and Llama-3B → Ministral-8B.
Figure 14: Channel attention diagnostics. A lightweight channel probe is trained on source-model relational features and evaluated on target-model examples, with all relational features frozen. The four channel weights sum to one for each example. Panels (A) and (B) show label-conditioned mean profiles and per-example channel-weight variation. Panel (C) shows pair-specific deviations after subtracting each pair’s mean channel weight, so colors emphasize relative channel preference within each transfer pair. Panels (D) and (E) report normalized entropy and maximum channel weight on separate axes. Panel (F) shows the dominant channel frequency. Together, these diagnostics indicate that the useful relation mixture is not well described by a single fixed channel weighting.
Method
Target Model
Llama3.2-3B
Llama3.1-8B
Mean AUROC ↑
Mean AUROC ↑
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
Training-free methods
Perplexity
0.6329
0.6464
0.6448
0.6296
0.6536
0.6188
0.6621
0.6412
0.6546
0.6452
0.6545
0.6395
0.6523
0.6024
0.6857
0.6477
Semantic Entropy
0.6862
0.7095
0.7006
0.6934
0.7112
0.6571
0.7223
0.6972
0.6991
0.7113
0.7065
0.6919
0.7258
0.6636
0.7438
0.7060
Appendix
Table 4: Full ablation results under the cross-family evaluation protocol. Values are AUROC. Best results are bold and second-best results are underlined .
Method
Scale-Group Transfer
Small → Medium
Medium → Small
Mean AUROC ↑
Mean AUROC ↑
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
Training-free methods
Perplexity
0.6371
0.6580
0.6457
0.6438
0.6444
0.6097
0.6743
0.6447
0.6416
0.6411
0.6588
0.6407
0.6560
0.6175
0.6747
0.6472
Semantic Entropy
0.6846
0.6946
0.6986
0.6944
0.7148
0.6486
0.7239
0.6942
0.6880
0.6952
0.7087
0.6928
0.7140
0.6533
0.7307
0.6975
Appendix
Table 5: Cross-size transfer results. Models are grouped by parameter scale; training-based methods are trained on source-size-group models and evaluated on target-size-group models. Values are AUROC averaged over target models in the target group. Best results are bold and second-best results are underlined .
Method
Target Model
Llama3.2-3B
Llama3.1-8B
Mean AUROC ↑
Mean AUROC ↑
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
Training-free methods
Perplexity
0.6329
0.6464
0.6448
0.6296
0.6536
0.6188
0.6621
0.6412
0.6546
0.6452
0.6545
0.6395
0.6523
0.6024
0.6857
0.6477
Semantic Entropy
0.6862
0.7095
0.7006
0.6934
0.7112
0.6571
0.7223
0.6972
0.6991
0.7113
0.7065
0.6919
0.7258
0.6636
0.7438
0.7060
Appendix
Table 6: Full edge-ablation results under the cross-family evaluation protocol. Values are AUROC. Best results are bold and second-best results are underlined .
Method
Target Model
Llama3.2-3B
Llama3.1-8B
Mean AUROC ↑
Mean AUROC ↑
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
TruthfulQA
HaluEval
TriviaQA
CoQA
GSM8K
MBPP
xLAM
Average
Training-free methods
Perplexity
0.6329
0.6464
0.6448
0.6296
0.6536
0.6188
0.6621
0.6412
0.6546
0.6452
0.6545
0.6395
0.6523
0.6024
0.6857
0.6477
Semantic Entropy
0.6862
0.7095
0.7006
0.6934
0.7112
0.6571
0.7223
0.6972
0.6991
0.7113
0.7065
0.6919
0.7258
0.6636
0.7438
0.7060
Appendix
Table 7: Cross-dataset hallucination detection performance under leave-one-benchmark-out evaluation. Training-based methods are trained on six source benchmarks and evaluated on the held-out target benchmark for each target model, while training-free methods are directly applied to the same targets. The table is split into four blocks only due to space constraints. We report AUROC on each held-out benchmark and the average across benchmarks for each target model. Best results are bold , second-best are underlined .
Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal-state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection using internal representations from six LLMs on the generative question-answering task. We conduct a systematic Arabic <-> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross-lingual transfer, cross-domain transfer, and combined cross-lingual and cross-domain transfer. The results reveal that internal-state hallucination signals in LLMs transfer across languages and domains for most models, with cross-lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross-domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.
Aisha Alansari, Malak Alkhorasani, Hamzah Luqman
King Fahd University of Petroleum and Minerals · Imam Abdulrahman bin Faisal University
As Large Language Models (LLMs) increasingly serve as foundational reasoning engines, their tendency to hallucinate remains a critical vulnerability. While recent internal state probes offer a promising alternative to slow external retrieval systems, they largely reduce hallucination detection to a token-wise binary classification task, failing to capture the structured, sequential boundaries of semantic drift. Here, we introduce an internal hidden state framework for fine-grained, span-level hallucination detection. By inspecting layer-wise activation patterns, we attempt to detect the exact hallucination onset and continuation tokens in an LLM generation. Our experiments show that this approach successfully isolates hallucination onsets, achieving substantial improvements in Precision-Recall AUC over random baselines despite extreme class imbalance. Ultimately, we propose a novel cross-model detection framework in which one model observes the internal representations elicited by another model's generation. We find that an external observer can match or exceed a generator's self-detection of its own hallucination onsets, including when the observer is the smaller model, suggesting that self-detection is not the ceiling for onset localisation.
Kingshuk Gupta, Davide Buscaldi
LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord
With the large-scale adoption of Large Language Models (LLMs) in various applications, there is a growing reliability concern due to their tendency to generate inaccurate text, i.e. hallucinations. In this work, we propose Cross-Layer Attention Probing (CLAP), a novel activation probing technique for hallucination detection, which processes the LLM activations across the entire residual stream as a joint sequence. Our empirical evaluations using five LLMs and three tasks show that CLAP improves hallucination detection compared to baselines on both greedy decoded responses as well as responses sampled at higher temperatures, thus enabling fine-grained detection, i.e. the ability to disambiguate hallucinations and non-hallucinations among different sampled responses to a given prompt. This allows us to propose a detect-then-mitigate strategy using CLAP to reduce hallucinations and improve LLM reliability compared to direct mitigation approaches. Finally, we show that CLAP maintains high reliability even when applied out-of-distribution. Code available at: https://github.com/itsmemala/CLAP