Adaptive Transfer of Representational Signatures for LLM Reliability Detection

Yongtong Li; Yixuan Florence Wu; Jiawei Zhang; Naichen Shi

cs.CLOct 24, 2026

Adaptive Transfer of Representational Signatures for LLM Reliability Detection

Authors: Yongtong Li, Yixuan Florence Wu, Jiawei Zhang, Naichen Shi

Organizations: Northwestern University · University of Wisconsin–Madison

Abstract

Large language models (LLMs) are reported to be susceptible to reliability issues including hallucinations and reasoning errors. Existing quality detectors can predict unreliable outputs based on internal representations with decent accuracies. However, training these detectors often requires substantial domain-specific labeled data, which limits their applicability to new domains. We propose to address this challenge by transferring representational signatures across related domains. More specifically, we build a statistical framework that first extracts low-dimensional signatures for each domain, then uses cross-fitting to learn composition weights of domain-specific predictors. The weights adapt to each source domain's predictive relevance to the target domain. Our asymptotic analysis of a least-squares analogue characterizes the limiting prediction risk and aggregation weights in terms of domain discrepancies and representation bias. Experiments across multiple hallucination datasets and open-weight LLMs show that, with only 25 labeled questions in the new domain, adaptive transfer improves hallucination-detection AUROC by 6.8%. For reasoning-step prediction, it improves AUROC by 8.4%.

Figures & tables

Figure 1: Overview of the proposed AdaST framework.

Figure 2: Synthetic validation of simplex-constrained least-squares aggregation. A. Target weight $\widehat{w}_{T}$ against $n_{T}$ for several source drifts $\tau$ . B. Excess risk $\mathcal{R}_{T}-\sigma_{T}^{2}$ against $n_{T}$ at $\tau=0.5$ . Solid lines summarize Monte Carlo means with $95\%$ confidence bands for the means. Dashed lines mark the constrained population weights and risks from Theorem 1 .

		$n_{T}=25$		$n_{T}=50$		$n_{T}=100$		Full
Task	Data access	AUC	F1	AUC	F1	AUC	F1	AUC	F1
Hallucination detection	Target-only	74.2 (1.0)	62.8 (1.5)	77.3 (0.9)	65.1 (1.4)	79.8 (0.8)	66.9 (1.3)	84.9 (0.5)	71.5 (1.1)
	AdaST	81.0 (0.9)	66.2 (1.4)	81.7 (0.8)	67.3 (1.4)	82.3 (0.8)	68.4 (1.3)	85.0 (0.5)	71.5 (1.1)
Reasoning-step correctness	Target-only	81.1 (1.0)	71.9 (1.0)	85.5 (1.0)	76.3 (0.8)	87.6 (0.9)	78.8 (1.0)	88.9 (0.9)	80.7 (0.9)
	AdaST	89.5 (0.9)	80.8 (1.0)	89.8 (0.9)	81.2 (1.0)	90.2 (0.9)	82.2 (1.1)	90.1 (0.8)	82.2 (0.9)

Table 1: Performance of target-only detectors and AdaST across different target-label budgets, averaged over six datasets for hallucination detection and seven PRM800K MATH domains for reasoning-step correctness. Subscripts report standard errors.

Figure 3: Hallucination detection at $n_{T}=25$ , by target dataset. Each bar is the mean over the five evaluated models and error bars are standard errors across models.

Figure 4: Learned AdaST combination weights. (a) Detector weights at $n_{T}=25$ . (b) Target weight $w_{T}$ and source weight $1-w_{T}$ across target training sizes. (c) Target weight $w_{T}$ across model families.

SinkProbe as base detector	OPT-6.7B	Mistral-7B	Llama-3.1-8B	DeepSeek-7B	Gemma-7B
Target-only	$71.8$ (4.6)	$81.4$ (2.4)	$79.4$ (3.5)	$72.9$ (3.4)	$80.4$ (3.8)
AdaST	$\mathbf{73.3}$ (3.9)	$\mathbf{82.7}$ (3.4)	$\mathbf{81.7}$ (3.5)	$\mathbf{74.6}$ (3.7)	$\mathbf{81.8}$ (4.2)

Table 2: Robustness to base detector choice at

n_{T}=25

. AUROC (%).

Method	$P=1$	$P=2$	$P=3$
Target-only	69.6 (4.0)	71.9 (2.9)	72.8 (2.5)
AdaST	76.2 (2.8)	77.7 (2.3)	78.2 (2.3)

Table 3: Effect of paraphrase augmentation (AUROC, %).

Setting	Semantic accuracy	Hallucination	Abstention
No steering	27.2	67.8	5.0
AdaST steering	30.0 (0.7)	64.6 (0.9)	5.4 (0.4)

Table 4: Activation steering results on Llama-3.1-8B, averaged over three seeds.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

		OPT-6.7B		Llama-3.1-8B		Gemma-7B		Mistral-7B		DeepSeek-7B		Avg.
Task	Data access	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1
Hallucination	Target-only	74.1 (2.3)	61.4 (3.5)	74.1 (2.7)	65.2 (3.7)	74.9 (2.7)	64.4 (4.0)	72.4 (1.1)	59.3 (1.0)	75.7 (2.0)	63.9 (3.1)	74.2 (1.0)	62.8 (1.5)
	AdaST	78.6 (2.2)	62.9 (2.9)	82.6 (1.8)	69.7 (2.9)	81.4 (2.1)	66.5 (3.7)	83.0 (0.5)	66.3 (1.0)	79.6 (2.3)	65.6 (3.2)	81.0 (0.9)	66.2 (1.4)
Reasoning-step	Target-only	84.8 (1.5)	74.7 (1.5)	80.2 (1.9)	71.9 (2.5)	73.3 (1.8)	65.1 (1.6)	83.5 (1.2)	73.7 (1.4)	83.7 (1.2)	74.0 (1.3)	81.1 (1.0)	71.9 (1.0)
	AdaST	92.3 (0.8)	84.8 (0.6)	87.5 (1.3)	78.3 (1.3)	82.5 (1.9)	72.7 (1.9)	92.4 (0.5)	83.4 (0.8)	92.8 (1.0)	84.9 (1.5)	89.5 (0.9)	80.8 (1.0)

Appendix

Table 5: Per-model breakdown of Table 1 at

n_{T}=25

		NQ		TriviaQA		CoQA		HaluEval		SimpleQA		HotpotQA		Avg.
$n_{T}$	Data access	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1
$25$	Target-only	73.1 (1.5)	57.9 (1.3)	79.5 (1.4)	70.6 (2.4)	70.6 (1.6)	62.8 (1.4)	80.8 (2.4)	74.0 (3.0)	70.0 (1.3)	54.6 (0.6)	71.5 (1.0)	57.2 (0.5)	74.2 (1.0)	62.8 (1.5)
	AdaST	84.3 (1.3)	64.4 (1.7)	83.8 (1.0)	72.6 (2.3)	73.8 (2.3)	64.0 (0.9)	84.6 (0.6)	76.5 (2.0)	80.5 (1.0)	58.6 (1.9)	79.2 (1.6)	61.2 (1.5)	81.0 (0.9)	66.2 (1.4)
$50$	Target-only	77.4 (2.0)	60.9 (1.6)	80.4 (1.4)	71.3 (2.5)	74.1 (1.9)	65.8 (1.1)	83.0 (1.6)	75.5 (2.8)	73.2 (0.6)	56.5 (1.0)	75.9 (1.3)	60.3 (0.6)	77.3 (0.9)	65.1 (1.4)
	AdaST	84.7 (1.4)	65.4 (1.9)	84.2 (0.9)	74.0 (1.9)	75.4 (2.3)	66.3 (1.0)	85.1 (0.6)	77.1 (2.1)	81.0 (1.0)	59.5 (1.7)	79.7 (1.6)	61.4 (1.8)	81.7 (0.8)	67.3 (1.4)
$100$	Target-only	81.1 (1.6)	63.4 (1.7)	82.3 (1.2)	72.8 (2.2)	75.9 (2.0)	67.0 (1.5)	84.4 (1.0)	76.8 (2.4)	77.8 (0.7)	59.1 (0.8)	77.6 (1.2)	62.1 (0.9)	79.8 (0.8)	66.9 (1.3)

Appendix

Table 6: AdaST for hallucination detection against the same detector without source data, by dataset and target label budget

n_{T}

		Algebra		C.&Prob.		Geometry		Int. Alg.		N. Theory		Prealg.		Precalc.		Avg.
$n_{T}$	Data access	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1
$25$	Target-only	77.1 (3.5)	67.6 (4.1)	78.8 (2.5)	70.6 (2.3)	82.6 (1.0)	73.9 (0.4)	82.9 (2.4)	73.8 (2.5)	81.1 (2.3)	70.5 (1.8)	82.3 (2.9)	74.0 (2.2)	82.8 (1.9)	72.7 (1.7)	81.1 (1.0)	71.9 (1.0)
	AdaST	85.6 (3.7)	77.3 (3.9)	89.2 (1.5)	81.0 (1.6)	91.3 (1.2)	81.9 (1.9)	90.4 (2.0)	81.1 (2.8)	90.9 (1.6)	82.0 (2.6)	87.8 (2.2)	79.8 (2.3)	91.4 (1.3)	82.7 (1.3)	89.5 (0.9)	80.8 (1.0)
$50$	Target-only	81.9 (3.6)	73.9 (3.2)	85.1 (2.1)	77.0 (1.6)	87.5 (1.4)	76.5 (1.4)	85.7 (2.7)	77.4 (2.6)	86.7 (2.1)	77.6 (2.1)	84.0 (2.5)	74.0 (1.7)	87.6 (1.8)	78.0 (1.8)	85.5 (1.0)	76.3 (0.8)
	AdaST	86.1 (3.8)	78.2 (3.9)	89.3 (1.3)	80.1 (1.5)	91.5 (1.4)	83.0 (2.6)	90.9 (1.9)	81.4 (2.6)	90.9 (1.6)	83.5 (1.7)	88.3 (2.4)	80.1 (2.7)	91.6 (1.4)	81.9 (1.7)	89.8 (0.9)	81.2 (1.0)
$100$	Target-only	84.0 (3.3)	75.6 (3.3)	87.7 (1.6)	77.9 (1.8)	88.1 (1.8)	78.1 (2.1)	88.0 (1.8)	79.5 (2.5)	88.1 (2.1)	80.1 (2.2)	–	–	89.9 (1.5)	81.7 (1.3)	87.6 (0.9)	78.8 (1.0)

Appendix

Table 7: AdaST for Reasoning-step correctness against the same detector without source data, by MATH domain and target label budget

n_{T}

. Prealgebra has fewer than

100

labeled training questions, so the

n_{T}=100

row omits it and averages over the remaining six domains.

		NQ		TriviaQA		CoQA		HaluEval		SimpleQA		HotpotQA		Avg.
Model	Method	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1	AUC	F1
OPT-6.7B	HaloScope	57.3 (1.5)	52.9 (0.6)	61.3 (2.7)	55.6 (1.9)	53.6 (1.6)	51.2 (1.6)	68.0 (3.6)	61.9 (3.1)	61.4 (3.0)	52.9 (1.8)	51.7 (2.6)	49.7 (0.8)	58.9 (2.2)	54.0 (1.6)
	HARP	66.0 (1.2)	52.4 (1.4)	67.7 (1.0)	54.5 (2.6)	63.8 (0.7)	55.2 (1.8)	81.8 (0.5)	68.3 (3.7)	60.8 (3.2)	49.7 (2.6)	59.9 (2.2)	49.1 (1.8)	66.6 (3.0)	54.9 (2.6)
	SinkProbe	75.0 (1.1)	49.0 (0.5)	74.8 (0.6)	54.8 (3.0)	58.9 (0.6)	41.5 (2.4)	82.6 (0.6)	69.9 (1.1)	74.1 (2.9)	52.1 (1.8)	75.7 (1.0)	52.7 (2.5)	73.5 (2.8)	53.3 (3.4)
	Target-only	71.9 (1.0)	53.5 (1.0)	75.7 (0.6)	65.2 (0.6)	65.8 (1.0)	60.7 (1.2)	84.6 (0.3)	77.8 (0.2)	73.3 (2.3)	52.5 (1.5)	73.5 (1.5)	58.6 (1.7)	74.1 (2.3)	61.4 (3.5)
	AdaST (Ours)	82.2 (0.4)	60.7 (0.7)	79.9 (0.5)	64.6 (1.4)	68.9 (0.4)	62.5 (0.8)	85.4 (0.2)	77.3 (0.8)	79.9 (2.0)	55.1 (1.3)	75.2 (1.3)	57.3 (1.3)	78.6 (2.2)	62.9 (2.9)

Appendix

Table 8: Matched-budget hallucination detection at

n_{T}=25

, by model and target dataset. Subscripts are bootstrap standard errors

Language Model Error Detection

Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.

See all 91 papers

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?
Jul 4, 2026Aisha Alansari, Malak Alkhorasani, Hamzah LuqmanMultilingual Language Model Evaluation LLM Hallucination Detection
King Fahd University of Petroleum and Minerals · Imam Abdulrahman bin Faisal University
Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs
Mar 27, 2026Snehit Vaddi, Pujith VaddiHallucination in Language Models LLM Hallucination Detection
Independent Researcher
When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study
Aug 25, 2026Mohit Singh Chauhan, Vipin Gyanchandani, Dylan BouchardLLM Hallucination Detection LLM Uncertainty Estimation
CVS Health®, Wellesley, MA, USA

Jul 4, 2026J/K move · Enter open · S save

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

Aisha Alansari, Malak Alkhorasani, Hamzah Luqman

King Fahd University of Petroleum and Minerals · Imam Abdulrahman bin Faisal University

Multilingual Language Model Evaluation LLM Hallucination Detection Language Model Probing

Open paper AskPDF arXiv

Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal-state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection using internal representations from six LLMs on the generative question-answering task. We conduct a systematic Arabic <-> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross-lingual transfer, cross-domain transfer, and combined cross-lingual and cross-domain transfer. The results reveal that internal-state hallucination signals in LLMs transfer across languages and domains for most models, with cross-lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross-domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.

Abstract

Language Model Error Detection

Explore similar work

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs

When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study