cs.CLOct 3, 2026

Grounding Probes: Generator-Independent Hallucination Detection from Observer Model Hidden States

Authors: Michael Rathmayr, Ádám Kovács, Gábor Recski

Organizations: TU Wien, Austria · KR Labs

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Detecting responses that retrieval-augmented generation does not ground in its context trades speed against accuracy: surface checks miss paraphrased fabrication, sampling-based methods cost extra generations. Hidden-state probes sit between the two, but every existing one reads the generating model's own activations, so a change of generator invalidates the detector and a closed-weight generator is out of reach. This paper removes that coupling. The Grounding Probe is logistic regression over the mean-pooled middle-layer hidden states of an observer language model that reads the context, question, and response in one forward pass and generates nothing, with the recipe it needs: pool over response tokens, read a middle layer, and control capacity, which closes the train-test AUROC gap from 0.087-0.202 to 0.009-0.013. Asking the observer outright, rather than reading its hidden state, costs at least +0.166 AUROC in every one of four models. Fitted on 15,090 annotated responses it reaches 0.879-0.894 AUROC on RAGTruth test across four observers, and 0.924 AUROC with 0.820 [email protected] averaged with a supervised span detector, 0.060 above that detector alone. One probe holds across six generators, and hold-out controls, including one in which no evaluation prompt appears in training, bound the cost of removing a generator at about 0.02 AUROC. Code, probes, and predictions are released.

Figures & tables

Figure 1: The Grounding Probe. A frozen observer reads a response it did not produce and generates no token; the probe reads its hidden states. One trained probe serves any generator, including one behind a closed API.
MethodAUROC95% CI[email protected]Latency (ms)
Prior work on RAGTruth (per-generator protocol)
SAPLMA ( Azaria and Mitchell, 2023 )0.803–––
SEP ( Kossen et al., 2024 )0.809–––
ReDeEP ( Sun et al., 2025 )0.824–––
LUMINA ( Yeh et al., 2026 )0.857–––
LettuceDetect-v2-2B ( Kovács et al., 2026 )––0.818–
Table 1: Response-level detection on the RAGTruth test split ( n=2700n=2700 ), with all six generators pooled for the rows in the lower block. The first three prior-work rows are as reported by Sun et al. (2025) on single-generator LLaMA outputs, and LUMINA reports its own figure evaluating GPT-4 and ChatGPT separately, so the blocks are not comparable; the matched comparison is per generator (Table 2 , Section 5.3 ). MiniCheck is a verdict-style entailment verifier we ran ourselves on these rows, so it is comparable to the rows below it; † its latency was measured on a GTX 1080 and is not comparable to the A100 column. The prompted judge is the same observer as the 3B probe row, at the same precision, so those two differ only in how the answer is read; ‡ its latency comes from a separate rented A100 and is likewise not comparable with the column. Kovács et al. (2026) report example-level F1 only, hence the empty AUROC cell. Latency is otherwise a measured per-sample mean on an A100 over these same 2700 rows, for a probe row the observer forward pass, and for a blend the per-sample max⁡(tS1,tS3)\max(t_{\mathrm{S1}},t_{\mathrm{S3}}) , which measured concurrent execution exceeds by 1.33 ×\times , so a blend entry is a floor rather than an estimate (Section 6.4 ). CIs are percentile bootstrap over 10,000 resamples.
GeneratornnHall. rate (%)LettuceDetect-largeGrounding ProbeBlend95% CI
GPT-44509.30.7670.8110.872[0.807, 0.930]
GPT-3.5-turbo45010.20.7310.8240.858[0.789, 0.918]
Mistral-7B-Instruct †45055.80.8820.8860.931[0.906, 0.953]
Llama-2-7B-chat †45050.20.8200.8420.875[0.841, 0.906]
Llama-2-13B-chat †45046.00.8850.9160.934[0.908, 0.957]
Llama-2-70B-chat †45038.00.8730.8650.904[0.871, 0.934]
Table 2: Detection AUROC by the generator that produced the response. † marks open-weight generators, 1800 of the 2700 test responses. One probe, fitted once, is applied to all six with no per-generator tuning. The pooled figure exceeds the macro-average by +0.028 because hallucination rates range from 9.3% to 55.8%, so pooling adds base-rate separation and makes the task easier, not harder. Comparisons against per-generator literature must use the matched generator subset (Section 5.3 ).
 Qwen2.5Llama
Configuration3B7B14B8B
Token aggregation and layer (5-fold CV, 5,000 train)
Mean pool, selected layer0.7330.7780.8030.785
Mean pool, final layer0.6680.7000.7360.732
Second-last token (SLT)0.6320.6550.7030.685
Prompt-final token (TBG)0.5240.5230.5220.523
Table 3: Ablations, AUROC. The first block is 5-fold cross-validation on 5,000 training samples, comparable only within the block. SLT and TBG are the positions used by Kossen et al. (2024) , each at its own best layer, which favours them; mean pooling still wins by 0.100–0.123. “Selected layer” is the deployed layer ( −15-15 / −12-12 / −20-20 / −16-16 for 3B/7B/14B/8B), chosen by this sweep. The supervision block changes the training data: 428,619 QA samples scored by answer-overlap F1 without context (100,000 for Llama 3.1 8B) against 15,090 RAGTruth samples with human labels and context. Those QA-supervised probes are also full-dimensional, at their own layer, and 4-bit, so the row is not single-factor; against the full-dimensional row above the gap narrows to 0.080–0.208.
Figure 2: Capacity control, not dimensionality. Left: test AUROC against the number of components, at the default C=1.0C=1.0 and at a CC cross-validated on the training split. The default- CC curve climbs steeply out of full dimensionality and peaks at or near 512 components, which is what makes the projection look necessary; the tuned- CC curve is flat, with full dimensionality slightly ahead except at the smallest observer. Right: the train–test gap, which PCA-512 collapses from 0.087–0.202 at the default CC to 0.009–0.013. Tuning CC closes most of the same gap without projecting, the middle bar. Section 6.1 reads the two as substitutes and keeps the projection, which needs no search.
Figure 3: Transfer to HaluEval for probes trained only on the RAGTruth training split. Question answering holds up (0.837–0.943); dialogue and summarization degrade but stay well above chance. RAGTruth contains no dialogue data, which is the most plausible reason that split is weakest.
Figure 4: Accuracy against latency. Per-stage latencies are measured per sample on an A100 over the evaluation rows; blend latency is the per-sample max⁡(tS1,tS3)\max(t_{\mathrm{S1}},t_{\mathrm{S3}}) , standing in for concurrent execution on separate devices and excluding coordination overhead. Measured concurrent execution costs 1.33 ×\times that maximum on a two-A100 host (Section 6.4 ). Semantic entropy is not plotted: at seconds per query it sits one to two orders of magnitude to the right, on a different benchmark and setup ( Farquhar et al., 2024 ; Kossen et al., 2024 ) .
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

ConfigurationDetectorBlend[email protected]
LD-base + Qwen 3B0.8400.8990.776
LD-base + Qwen 14B0.8400.9100.791
LD-large + Qwen 3B0.8640.9150.809
LD-large + Qwen 14B0.8640.9240.820
Appendix
Table 4: Score blending at α=0.45\alpha=0.45 across LettuceDetect (LD) base/large and two observer sizes, RAGTruth test AUROC. The “Detector” column is the token-level model alone. Every pairing improves on its token-level detector, and the ordering of the pairings is unchanged at the untuned α=0.50\alpha=0.50 . Confidence routing, which forwards only uncertain samples and replaces rather than combines the score, raises AUROC over the detector alone but costs 0.026–0.042 OptF1 for the large detector.
More in this topic

Hallucination in RAG

See all 19 papers

Explore similar work

CardsList

  1. Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probing

    Mar 20, 2026Dip Roy, Rajiv Misra, Sanjay Kumar Singh +1LLM Hallucination DetectionHidden-State Probing

    Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · Department of Computer Science, Rajarshi School of Management and Technology, Varanasi, India · Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India

  2. Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

    Jul 5, 2026Mohamed Aly BoukeRetrieval-Augmented GenerationLLM Grounding

    Centre for Intelligent Cloud Computing, CoE for Advanced Cloud, Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Bukit Beruang, 75450, Melaka, Malaysia

  3. Pre-Generation Hallucination Detection in Large Language Models via Soft-Target Attention Probing

    Jun 20, 2026Amina Miftakhova, Alexey ZaytsevLanguage Model Error DetectionLLM Hallucination Detection

    Applied AI Institute