Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

Renato Vukovic; Hsien-chin Lin; Carel van Niekerk; Benjamin Ruppik; Michael Heck; Shutong Feng; Nurul Lubis; Milica Gasic

cs.CLSep 4, 2026

Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

Authors: Renato Vukovic, Hsien-chin Lin, Carel van Niekerk, Benjamin Ruppik, Michael Heck, Shutong Feng, Nurul Lubis, Milica Gasic

Organizations: Heinrich Heine University Düsseldorf, Germany

Abstract

Hallucination-where a language model generates outputs that are factually incorrect or unsupported by the source-is a major challenge for both prompted and fine-tuned language models. Detecting hallucinations is difficult due to the opaque reasoning processes of LLMs, which often provide little insight into why a model's output may be inaccurate. In this work, we investigate whether an LLM can use an alternative, low level, symbolic competence such as SQL for unsupervised hallucination detection in some high level task. For this, we make an LLM build an SQL database from reference documents. This SQL database is then used for reasoning over the reference and the sampled response in a hallucination detection pipeline that is grounded in the database, thereby providing a neurosymbolic checkup. On RAGTruth and DiaHalu hallucination detection datasets, we find that our approach improves on direct prediction and competes with state-of-the-art hallucination detection methods, while not requiring domain-specific fine-tuning. Instead it relies on a low-level general competence already present in LLMs. This warrants further investigation of low-level LLM competences in neurosymbolic approaches.

Figures & tables

Figure 1: Grounding hallucination detection using low-level symbolic competences.

Figure 2: Text-to-SQL grounded hallucination detection with an example from the RAGTruth QA task.

Methods	Question Answering			Data-to-text Writing			Summarization			Overall Macro
Methods	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Niu et al. (2024) initial baselines
Prompt ${}_{\text{gpt-3.5-turbo}}$	18.8	84.4	30.8	65.1	95.5	77.4	23.4	89.2	37.1	35.8	89.7	48.4
Prompt ${}_{\text{gpt-4-turbo}}$	33.2	90.6	45.6	64.3	100.0	78.3	31.5	97.6	47.6	43.0	96.1	57.2
SelfCheckGPT ${}_{\text{gpt-3.5-turbo}}$	35.0	58.0	43.7	68.2	82.8	74.8	31.1	56.5	40.1	44.8	65.8	52.9
LMvLM ${}_{\text{gpt-4-turbo}}$	18.7	76.9	30.1	68.0	76.7	72.1	23.3	81.9	36.2	36.7	78.5	46.1

Table 1: The response-level hallucination detection performance for each baseline method and different models, with overall macro averages over the different tasks from Niu et al. (2024) . Song et al. (2024) ; Kovács and Recski (2025) are fine-tuned methods. ReDeEP results with the token-level method taken from Dubanowska et al. (2025) . Bold indicates the best F1-score for unsupervised methods.

Task	Parsing Accuracy (%)
Question Answering	80.00
Data-to-text	94.00
Summarisation	82.00
Overall	85.33

Table 2: Manual evaluation of SQL parsing accuracy on 50 randomly sampled reference documents per task. Accuracy denotes the percentage of instances where the generated SQL queries fully capture all source text information without data loss.

Method	Precision	Recall	F1
Direct Prediction	48.2	95.4	61.6
Symbolic Prediction	39.9	51.3	39.7
SELECT Only IDs	62.4	80.1	69.7
TeQHallu	61.7	88.9	71.3

Table 3: Ablation study on the RAGTruth dataset showing overall macro-average precision, recall, and F1.

Model	Method	Precision	Recall	F1
Gemini-2.5-Flash (Spider EM: 51.3)	Direct Prediction	48.2	95.4	61.6
Gemini-2.5-Flash (Spider EM: 51.3)	TeQHallu	61.7	88.9	71.3
Qwen3 235B/22B (Spider EM: 29.0)	Direct Prediction	50.9	94.0	64.9
Qwen3 235B/22B (Spider EM: 29.0)	TeQHallu	68.3	67.6	67.2
GPT-OSS 20B/3.6B (Spider EM: 22.4)	Direct Prediction	69.2	69.0	68.7
GPT-OSS 20B/3.6B (Spider EM: 22.4)	TeQHallu	64.0	74.7	68.6

Table 4: Model hallucination detection results on RAGTruth, reporting macro-average precision, recall, and F1-scores. Total/active parameters and Spider Yu et al. (2018) multi-turn text-to-SQL exact match (EM) performance are provided in brackets.

Model	Variant	Precision	Recall	F1 Score
Random	–	31.86	48.00	38.30
SelfCheckGPT	BertScore	35.38	30.67	32.86
	NLI	38.84	62.67	47.96
	Prompt	48.00	32.00	38.40
FOCUS	–	34.09	60.00	43.48
LLaMA-30B	–	30.77	5.33	9.09

Table 5: Hallucination detection performance on the DiaHalu TOD portion. Baseline results from Chen et al. (2024b) . Bolding indicates the best F1-score for each model group using our TeQHallu pipeline.

	NF	Ic	Ir	Ov	ALL
Gemini-1.5 PRO	27.63	42.67	23.53	30.77	31.39
TeQHallu (ours)	45.33	50.00	11.11	40.00	42.38
GPT-4	33.33	44.78	0.00	0.00	33.87
TeQHallu (ours)	40.00	45.00	0.00	40.00	37.13
Gemini-2.5-Flash	27.63	42.67	23.53	30.77	31.39
TeQHallu (ours)	48.65	47.83	10.53	40.00	42.95

Table 6: Fine-grained hallucination-type recognition F1-scores on the TOD portion. Micro average is reported over all types. Bolding indicates the best overall performance of each model.

Hallucination Detection

Hallucination detection identifies unsupported, fabricated, or evidence-inconsistent content in language and multimodal model outputs.

See all 78 papers

Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments
May 5, 2026Hao Mi, Qiang Sheng, Shaofei Wang +7Hallucination in Language Models LLM Hallucination Detection
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts
May 16, 2026Khizar Hussain, Murat KantarciogluLLM Evaluation Language Model Error Detection
Virginia Tech
HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders
Apr 6, 2026Boshui Chen, Zhaoxin Fan, Ke Wang +5LLM Hallucination Detection
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University, Beijing, China. · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. · Beijing Academy of Blockchain and Edge Computing, Beijing, China. +1

May 5, 2026J/K move · Enter open · S save

Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments

Hao Mi, Qiang Sheng, Shaofei Wang, Beizhe Hu+6

Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences

Hallucination in Language Models LLM Hallucination Detection Generative AI Evaluation

Open paper AskPDF arXiv

Large Language Models (LLMs) are prone to factual hallucinations, risking their reliability in real-world applications. Existing hallucination detectors mainly extract micro-level intrinsic patterns for uncertainty quantification or elicit macro-level self-judgments through verbalized prompts. However, these methods address only a single facet of the hallucination, focusing either on implicit neural uncertainty or explicit symbolic reasoning, thereby treating these inherently coupled behaviors in isolation and failing to exploit their interdependence for a holistic view. In this paper, we propose LaaB (Logical Consistency-as-a-Bridge), a framework that bridges neural features and symbolic judgments for hallucination detection. LaaB introduces a "meta-judgment" process to map symbolic labels back into the feature space. By leveraging the inherent logical bridge where response and meta-judgment labels are either the same or opposite based on the self-judgment's semantics, LaaB aligns and integrates dual-view signals via mutual learning and enhances the hallucination detection. Extensive experiments on 4 public datasets, across 4 LLMs, against 8 baselines demonstrate the superiority of LaaB.

Abstract

Hallucination Detection

Explore similar work

Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments

PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders