cs.AIAug 24, 2026

Walking on the DARKSIDE

Authors: Aldo Gangemi, Emanuele Bottazzi

Organizations: AlmaAI, University of Bologna · ISTC-CNR, Rome, Italy

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Models (LLMs) do not natively track the path of exclusions that a coherent discourse demands. When an input rests on a fabricated authority, a misapplied mechanism, or a surreptitious analogy, an unsteered LLM tends to engage with it as if it were well-posed, and this affects its generation. POLANYI++, an LLM-steering method that uses heuristics, ontologies and problem-solving methods for tacit-knowledge extraction, produces an Extended Knowledge Graph (XKG) in OWL2, but when a sophisticated nonsensical input is reified into the graph alongside the legitimate triples, it gets hardly detectable by automated reasoners, since the XKG is generated jointly with the wrong assumptions. We introduce DARKSIDE, a coherence-auditing method on top of POLANYI++. DARKSIDE formalises an explicit data structure of accumulated exclusions over discourse time, complemented by a warrant axis that classifies each named referent as Warranted, Unattested, Misattributed or Fabricated. The method is anchored in nine theoretical fragments unified under a shared deep frame of path integrity. The resulting DARKPOLANYI is evaluated as a steering layer over Gemini 3 on BSBench, a 100-item adversarial corpus of sophisticated-sounding nonsense across multiple domains, with Claude Sonnet 4.6 as an independent judge. DARKPOLANYI scores 1.89/2 mean versus 0.95/2 for the unsteered Gemini 3 Pro baseline; on the 97 cases with valid judgments in both arms, paired McNemar gives a paired bootstrap mean-diff = +0.92 (95% CI [+0.75, +1.08], p = 0.0001). The evidence supports an architectural claim: when an LLM forward pass is wrapped in an ontology-mediated auditing, structurally inevitable hallucination can be partially recovered. The XKG functions as the missing memory that LLMs lack, and the warrant axis as an epistemic firewall.

More in this topic

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 644 papers

Explore similar work

CardsList
Feb 23, 2026

KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

Large Language Models (LLMs) possess a remarkable capacity to generate persuasive and intelligible language. However, coherence does not equate to truthfulness, as the responses often contain subtle hallucinations. Existing benchmarks are limited by static and narrow questions, leading to limited coverage and misleading evaluations. We present KGHaluBench, a Knowledge Graph-based hallucination benchmark that assesses LLMs across the breadth and depth of their knowledge, providing a fairer and more comprehensive insight into LLM truthfulness. Our framework utilises the KG to dynamically construct challenging, multifaceted questions, whose difficulty is then statistically estimated to address popularity bias. Our automated verification pipeline detects abstentions and verifies the LLM's response at both conceptual and correctness levels to identify different types of hallucinations. We evaluate 25 frontier models, using novel accuracy and hallucination metrics. The results provide a more interpretable insight into the knowledge factors that cause hallucinations across different model sizes. KGHaluBench is publicly available to support future developments in hallucination mitigation.

Alex Robertson, Huizhi Liang, Mahbub Gani +2

School of Computing, Newcastle University · Sage Ai, Sage Group PLC · Current affiliation: Redis

Hallucination in Language ModelsLLM EvaluationLLM Hallucination Detection
AskPDF
Feb 19, 2026

Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study

Uncertainty quantification has emerged as an effective approach to closed-book hallucination detection for LLMs, but existing methods are largely designed for short-form outputs and do not generalize well to long-form generation. We introduce a taxonomy for fine-grained uncertainty quantification in long-form LLM outputs that distinguishes methods by design choices at three stages: response decomposition, unit-level scoring, and response-level aggregation. We formalize several families of consistency-based black-box scorers, providing generalizations and extensions of existing methods. We also introduce FactScore-STEM-Geo, a new 400-question long-form QA dataset spanning four categories across STEM and Geography. In our experiments across multiple LLMs and datasets, we find 1) claim-response entailment consistently performs better or on par with more complex claim-level scorers, 2) claim-level scoring generally yields better results than sentence-level scoring, and 3) uncertainty-aware decoding is highly effective for improving the factuality of long-form outputs. Our framework clarifies relationships between prior methods, enables apples-to-apples comparisons, and provides practical guidance for selecting components for fine-grained UQ.

Dylan Bouchard, Mohit Singh Chauhan, Viren Bajaj +1

CVS Health, Wellesley, MA

LLM Hallucination MitigationHallucination in Language ModelsFactual Consistency EvaluationLLM ReliabilityLLM Hallucination DetectionLLM Uncertainty Estimation
AskPDF
May 16, 2026

PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

Large language models (LLMs) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong. In medical, legal, and scientific applications this failure causes direct harm, and detecting it from internal model states offers a path to safer deployment. A growing body of work reports that this problem is increasingly tractable, with recent methods achieving high detection performance on widely used benchmarks. We show, however, that much of this apparent progress does not survive scrutiny. Four of the six corpora embed the ground-truth answer directly in the input prompt. A naïve text-similarity baseline we call TxTemb exploits this to achieve near-perfect detection scores without any access to model internals. To measure what genuine detection capability remains once these artifacts are controlled, we conduct a large-scale evaluation spanning twenty-two detection methods, twelve open-source models spanning six architectural families, and six corpora. We further introduce DRIFT, a supervised probe over inter-layer hidden-state transitions, as a point of comparison for live-generation detection. Our findings suggest that the field's reported progress on hallucination detection is substantially explained by benchmark construction artifacts in widely used corpora, and that the majority of established baselines perform near chance under controlled conditions; the consistent exceptions are SAPLMA and DRIFT, both supervised probes on upper-layer hidden states.

Khizar Hussain, Murat Kantarcioglu

Virginia Tech

LLM EvaluationLanguage Model Error DetectionHallucination in Language ModelsLLM ReliabilityLLM Hallucination DetectionLanguage Model Probing
AskPDF