Toward Epistemic Stability: Engineering Consistent Procedures for Industrial LLM Hallucination Reduction

Brian Freeman; Adam Kicklighter; Matt Erdman; Zach Gordon

cs.SEMar 8, 2026

Toward Epistemic Stability: Engineering Consistent Procedures for Industrial LLM Hallucination Reduction

Authors: Brian Freeman, Adam Kicklighter, Matt Erdman, Zach Gordon

Organizations: Trane Technologies, Davidson NC, USA

Abstract

Hallucinations in large language models (LLMs) are outputs that are syntactically coherent but factually incorrect or contextually inconsistent. They are persistent obstacles in high-stakes industrial settings such as engineering design, enterprise resource planning, and IoT telemetry platforms. We present and compare five prompt engineering strategies intended to reduce the variance of model outputs and move toward repeatable, grounded results without modifying model weights or creating complex validation models. These methods include: (M1) Iterative Similarity Convergence, (M2) Decomposed Model-Agnostic Prompting, (M3) Single-Task Agent Specialization, (M4) Enhanced Data Registry, and (M5) Domain Glossary Injection. Each method is evaluated against an internal baseline using an LLM-as-Judge framework over 100 repeated runs per method (same fixed task prompt, stochastic decoding at tau = 0.7. Under this evaluation setup, M4 (Enhanced Data Registry) received ``Better'' verdicts in all 100 trials; M3 and M5 reached 80% and 77% respectively; M1 reached 75%; and M2 was net negative at 34% when compared to single shot prompting with a modern foundation model. We then developed enhanced version 2 (v2) implementations and assessed them on a 10-trial verification batch; M2 recovered from 34% to 80%, the largest gain among the four revised methods. We discuss how these strategies help overcome the non-deterministic nature of LLM results for industrial procedures, even when absolute correctness cannot be guaranteed. We provide pseudocode, verbatim prompts, and batch logs to support independent assessment.

Figures & tables

Figure 1: Taxonomy of the five hallucination-reduction strategies and the root cause each addresses. M1 and M2 target prompt reasoning and structure; M3 targets agent architecture; M4 and M5 target input data quality.

Figure 2: Iterative convergence profile for M1 across three representative trials. Diamonds mark the iteration reaching

\sigma_{\text{sim}}=0.85

Field	Type	Description
component_type	string	Physical role (e.g., “cooling coil valve”)
normal_range	string	Expected value range in operational state
fault_threshold	string	Value indicating fault or alarm condition
depends_on	list	IDs of upstream components
fault_implication	string	What downstream effect this fault causes
units	string	Engineering unit (e.g., %, ∘ F, GPM)

Table 1: Enhanced registry fields added to each raw sensor record (M4).

Figure 3: LLM-as-Judge evaluation pipeline. Each trial generates a method-specific baseline and an enhanced response independently, then a zero-temperature judge produces dimension-level scores and an aggregate verdict.

ID	Domain	Methods	Description
T1	Enterprise software	M1, M2	3-sprint agile plan for a cloud-based real-time IoT telemetry pipeline; Fibonacci story points, role assignments, acceptance criteria, inter-sprint dependencies
T2	ERP incident response	M3	Production ERP outage at 02:14 EST: 6 simultaneous symptoms spanning connection pool, memory, CPU, load balancer, 6 microservices, 3 cron jobs
T3	HVAC diagnosis	M4	Warm-air fault diagnosis; component identifiers (CMP-1, VLV-3, SNS-9) with raw state values; relationship between compressor and TXV
T4	BMS/HVAC troubleshooting	M5	AHU fault query containing 17 domain acronyms; DX coil lockout, chilled water coil, VAV/DDC/VFD interactions

Table 2: Task scenarios used across the five methods. The prompt text is reproduced verbatim in Appendix A.10 .

Method	Better	Same	Worse	Notes
	(%)	(%)	(%)
M1 (Iterative)	75	18	7	Stability proxy; some variance
M2 (Decomposed)	34	25	41	Net negative; context lost in synthesis
M3 (Agents)	80	19	1	Consistent improvement
M4 (Registry)	100	0	0	No worse trials; see confound note
M5 (Glossary)	77	22	1	Consistent improvement

Table 3: D1 Results: 100 Trials, v1 Methods

Figure 4: D1 baseline 100-trial results. All five methods shown. M4 received “Better” in all 100 trials under this judge rubric; M2 was net negative.

Method	Better	Same	Worse	Notes
	(%)	(%)	(%)
M1 v2 (Self-Critique)	100	0	0	10/10 trials; provisional
M2 v2 (Context-Aware)	80	10	10	Large gain over v1
M3 v2 (Consensus)	100	0	0	10/10 trials; provisional
M4 Registry (unch.)	100	0	0	Consistent with D1
M5 v2 (Dynamic)	60	40	0	See Section 8.4

Table 4: D2 Results: 10 Trials, v2 Methods (exploratory)

Figure 5: D2 verification results: 10-trial batch. M1 v2, M3 v2, and M4 each received “Better” in all 10 trials; these should be treated as provisional. M2 v2 shows the largest single-method gain over its v1 baseline (+46 points). M5 v2 discussion in Section 8.4 .

Figure 6: Left: v1 (D1) vs. v2 (D2) “Better” rate per method. Right: improvement delta $\Delta=\text{v2}-\text{v1}$ . M2 shows the largest gain (+46 points) under this rubric; M5 shows a nominal decline attributable primarily to sample size.

Figure 7: “Better” ratings across both evaluation datasets (D1 baseline and D2 verification).

Method	D1 v1	D2 v2	Interpretation
	n=100	n=10
M1	75	100	v2 gain likely; n=10 provisional
M2	34	80	Large gain; 100-trial follow-up warranted
M3	80	100	v2 gain likely; n=10 provisional
M4	100	100	Consistent; confound risk noted
M5	77	60	Variance dominates at n=10

Table 5: “Better” (%) Summary: D1 (n=100) and D2 (n=10)

LLM Hallucination Mitigation

Large language model hallucination mitigation develops ways to detect and reduce unsupported, fabricated, or inconsistent outputs.

See all 178 papers

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models
May 30, 2026S M Tahmid Siddiqui, Akib Jawad Ononto, Anoop Singhal +1LLM Hallucination Mitigation Question Answering
The University of Texas at Dallas, Richardson, TX 75080, USA · National Institute of Standards and Technology, Gaithersburg, USA
Reducing Hallucinations in Large Language Models Through Integrated Self-Verification and Retrieval-Augmented Generation
Aug 12, 2026Ashly JosephLLM Hallucination Mitigation LLM Answer Verification
Cisco Systems, San Jose, CA, USA
Target-Aware Curation of Hallucination Stress Tests for Large Language Models
Oct 19, 2026Jiho Lee, Bingquan Zhang, Lin Ting Liang +2LLM Evaluation Language Model Generation Evaluation
Dartmouth College · Oakland University

May 30, 2026J/K move · Enter open · S save

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models

S M Tahmid Siddiqui, Akib Jawad Ononto, Anoop Singhal, Latifur Khan

The University of Texas at Dallas, Richardson, TX 75080, USA · National Institute of Standards and Technology, Gaithersburg, USA

LLM Hallucination Mitigation Question Answering Prompt Tuning LLM Reliability LLM Abstention

Open paper AskPDF arXiv

Large language models (LLMs) have seen widespread adoption across various domains, yet their reliability is frequently undermined by hallucinations - responses that are plausible-sounding but factually incorrect. In high-stakes domains, these errors can reduce trust and introduce real-world risk. To address this challenge, we present a parameter-efficient approach that uses soft prompts to mitigate hallucinated content and promote responsible abstention in generative question-answering (QA) tasks. Our method, called Responsible Contrastive Soft Prompting (RCSP), uses a composite loss to train soft prompts that balance three goals: suppressing hallucinatory content, encouraging abstention under uncertainty, and preserving or improving factual recall. To achieve these goals, we incorporate contrastive loss, curriculum learning, and KL regularization into our training mechanism. We evaluate our approach on five diverse generative QA datasets using an LLM-as-a-Judge framework. Experimental results on the Gemma 3 (12B) and Llama 3.1 (8B) backbones demonstrate that RCSP effectively balances factual recall with hallucination suppression and abstention, yielding a generally superior F-score over standard reasoning and instruction-based prompting baselines. Notably, these improvements are achieved by training only a fraction of the parameters required by other tuning techniques. Our results demonstrate that soft prompts provide a modular and computationally efficient path toward improving LLM reliability.

Abstract

LLM Hallucination Mitigation

Explore similar work

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models

Reducing Hallucinations in Large Language Models Through Integrated Self-Verification and Retrieval-Augmented Generation

Target-Aware Curation of Hallucination Stress Tests for Large Language Models