Suppressing Domain-Specific Hallucination in Construction LLMs: A Knowledge Graph Foundation for GraphRAG and QLoRA on River and Sediment Control Technical Standards

Takato Yasuno

cs.IRMar 3, 2026

Suppressing Domain-Specific Hallucination in Construction LLMs: A Knowledge Graph Foundation for GraphRAG and QLoRA on River and Sediment Control Technical Standards

Authors: Takato Yasuno

Abstract

This paper addresses the challenge of answering technical questions derived from Japan's River and Sediment Control Technical Standards -- a multi-volume regulatory document covering survey, planning, design, and maintenance of river levees, dams, and sabo structures -- using open-source large language models running entirely on local hardware. We implement and evaluate three complementary approaches: Case A (plain 20B LLM baseline), Case B (8B LLM with QLoRA domain fine-tuning on 715 graph-derived QA pairs), and Case C (20B LLM augmented with a Neo4j knowledge graph via GraphRAG). All three cases use the Swallow series of Japanese-adapted LLMs and are evaluated on a 100-question benchmark spanning 8 technical categories, judged automatically by an independent LLM (Qwen2.5-14B, score 0--3). The key finding is a performance inversion: the 8B QLoRA fine-tuned model (Case B) achieves a judge average of 2.92/3 -- surpassing both the 20B plain baseline (Case A: 2.29/3, $+$ 0.63) and the 20B GraphRAG approach (Case C: 2.62/3, $+$ 0.30) -- while running at 3 $\times$ faster latency (14.2s vs. 42.2s for Case A). GraphRAG provides moderate gains ( $+$ 0.33 over baseline) but is outperformed by domain-specific fine-tuning in both quality and efficiency. We document the full engineering pipeline, including knowledge graph construction (200 nodes, 268 relations), QLoRA training data generation from Neo4j relations, training on a single GPU (16 GB VRAM) using unsloth, GGUF Q4_K_M quantisation and Ollama deployment, and the graph retrieval and re-ranking design. High-level engineering lessons are distilled in the main body; implementation pitfalls and toolchain details are documented in Supplementary Materials.

Figures & tables

Relation	From $\to$ To	Role
HAS_CHAPTER	Std $\to$ Ch	Doc. structure
HAS_SECTION	Ch $\to$ Sec	Doc. structure
HAS_ITEM	Sec $\to$ Item	Doc. structure
DESCRIBED_IN	Fac $\to$ Ch/Sec	Rule location
SUBJECT_TO	Fac $\to$ Haz	Applicable hazard
MITIGATES	Fac $\to$ Haz	Countermeasure

Table 1: Knowledge graph relation schema (268 relations total).

Panel of Figure 1 — Figure 1: Knowledge graph schema (Node & Relation Map). The Structural Hierarchy (left, orange) encodes four-level document structure. The Domain Semantics (right, blue) encodes engineering entities and their mutual relations. Dashed grey arrows cross-link domain nodes to structural locations ( DESCRIBED_IN , DEFINED_IN ). In total: 9 node types, 200 nodes, 11 relation types, 268 relations.

Panel of Figure 2 — Figure 2: GraphRAG inference pipeline (Case C). At query time, keywords extracted from the user question drive five parallel Neo4j Cypher queries whose results are deduplicated and scored. If fewer than 25 hits are returned, an adaptive retry doubles TOP_K and broadens the match to substring search. The top-scoring 80% of records (up to 2,000 chars) are prepended to the plain-LLM prompt before generation by Swallow-20B. Qwen2.5-14B then evaluates the output and returns $s_{\mathrm{J}}\in\{0,1,2,3\}$ .

Relation Type	Count	Share (%)
HAS_CHAPTER	212	29.7
DESCRIBED_IN	89	12.4
HAS_SECTION	86	12.0
HAS_ITEM	63	8.8
REQUIRES	56	7.8
SUBJECT_TO	55	7.7

Table 2: Training QA pair distribution by relation type.

Parameter	Value
lora_r	16
lora_alpha	16
lora_dropout	0.0
Target modules	q/k/v/o/gate/up/down proj
load_in_4bit	True (NF4)
max_seq_length	2,048

Table 3: QLoRA hyperparameters (Case B).

Subset	QA Pairs	Final Loss	Time
100	100	0.7958	4.4 min
250	250	0.6859	10.2 min
500	500	0.6045	20.0 min
715	715	0.5565	28.5 min

Table 4: QLoRA convergence check on stratified subsets.

Case	Model	Avg	$\Delta$ Avg	Sc-3 (%)	Sc-2 (%)	Low-qual. (%)	Lat. (s)	Speedup
A (Plain LLM)	Swallow-20B	2.29	—	60	12	28	42.2	$1.0\times$
C (GraphRAG)	Swallow-20B	2.62	$+0.33$	77	10	13	31.1	$1.4\times$
B (QLoRA FT)	Swallow-8B	2.92	$\mathbf{+0.63}$	92	8	0	14.2	$\mathbf{3.0\times}$

Table 5: Overall evaluation results (100-question benchmark).

\Delta

Avg = improvement over Case A (plain LLM baseline). Low-qual. = score-0 or score-1 responses.

Figure 3: Approach trade-off: inference speed vs. answer quality (normalised). Case B (QLoRA FT) occupies the ideal upper-right quadrant — highest quality and fastest inference.

Figure 4: Judge score distributions (0–3) for Cases A, B, and C across 100 questions.

Figure 5: Evolution of approaches: accuracy improvement across experimental phases. QLoRA FT (Case B) achieves the highest score at the final stage.

Category	$N$	A avg	B avg	C avg
Survey	7	2.14	2.86	2.57
Planning	8	2.75	2.88	2.62
Design	15	2.13	2.93	2.60
Maint. (River)	20	2.05	2.95	2.55
Maint. (Dam)	15	2.47	2.93	2.73
Maint. (Sabo)	15	2.33	2.93	2.53

Table 6: Judge average score (/3) by technical category.

Metric (s)	Case A	Case B	Case C
Mean	42.2	14.2	31.1
Median	43.5	—	33.4
Min	22.0	—	7.9
Max	43.9	—	35.5
Total (100 Q, min)	70.3	23.7	51.9

Table 7: Inference latency comparison.

Q#	Sub-category	A	B	C
Q5	Levee life-extension	0	3	3
Q14	Cycle-type maint. flow	3	3	3
Q24	Dam life-extension	0	3	3
Q26	Dam sedimentation ratio	3	2	2
Q37	Sabo dam emergency insp.	1	3	0
Q52	Basin avg. rainfall	3	2	1

Table 8: Scores for 10 representative questions (A/B/C, 0–3).

Question Answering

Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.

See all 260 papers

Jul 13, 2026

RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

Graph retrieval-augmented generation (GraphRAG) enhances large language models with structured knowledge, yet existing systems construct knowledge graphs in a single extraction pass, producing noisy entities and brittle retrieval. RAGU, an open-source modular GraphRAG engine, addresses this by separating extraction from consolidation: entities and relations pass through two-stage typed extraction, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection. A key insight motivates a compact extractor: the skills an in-pipeline LLM needs - comprehension, extraction, reasoning over context - are language skills that grow only weakly with model size, unlike factual world knowledge. Accordingly, we train Meno-Lite-0.1, a 7B model optimized for language skills, which outperforms Qwen2.5-32B on knowledge-graph construction (+12.5% relative harmonic mean) and matches it on English GraphRAG tasks. On GraphRAG-Bench (Medical), RAGU retrieves the most complete context at every factoid level (evidence recall up to 0.84 vs.

\leq

0.76) and overtakes HippoRAG2 on synthesis tasks; on multi-hop factoid QA, the apparent HippoRAG2 advantage is shown to be largely an answer-format artifact. RAGU is installable via

\texttt{pip install graph_ragu}

, runs on a single GPU, and is released under MIT. The source code is publicly available at https://github.com/RaguTeam/RAGU, and the Meno-Lite-0.1 model can be obtained from https://huggingface.co/bond005/meno-lite-0.1.

Mikhail Komarov, Ivan Bondarenko, Stanislav Shtuka +5

1ITMO University · 2Novosibirsk State University · 3Far Eastern Federal University +1

RAG Evaluation KG Construction LLM Reasoning with Graphs Graph Retrieval-Augmented Generation Small Language Models Knowledge Representation+2

Ask PDF

Sep 16, 2026

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

Large language models (LLMs) suffer from a long-tail deficit: culturally specific facts, particularly those concerning underrepresented regions such as Latin America, appear too rarely in pretraining corpora to be reliably memorized. Retrieval-Augmented Generation (RAG) addresses this by grounding generation in external text, but structured alternatives such as Knowledge Graphs (KGs) offer tighter control over what enters the context, along with potential gains in explainability and updatability. We benchmark Graph-RAG against standard RAG on LatamQA, a culturally grounded multiple-choice dataset spanning eight thematic categories. The graphs are built end-to-end from Wikipedia articles with KGGen, a recent open-domain extractor, without manual curation in our main setting. G-Retriever is competitive with RAG and reduces the error of the base LLM by 72% with a standard KG and 78% with a benchmark-aware variant, the gap to RAG narrowing further as the graph is oriented toward task-relevant content. The trained projection transfers zero-shot to Portuguese without target-language fine-tuning, indicating multilingual reach.

Pablo Poulenard, Yannis Karmim, Valentin Barrière

DCC, Universidad de Chile, Santiago, Chile · École Polytechnique, Palaiseau, France · Inria, Almanach, Paris, France +2

Retrieval-Augmented Generation Graph Retrieval-Augmented Generation Question Answering Cultural Knowledge in Language Models

Ask PDF

Jun 4, 2026

Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)

Large language models (LLMs) have fundamentally transformed the landscape of Natural Language Processing. Despite these advances, LLMs and LLM-based systems remain prone to a variety of failure modes. Retrieval-augmented generation (RAG) systems have emerged as a common deployment scenario seeking to both avoid the well known risk of the LLM "hallucinating" information, and to enable reasoning and question answering over proprietary information that the LLM did not have access to during training without resorting to expensive model fine-tuning. In this work, we explore the idea of using a lightweight graph structure with a relatively simple graph schema, to support the RAG subsystem via a dedicated toolset. We design an agentic system with a variety of vector search and graph query tools operating over a structured dataset based on a curated subset of English Wikipedia articles, and evaluate its performance on questions from MoNaCo, a challenging Wikipedia QA benchmark of complex query answering tasks. Our results show that the introduction of graph-based tools can significantly increase the precision and recall of factual correctness, can halve the number of hallucinated answers, and achieves the highest fine-grained truthfulness score among the three evaluated scenarios. All this with a modest increase in token usage.

Christopher J. Wedge, Joshua Stutter, Danny Dixon +1

National Innovation Centre for Data, Newcastle University

Multi-Hop QA Tool-Augmented Language Model Agents LLM Hallucination Mitigation Graph Retrieval-Augmented Generation Agentic RAG Hallucination in RAG

Ask PDF

Abstract

Question Answering

Explore similar work

RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)