RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering

Zavier Ndum Ndum; Jian Tao; John Ford; Mansung Yim; Yang Liu

cs.IRMar 4, 2026

RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering

Authors: Zavier Ndum Ndum, Jian Tao, John Ford, Mansung Yim, Yang Liu

Organizations: Department of Nuclear Engineering, Texas A&M University, College Station, TX, USA · College of Performance, Visualization and Fine Arts, Texas A&M University, College Station, TX, USA

Abstract

Reliable decision support in nuclear engineering requires traceable, domain-grounded knowledge retrieval, yet safety and risk analysis workflows remain hampered by fragmented documentation and hallucination when use pre-trained large language model (LLM) in specialized nuclear domains. To address these challenges, this paper presents RADIANT-LLM (Retrival-Augumented, Domain-Intelligent Agent for Nuclear Technologies using LLM), a multi-modal retrieval-augmented generation (RAG) framework designed for nuclear safety, security, and safeguards applications. The framework uses a local-first, model-agnostic architecture that pairs a multi-modal document ingestion pipeline with a structured, metadata-rich knowledge base, supporting page- and figure-level retrieval from technical documents. An agentic layer coordinates domain-specific tools, enforces citation-backed responses with provenance tracking, and supports human-in-the-loop validation to reduce hallucination risks. To rigorously evaluate this framework, we develop and apply a suite of domain-aware metrics, including Context Precision (CoP), Hallucination Rate (HR), and Visual Recall (ViR), to expert-curated benchmarks derived from Used Nuclear Fuel Storage Facility design guidance. Across varying knowledge base sizes, CoP and ViR remain within an 85--98% band, and hallucination rates are substantially lower than those observed in general-purpose deployments. When the same queries are posed to commercial LLM platforms without the RAG layer, hallucinations and citation errors increase markedly. These results indicate that a locally controlled, multi-modal RAG framework with domain-specific retrieval and provenance enforcement is necessary to achieve the factual accuracy, transparency, and auditability that nuclear engineering workflows demand.

Figures & tables

Figure 1: Interfaces and interdependencies between Safety, Security, and Safeguards (3S) in nuclear facility design. Adapted from Badwan and Demuth [ 6 ] .

Figure 2: Conceptual illustration of LLM augmentation in RADIANT-LLM. A frozen pretrained language model (yellow) acts as a replaceable reasoning core and is supported at inference time by structured system prompts, RAG, and custom tool calls coordinated by a supervisor agent. This approach enables local deployment, explicit citation requirements, and multimodal reasoning without fine tuning model weights.

Figure 3: Architectural comparison of three RAG configurations. (a) Baseline RAG: similarity search over a flat, text only vector store. (b) Advanced RAG (this work): metadata aware retrieval using a structured knowledge map and version control to prefer authoritative sources. (c) Agentic Visual-RAG with a human in the loop (this work).

Figure 4: End to end Visual-RAG pipeline in RADIANT-LLM. Documents are parsed into linked text, visual, and metadata knowledge bases (ingestion), embedded into a local vector index (embedding), and retrieved to build context for generation by a pretrained LLM (retrieval).

Tool Name	Description
PDFReaderTool (Core)	Main RAG engine for documents. Parses PDFs using text and vision models, updates the local JSON KB with text, equations, tables, figure descriptions, and metadata, embeds content, and performs semantic search.
ImageAnalysisTool	Vision focused pipeline that extracts technical information from standalone images (plots, schematics, flow diagrams) and stores textual descriptions and tags in the JSON KB.
CSVDataQueryTool	Performs focused RAG style queries on CSV or Excel files, complementing the capabilities of CSVDataFinderTool .
Web Search Tools	Optional tools that connect to public search engines and Wikipedia for current retrieval of non sensitive information and references. Results can be inspected and, if appropriate, added to the local KB.
FileDownloaderTool	Retrieves documents from open repositories (e.g., OSTI or NRC ADAMS) into the secure local directory, where they are ingested by the PDF pipeline.
PythonREPLTool	Executes Python code for calculations, simple data analysis, and scripted operations on files within the working directory.

Table 1: Custom tool suite used by RADIANT-LLM. The supervisor agent selects these tools to execute workflows while keeping raw data local.

Figure 5: Comparison of text only and multimodal PDF parsing on a calculus page. (a) Original PDF page [ 30 ] showing text, equations, and a figure. (b) Output from the Nougat parser showing loss of visual information (modified from Figure B.1 in [ 7 ] ). (c) RADIANT-LLM parsing output, where a vision language model recovers the diagram semantics (labels and topology) into a retrievable JSON representation.

Figure 6: Technical pages used in the page level benchmark. Left: homogeneous cylindrical reactor neutron flux and heat generation profiles (Figure 3–3). Right: thermal shield geometry and power deposition model (Figure 3–7). These pages contain dense visual annotations and served as sources for figure only, text only, and hybrid queries.

Model	Case	CoP	CiP	CiH	HR $\downarrow$	ViR
GPT-5.2	Calculus (Q1–5)	1.0000	1.0000	1.0000	0.0000	1.0000
GPT-5.2	Reactor Physics (Q6–10)	0.9000	1.0000	1.0000	0.0180	0.9700
GPT-5.2	Thermal Shield (Q11–15)	1.0000	1.0000	1.0000	0.0000	1.0000
GPT-5.2	Average (15Q)	0.9667	1.0000	1.0000	0.0060	0.9900
GPT-5.1	Calculus (Q1–5)	0.7911	1.0000	1.0000	0.3500	0.2767
GPT-5.1	Reactor Physics (Q6–10)	0.7000	1.0000	1.0000	0.4056	0.2917

Table 2: Overall average performance across page level Visual-RAG queries. Reported metrics include CoP, HR (lower is better), and ViR. Averages are computed over three cases (15 queries total).

Panel of Figure 7 — Figure 7: Page level benchmark results averaged over 15 queries. Shown are mean CoP ( $\overline{\mathrm{CoP}}$ ), HR ( $\overline{\mathrm{HR}}$ , lower is better), and ViR ( $\overline{\mathrm{ViR}}$ ) for GPT-5.2, GPT-5.1, GPT-4.1, and GPT-4o. Higher ViR generally coincides with higher CoP and lower HR, indicating the importance of visual grounding for factual correctness.

Query	Case	Model	Observed Failure	Impacted Metrics	Model Summary
Q1	Calculus	GPT-4o	Visual omission	CoP $\downarrow$ , ViR $\downarrow$	The model reported that the figure contains no geometric information, despite being explicitly labeled with dimensions ( $10ft$ and $6ft$ ).
Q1	Calculus	GPT-4.1	Axis vs. dimension confusion	CoP N $\downarrow$ , HR $\uparrow$	Interprets plotted coordinate axes as physical dimensions and fabricates a width of “4”.
Q1	Calculus	GPT-5.1	Partial numeric hallucination	CoP N $\downarrow$ , HR $\uparrow$	Correctly identifies schematic intent but fabricates vertical dimensions (“2 ft”, “4 ft”).
Q6	Reactor Physics	GPT-4o	Figure not recognized	CoP $\downarrow$ , ViR $\downarrow$	Claims Figure 3–3 provides no spatial or coordinate information despite clearly labeled geometry.
Q10	Reactor Physics	GPT-5.1	Axial omission	ViR $\downarrow$ , CoP S $\downarrow$	Correctly interprets radial profiles but incorrectly states that axial dependence is absent.
Q15	Thermal Shield	GPT-4o	Geometry abstraction error	CoP S $\downarrow$	Models the thermal shield as a cylindrical shell instead of a slab, contradicting explicit textual assumptions.

Table 3: Representative failure modes observed in the page level benchmark. The table summarizes characteristic error patterns across models and queries and highlights how different failure mechanisms impact CoP, ViR, and HR.

Panel of Figure 8 — Figure 8: Effect of knowledge base fidelity on downstream model performance. Page level benchmark results averaged across all 15 queries, comparing GPT-5.2 queried on its own KB, GPT-4o queried on a GPT-5.2 built KB, and GPT-4o queried on its own KB. Querying GPT-4o on a higher fidelity multimodal KB improves CoP and ViR and reduces HR.

Query	Case	Baseline: (GPT-4o on its own KB)	GPT-4o on GPT-5.2 KB	Observed Improvement Mechanism
Q1	Calculus	Confused plotted axes with physical dimensions; fabricated unlabeled distances (e.g., “4 ft”), reducing CoP and increasing HR.	Correctly identifies labeled dimensions (10 ft, 6 ft) as physical lengths and explicitly rejects axis interpretation; no fabricated values.	High fidelity visual fact extraction during KB construction eliminated ambiguity between coordinate axes and physical dimensions, reducing hallucination and restoring numeric precision.
Q6	Reactor Physics	Claims figure provides no spatial or coordinate information; omits axial markers and extrapolation lengths.	Correctly identifies radial and axial directions, extrapolated boundaries, and cylindrical geometry using figure labels.	Dense visual annotations (e.g., $r=0$ , $R$ , $R_{e}$ , $\pm L/2$ , $\pm L_{e}/2$ ) preserved in the KB enabled accurate visual grounding without requiring stronger model reasoning.
Q15	Thermal Shield	Applies incorrect geometric abstraction (cylindrical shell) despite textual assumptions, reducing CoP.	Correctly treats the thermal shield as a slab and the core as an infinite plane source, consistent with both text and figure.	Explicit encoding of geometric assumptions during ingestion reduced reliance on implicit reasoning, preventing higher level modeling errors.

Table 4: Representative query level improvements when GPT-4o queried a GPT-5.2 constructed knowledge base versus its own page level KB. Higher fidelity visual parsing during KB construction improved answer correctness, visual grounding, and hallucination suppression for a lower capability reasoning model.

Figure 9: Average CoP, CiP, CiH, HR (lower is better), and ViR across UNFSF queries for different GPT-5.2 deployments.

Query	Model	Observed Behavior	Failure Mode	Metric Impact
Q9	TAMU Chat	Correctly states Pu-239 detection mass but assigns incorrect shielding material and confidence level; omits false alarm constraint.	Confident numeric substitution inconsistent with UNFSF Section 3.3.2 requirements.	CoP $\downarrow$ , HR $\uparrow$
Q11	TAMU Chat	Claims detection of 10 g U-233 within $\sim$ 1 s under lead shielding.	Severe numeric hallucination contradicting UNFSF thresholds by orders of magnitude.	CoP $\downarrow$ , HR $\uparrow$
Q12	TAMU Chat	Provides precise metal and explosive detector thresholds not present in UNFSF text.	Fabrication of plausible but unsupported detector performance values.	CoP $\downarrow$ , HR $\uparrow$
Q6	GPT-5.2 Frozen	Identifies surveillance/CCTV as the selected dual use system.	Incorrect system selection; UNFSF specifies an exit/entry control system (Section 3.2).	CoP $\downarrow$ , HR $\uparrow$
Q4 to Q8	GPT-5.2 Frozen	Cites NRC and IAEA regulations accurately but fails to recover UNFSF design logic.	Source drift: correct external citations without document specific grounding.	CiP $\uparrow$ , CiH $=0$
Q1 to Q3	ChatGPT (web)	Explains 3S concepts correctly but fails to recover figure specific overlap labels.	Partial visual grounding without figure level semantic recovery.	ViR $\downarrow$ , CiH $=0$

Table 5: Representative per query failure modes explaining metric degradation in general purpose GPT-5.2 deployments on the UNFSF benchmark. Each case highlights confident but incorrect behavior or citation drift observed in the YAML scoring files.

Context level	$\overline{\text{CoP}}$	$\overline{\text{CiP}}$	$\overline{\text{CiH}}$	$\overline{\text{HR}}$	$\overline{\text{ViR}}$
1 source (27p)	0.979	0.848	0.583	0.000	1.000
5 sources (265p)	0.946	1.000	1.000	0.036	0.780
10 sources (408p)	0.946	0.958	0.917	0.031	0.953
20 sources (826p)	0.967	1.000	0.917	0.014	1.000
50 sources (2004p)	0.849	1.000	0.667	0.163	0.952
100 sources (6349p, R1)	0.599	0.708	0.750	0.467	0.083

Table 6: Average performance metrics for GPT-5.2 evaluated on the UNFSF visual-RAG benchmark as a function of retrieved context size.

Panel of Figure 10 — Figure 10: Context scaling for GPT-5.2 on the UNFSF Visual-RAG benchmark. Shown are average CoP, CiP, CiH, HR (lower is better), and ViR as a function of retrieved context size. Panel (a) includes all context levels from run 1 and highlights a degraded regime at 100 sources. Panel (b) shows a repeat of the 100 source setting (run 2), where performance returned to the typical high performance band.

Q	Question focus	Context regime	Observed behavior	Impacted metrics
Q1	3S shared functions (figure)	100 (R1)	Central overlap omitted	ViR $\downarrow$ , HR $\uparrow$
Q2	Domain objectives (figure)	100 (R1)	Role labels partially inferred	ViR $\downarrow$
Q3	Safeguards and security interface	100 (R1)	Interface functions mixed	HR $\uparrow$
Q9	Pu-239 detection requirements	100 (R1)	False alarm constraint omitted	CoP $\downarrow$
Q10	SNM false alarm limits	100 (R1)	Isotope coverage incomplete	CoP $\downarrow$
Q7	Dual use system justification	10 to 50	Extra contextual citations introduced	CiH $\downarrow$

Table 7: Representative diagnostic questions illustrating failure modes and recovery trends across context scaling regimes. Each row highlights the question focus, observed model behavior, and impacted metrics.

Agentic RAG

Agentic retrieval-augmented generation lets language models plan searches, use tools, and refine evidence before producing grounded answers.

See all 246 papers

Jun 28, 2026

LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters. We frame this task as planning: an LLM-based agent observes the evidence collected so far, picks the next document fragment to inspect, and stops when the evidence is sufficient. The agent operates over a vectorless document tree using browse, read, and search tools, and maintains a dynamic knowledge graph (KG) as state. On a 200-question benchmark over NuScale Final Safety Analysis Report (FSAR) documents, the system reaches 81.5% accuracy with a RAGAS Faithfulness of 0.93. The dominant performance factor is planning: against PageIndex, which uses the same document tree without state-conditioned action selection, the gap is +38.0pp (43.5% to 81.5%, p<0.001). The system also outperforms LightRAG (73.0%, p<0.05), HippoRAG (70.5%, p<0.01), and GraphRAG (49.5%, p<0.001), and matches RAPTOR (75.5%, p=0.11) without offline indexing. Edge inference adds 2.8x cost without raising accuracy; we retain it as a traceability module. Of 7,391 inferred edges, 3 Violates edges (0.04%) flag scope boundaries (Q058) and partial conformance (Q176) as typed annotations that a human reviewer can audit.

Mingyu Jeon, Bokyeong Kim, Suwan Cho +2

1MODULABS, Seoul, Republic of Korea · 2Korea Atomic Energy Research Institute, Daejeon, Republic of Korea · University of Science & Technology, Daejeon, Republic of Korea

Retrieval-Augmented Generation Legal Document Analysis Document QA LLM Agent Planning LLM Reasoning

Ask PDF

Sep 10, 2026

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions -- non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents -- the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.

Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser

Faculty of Engineering University of Ottawa Ottawa, Canada

LLM Safety Benchmarks LLM Safety RAG Security RAG Evaluation

Ask PDF

Jun 11, 2026

SafeLLM: Extraction as a Hallucination-Resistant Alternative to Rewriting in Safety-Critical Settings

Large language models (LLMs) are increasingly used to access organisational documentation, including standard operating procedures (SOPs), HR policies and institutional guidelines. However, retrieval-augmented generation (RAG) systems that rely on free-form rewriting can introduce hallucinations and unstable trade-offs between completeness and conciseness, particularly in safety- and compliance-critical settings. Objectives: To evaluate extraction as a hallucination-resistant alternative to rewriting-based RAG and compare strategies that balance precision, recall and safety across document types and model scales. Methods: We compare multiple prompting strategies, including line-number-based source selection, extraction of relevant guideline sentences with explicit safety annotations, and a multi-stage pipeline that refines draft answers using supporting evidence from source guidelines. Experiments are conducted on documents of varying length and structure, including local NHS acute care and oncology guidelines and UK-wide NICE guidelines, using both frontier-scale and locally deployable models. Performance is assessed using automatic metrics and human expert evaluation of relevance and completeness. Results: Line-number selection achieves the strongest results, outperforming direct copying and safety-focused strategies across both large and small models while maintaining high term recall (up to 95%) and close alignment with source text. Safety-oriented approaches improve precision but introduce systematic omissions, while multi-stage filtering further amplifies this trade-off. Performance varies with document structure: line-based extraction excels in protocol-like content, whereas alternative strategies perform better on more verbose documents (up to 97% term recall).

Julia Ive, Felix Jozsa, Evridiki Georgaki +6

Institute of Health Informatics, University College London, London, UK · National Hospital for Neurology and Neurosurgery, Queen Square, London, UK · Somerset NHS Foundation Trust, UK +2

Retrieval-Augmented Generation LLM Grounding LLM Hallucination Mitigation RAG Evaluation

Ask PDF

Abstract

Agentic RAG

Explore similar work

LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

SafeLLM: Extraction as a Hallucination-Resistant Alternative to Rewriting in Safety-Critical Settings