AOP-Smart: A RAG-Enhanced Large Language Model Framework for Adverse Outcome Pathway Analysis

Qinjiang Niu; Lu Yan

cs.CLApr 13, 2026

AOP-Smart: A RAG-Enhanced Large Language Model Framework for Adverse Outcome Pathway Analysis

Authors: Qinjiang Niu, Lu Yan

Organizations: Nanyang Normal University, Nanyang, China

Abstract

Adverse Outcome Pathways (AOPs) are an important knowledge framework in toxicological research and risk assessment. In recent years, large language models (LLMs) have gradually been applied to AOP-related question answering and mechanistic reasoning tasks. However, due to the existence of the hallucination problem, that is, the model may generate content that is inconsistent with facts or lacks evidence, their reliability is still limited. To address this issue, this study proposes an AOP-oriented Retrieval-Augmented Generation (RAG) framework, AOP-Smart. Based on the official XML data from AOP-Wiki, this method uses Key Events (KEs), Key Event Relationships (KERs), and specific AOP information to retrieve relevant knowledge for user questions, thereby improving the reliability of the generated results of large language models. To evaluate the effectiveness of the proposed method, this study constructed a test set containing 20 AOP-related question answering tasks, covering KE identification, upstream and downstream KE retrieval, and complex AOP retrieval tasks. Experiments were conducted on three mainstream large language models, Gemini, DeepSeek, and ChatGPT, and comparative tests were performed under two settings: without RAG and with RAG. The experimental results show that, without using RAG, the accuracies of GPT, DeepSeek, and Gemini were 15.0%, 35.0%, and 20.0%, respectively; after using RAG, their accuracies increased to 95.0%, 100.0%, and 95.0%, respectively. The results indicate that AOP-Smart can significantly alleviate the hallucination problem of large language models in AOP knowledge tasks, and greatly improve the accuracy and consistency of their answers.

Figures & tables

Figure 1: Overview of the AOP-Smart framework. (A) In the absence of retrieval augmentation, the LLM directly generates responses based solely on its internal parameters, which may lead to incomplete or hallucinated outputs. (B) With the proposed AOP-Smart framework, user queries are first used to retrieve relevant KE candidates from an indexed KE list, followed by structured expansion to KE, KER, and AOP-level information from the knowledge base. The retrieved structured context is then combined with the user query and fed into the LLM to produce a knowledge-enhanced and more reliable response.

Figure 2: Overview of the KE-based expansion process, including KE augmentation, KER reconstruction, and AOP retrieval, followed by integration with LLM-based reasoning.

Figure 3: Overview of the Python-based system interface, including configuration, input, and output modules, with adjustable parameters for model selection and KE-based retrieval control.

Figure 4: Overview of the 20-question AOP benchmark, grouped into four task categories: KE identification, downstream KE retrieval, upstream KE retrieval, and complex AOP query.

Task	Description	DeepSeek		Gemini		GPT		Overall
		W/O RAG	W/ RAG	W/O RAG	W/ RAG	W/O RAG	W/ RAG	W/O RAG	W/ RAG
Task 1	KE Identification	80%	100%	60%	100%	60%	100%	66.67%	100%
Task 2	Downstream KE retrieval	20%	100%	20%	100%	0%	100%	13.33%	100%
Task 3	Upstream KE retrieval	20%	100%	0%	80%	0%	100%	6.67%	93.33%
Task 4	Complex task	20%	100%	0%	100%	0%	80%	6.67%	93.33%
Overall	All tasks combined	35%	100%	20%	95%	15%	95%	23.33%	96.67%

Table 1: Experimental results of different large language models on AOP-related tasks with and without RAG enhancement.

LLM Hallucination Mitigation

Large language model hallucination mitigation develops ways to detect and reduce unsupported, fabricated, or inconsistent outputs.

See all 178 papers

Oct 26, 2026

OA-RAG: Ontology-based Agentic RAG for Faithful LLM Reasoning

Large language models (LLMs) perform well across many real-world tasks but remain prone to hallucination, limiting their reliability in domains such as biomedicine and healthcare, where answers must be grounded in established knowledge. Retrieval-augmented generation (RAG) mitigates this problem by incorporating external information, but often overlooks logical relationships and relies heavily on the LLM's own reasoning, which can compromise answer faithfulness. We propose OA-RAG, an agent that integrates formal reasoning with LLM reasoning, drawing on both the logical structure and informal annotations of domain ontologies. For each answer, OA-RAG reports the ontology observations used as evidence and a model-assessed grounding level (entailed, supported, related, or unrelated) that characterises the strength of that evidence. Across benchmarks derived from three large-scale ontologies and six real-world medical question-answering tasks, OA-RAG achieves the best overall performance among the evaluated ontology-fine-tuned and popular RAG methods, with particularly strong results on reasoning-intensive tasks. The reported grounding levels also offer a useful indication of answer reliability, with stronger grounding generally associated with higher accuracy. Finally, our results demonstrate the value of ontologies as knowledge sources: OA-RAG is competitive with, and sometimes outperforms, domain-specific RAG systems that draw on sources such as textbooks.

Hui Yang, Jiaoyan Chen, Yongsheng Gao

The University of Manchester, Manchester, United Kingdom · SNOMED International, London, United Kingdom

Agentic RAG Retrieval-Augmented Generation LLM Grounding Ontology-Based Reasoning Medical LLMs

Ask PDF

May 20, 2026

AOP-Wiki EMOD 3.0: Data Model Expansions and Content Evaluation Framework for Using Agentic AI to Improve Integration between AOPs and New Approach Methodologies (NAMs)

Adverse Outcome Pathways (AOP) are logic models that causally link biological mechanisms that can be measured in a lab to adverse outcomes, relevant to chemical regulatory endpoints. AOPs contextualize new approach methodologies (NAMs), in vitro and in silico methods used as alternatives to animal testing and the sequential events in an AOP serve as multi-scale models spanning biological scales. The AOP-Wiki serves as the global repository for AOPs. While the AOP-Wiki has played a central role in AOP expansion over the past decade, constraints within the current data model and application infrastructure limit the AOP-Wiki from supporting continued AOP growth and evolution. Yet, the transformative power of agentic AI has re-invigorated AOP-Wiki data modernization efforts at a time when core AOP principles can be harnessed to inform use of AI for aggregating and structuring AOP-relevant information. Seizing upon this momentum, we present AOP-Wiki EMOD 3.0, the third in a series of evidence model prototypes, which concretely demonstrates data model expansions and our vision for how the AOP-Wiki might be transformed to better serve regulatory science and emergent use of AOPs in biomedical and One Health contexts. We aim to lay a foundation to support computationally-generated AOPs and quantitative AOPs (qAOPs) by focussing on solutions for AOP-Wiki internal quality improvement, evidence structuring to enhance AOP FAIRness and AI-readiness, and improved integration between the AOP framework and NAMs to better serve next generation risk assessment.

Virginia K. Hench, J. Harry Caufield, Sierra A. T. Moxon +2

Open BioData Modeling, Raleigh, NC, 27609, USA · Environmental Genomics and Systems Biology, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA · National Wildlife Research Centre, Environment and Climate Change Canada, Ottawa K1S 5R2, Canada. +1

Topic pending

Ask PDF

Sep 10, 2026

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions -- non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents -- the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.

Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser

Faculty of Engineering University of Ottawa Ottawa, Canada

LLM Safety Benchmarks LLM Safety RAG Security RAG Evaluation

Ask PDF

Abstract

LLM Hallucination Mitigation

Explore similar work

OA-RAG: Ontology-based Agentic RAG for Faithful LLM Reasoning

AOP-Wiki EMOD 3.0: Data Model Expansions and Content Evaluation Framework for Using Agentic AI to Improve Integration between AOPs and New Approach Methodologies (NAMs)

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety