cs.CLJan 19, 2025

Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks

Authors: Diego Gosmar, Deborah A. Dahl

Organizations: Torino, TO 10100, Italy

Abstract

Hallucinations remain a significant challenge for Generative AI, undermining trust in AI systems and their reliability. This study investigates how orchestrating multiple specialized AI agents can mitigate hallucinations, with a focus on systems that use Natural Language Processing (NLP) for agent interaction. We design a pipeline that feeds 310 prompts, purposefully crafted to induce hallucinations, into a front-end agent; its outputs are reviewed and refined by second- and third-level agents that detect unverified claims, add explicit disclaimers and clarify speculative content. We introduce four novel Key Performance Indicators (KPIs) - Factual Claim Density, Factual Grounding References, Fictional Disclaimer Frequency and Explicit Contextualization Score - aggregated into a Total Hallucination Score (THS) and assessed by a dedicated fourth-level evaluator agent. Inter-agent context is carried by the Open Floor Protocol (OFP, formerly OVON, Open Voice Network), whose structured JSON messages let each reviewer pass its assessment of hallucination likelihood, and the reasons behind it, to the next stage. Across the 310 prompts, THS improves by a mean of -0.131 per prompt from the front-end agent to the final reviewer (Cohen's d = 2.38), with lower scores on 302 of 310 prompts. The gain is carried mainly by explicit disclaimers and contextualization, making speculative content more transparent and clearly demarcated from factual claims. The results show that multi-agent orchestration with structured, NLP-based exchange of meta-information can substantially strengthen the epistemic framing of AI-generated responses and improve their explainability.

Figures & tables

Explore similar work

CardsList
  1. Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching

    May 27, 2026Diego Gosmar, Deborah A. DahlAI Agent ReliabilityMulti-Agent LLM Systems

  2. Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor

    Jul 21, 2025Siyuan Liu, Wenjing Liu, Zhiwei Xu +3LLM Hallucination MitigationHallucination in Language Models