cs.CVNov 12, 2026

Agri-MRAG: Parameter-Enhanced Multimodal Retrieval-Augmented Generation for Agricultural Visual Question Answering

Authors: Fuyong Liu, Xiantao Lei, Xining Zhang

Organizations: School of Information Science and Engineering, Xinjiang College of Science and Technology, Korla 841000, China · School of Computer Science, Wuhan University, Wuhan 430072, China

Abstract

Multimodal Retrieval-Augmented Generation (MRAG) has shown promise for Agricultural Visual Question Answering (Agri-VQA), but existing methods still face challenges in retrieving relevant knowledge while avoiding redundancy and in integrating retrieved evidence beyond simple prompt concatenation. To address these issues, we propose Agri-MRAG, a multimodal retrieval-augmented framework for agricultural VQA. Specifically, we first introduce a semantic-consistency-guided clustering mechanism that combines multimodal clustering and weighted relevance scoring to select high-confidence knowledge subsets and adaptively determine the retrieval size kk. We then design a parameterized knowledge injection strategy, where retrieved agricultural knowledge is encoded into LoRA adapters and merged into the feed-forward network (FFN) layers of the MLLM, allowing external knowledge to influence generation through parameter-level adaptation. In addition, a perplexity-guided (PPL-guided) routing mechanism is employed to switch between implicit parametric reasoning and explicit retrieval-assisted generation according to model uncertainty, balancing inference efficiency and answer reliability. For evaluation, we use PlantVillageVQA as a broad agricultural benchmark and construct PlantVQA, a region-specific forestry and fruit crop VQA dataset containing 7,514 images and 15,028 expert-verified question-answer pairs. Experiments show that Agri-MRAG achieves ROUGE-L scores of 41.21 and 46.58 on the two datasets, outperforming mRAG by 11.70 and 4.51 points, and VisRAG by 8.80 and 3.39 points, respectively.

Figures & tables

Explore similar work

CardsList
  1. Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learning

    Feb 28, 2026Ruoshuang Du, Xin Sun, Qiang Liu +4Multimodal RAGAdaptive RAG

  2. MMAgent-R2^2: Learning to Rerank and Reject for Agentic mRAG

    Jul 8, 2026Tao Zhang, Ziqi Zhang, Zongyang Ma +7Multimodal RAGAdaptive Retrieval

  3. Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation

    May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Multimodal RAGVisual Question Answering