Agri-MRAG: Parameter-Enhanced Multimodal Retrieval-Augmented Generation for Agricultural Visual Question Answering
Organizations: School of Information Science and Engineering, Xinjiang College of Science and Technology, Korla 841000, China · School of Computer Science, Wuhan University, Wuhan 430072, China
Abstract
Multimodal Retrieval-Augmented Generation (MRAG) has shown promise for Agricultural Visual Question Answering (Agri-VQA), but existing methods still face challenges in retrieving relevant knowledge while avoiding redundancy and in integrating retrieved evidence beyond simple prompt concatenation. To address these issues, we propose Agri-MRAG, a multimodal retrieval-augmented framework for agricultural VQA. Specifically, we first introduce a semantic-consistency-guided clustering mechanism that combines multimodal clustering and weighted relevance scoring to select high-confidence knowledge subsets and adaptively determine the retrieval size . We then design a parameterized knowledge injection strategy, where retrieved agricultural knowledge is encoded into LoRA adapters and merged into the feed-forward network (FFN) layers of the MLLM, allowing external knowledge to influence generation through parameter-level adaptation. In addition, a perplexity-guided (PPL-guided) routing mechanism is employed to switch between implicit parametric reasoning and explicit retrieval-assisted generation according to model uncertainty, balancing inference efficiency and answer reliability. For evaluation, we use PlantVillageVQA as a broad agricultural benchmark and construct PlantVQA, a region-specific forestry and fruit crop VQA dataset containing 7,514 images and 15,028 expert-verified question-answer pairs. Experiments show that Agri-MRAG achieves ROUGE-L scores of 41.21 and 46.58 on the two datasets, outperforming mRAG by 11.70 and 4.51 points, and VisRAG by 8.80 and 3.39 points, respectively.
Figures & tables
| Experimental Environment | Experimental Configuration |
| Python Version | 3.10.19 |
| Operating System | Ubuntu 20.04.1 LTS |
| GPU | NVIDIA GeForce RTX 4090 8 |
| GPU Memory | 24 GB per GPU |
| System Memory | 256 GB |
| Framework | PyTorch 2.8.0 |
| Dataset | Train Set | Test Set | Total |
| PlantVillageVQA | 154,977 | 4,000 | 158,977 |
| # Percentage | 97.48% | 2.52% | - |
| PlantVQA | 12,038 | 2,990 | 15,028 |
| # Percentage | 80.10% | 19.90% | - |
| Model | Params(B) | GFLOPS | PlantVillageVQA | PlantVQA | ||||
| BLEU | ROUGE-L | METEOR | BLEU | ROUGE-L | METEOR | |||
| Qwen3-VL-4B Bai et al. (2025) | 4 | 1,200 | 12.74 | 34.34 | 19.12 | 21.57 | 38.09 | 28.53 |
| Qwen3-VL-8B Bai et al. (2025) | 8 | 2,400 | 13.79 | 34.53 | 19.56 | 22.90 | 43.41 | 29.43 |
| LLaVA-OneVision-2-8B An et al. (2026) | 8 | 2,500 | 4.94 | 14.02 | 14.41 | 4.60 | 1.50 | 17.49 |
| GLM-4.1V-9B-Thinking Hong et al. (2025) | 9 | 2,700 | 13.42 | 33.08 | 19.25 | 18.57 | 34.01 | 27.96 |
| AgriChat (7B) Boudiaf et al. (2026) | 7 | 2,100 | 7.60 | 21.57 | 23.58 | 5.51 | 3.94 | 17.50 |
| Item | mRAG | VisRAG | Agri-MRAG |
| Retrieval Budget (Top-K) | 10 | 10 | 10 (adaptive ) |
| Prompt Length (tokens/sample) | 197.41 | 197.15 | 179.10 |
| Context Length (tokens/sample) | 172.41 | 172.15 | 71.71 |
| Training / Tuning Procedure | No training | No training | LoRA adapter training |
| Decoding Settings | Greedy decoding | Greedy decoding | Greedy decoding |
| Compute Budget | Retrieval + Inference | Retrieval + Inference | Adapter Training + Retrieval + Inference |
| Module | PlantVillageVQA | PlantVQA | ||||||
| Cluster | Adapter | PPL | BLEU | ROUGE-L | METEOR | BLEU | ROUGE-L | METEOR |
| 11.75 | 29.10 | 19.25 | 21.14 | 41.71 | 27.91 | |||
| ( 0.11) | ( 0.30) | ( 0.20) | ( 0.13) | ( 0.14) | ( 0.14) | |||
| ✓ | 14.20 | 34.93 | 22.31 | 23.33 | 45.59 | 30.57 | ||
| ( 0.23) | ( 0.26) | ( 0.34) | ( 0.29) | ( 0.34) | ( 0.15) | |||
| ✓ | ✓ | 16.05 | 38.88 | 24.42 | 23.82 | 46.13 | 31.11 | |
| Top-K | PlantVillageVQA | PlantVQA | ||||||||
| BLEU | ROUGE-L | METEOR | ITPS | Token_doc | BLEU | ROUGE-L | METEOR | ITPS | Token_doc | |
| 11.87 | 29.45 | 19.48 | 9.78 | 0.00 | 21.00 | 41.60 | 27.77 | 1.85 | 0.00 | |
| 15.91 | 37.96 | 23.86 | 12.39 | 30.10 | 23.34 | 45.75 | 30.63 | 5.90 | 28.56 | |
| 16.71 | 41.21 | 25.62 | 13.92 | 70.07 | 24.29 | 46.58 | 31.73 | 7.69 | 73.91 | |
| 18.68 | 42.34 | 26.63 | 15.37 | 116.53 | 23.88 | 46.35 | 31.16 | 9.33 | 124.05 | |
| 19.13 | 43.04 | 27.20 | 16.90 | 165.03 | 24.08 | 46.58 | 31.23 | 10.88 | 178.38 | |
| Cluster Number | PlantVillageVQA | PlantVQA | ||||
| BLEU | ROUGE-L | METEOR | BLEU | ROUGE-L | METEOR | |
| 16.39 | 40.29 | 25.41 | 23.89 | 46.28 | 31.17 | |
| 16.52 | 40.55 | 25.58 | 23.70 | 46.18 | 30.94 | |
| 16.71 | 41.21 | 25.62 | 24.29 | 46.58 | 31.73 | |
| 16.49 | 40.53 | 25.51 | 23.61 | 45.75 | 30.87 | |
| PPL Threshold | PlantVillageVQA | PlantVQA | ||||||
| BLEU | ROUGE-L | METEOR | ITPS (s) | BLEU | ROUGE-L | METEOR | ITPS (s) | |
| 19.69 | 45.76 | 28.18 | 15.49 | 24.51 | 46.63 | 32.11 | 10.89 | |
| 16.71 | 41.21 | 25.62 | 15.03 | 24.29 | 46.58 | 31.73 | 10.41 | |
| 14.43 | 34.80 | 22.25 | 14.61 | 23.35 | 45.61 | 30.59 | 10.27 | |
| 13.28 | 32.55 | 20.96 | 15.05 | 23.32 | 45.65 | 30.53 | 10.21 | |
| PPL Threshold | PlantVillageVQA | PlantVQA | ||||||
| Route1 | Route2 | ITPS1 (s) | ITPS2 (s) | Route1 | Route2 | ITPS1 (s) | ITPS2 (s) | |
| 1166 | 2834 | 15.41 | 15.66 | 1671 | 1319 | 10.85 | 10.95 | |
| 1535 | 2465 | 14.72 | 15.22 | 2154 | 836 | 10.32 | 10.62 | |
| 3095 | 905 | 14.38 | 15.40 | 2569 | 421 | 10.26 | 10.29 | |
| 3837 | 163 | 14.97 | 16.98 | 2792 | 198 | 10.19 | 10.46 | |
| Item | PlantVillageVQA | PlantVQA |
| Number of Trained Adapters | 154,977 | 12,038 |
| Number of Stored Adapters | 154,977 | 12,038 |
| Disk Usage | 801 GB | 63 GB |
| Training Time (GPU-hours) | 5.66 | 0.45 |
| Peak Memory | 4344 MB | 4346 MB |
| Loading and Fusion Latency | 0.29 s/adapter | 0.85 s/adapter |
Visual Question Answering
Visual Question Answering (VQA) develops systems that answer natural-language questions about images, videos, and visual documents.
See all 272 papers