MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning
Authors: Suifeng Zhao, Zida Liu, Xinyu Lei, Lei Sun, Jun Gao, Sujian Li
Organizations: Key Laboratory of High Confidence Software Technologies, CS, Peking University, China · Key Laboratory of Computational Linguistics(MOE), CS, Peking University, China · Panasonic Connect Co., Ltd. Tokyo, Japan
Multimodal Retrieval-Augmented Generation (RAG) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs. However, current RAG and SFT-based methods struggle to achieve robust cross-modal reasoning, causing imprecise visual citations or decoupling between the citation and the generated answers. To address these limitations, we propose MCite-RL, a citation-enhanced agentic reinforcement learning framework designed for reliable multimodal RAG. MCite-RL introduces an Agentic Refinement module for visual citation that employs iterative retrieval, reasoning, and recursive cropping to progressively narrow the search space, transforming citation into a dynamic, evidence-driven reasoning process rather than a static step. Furthermore, we incorporate a Citation-enhanced Reward mechanism that integrates both process-level and outcome-level feedback within a reinforcement learning paradigm to jointly optimize answer accuracy and source traceability. Extensive experiments on benchmarks such as Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc demonstrate that MCite-RL effectively achieves the joint optimization of citation precision and answer quality.
Figures & tables
Figure 1: Typical success and failure patterns of visual citation in multimodal RAG. A reliable case (top) exhibits precise visual citation that directly supports the generated answer. In contrast, existing models frequently suffer from imprecise citations (middle) or scenarios where citations become decoupled from the generated answer (bottom).
Figure 2: Architecture of MCite-RL. It features a two-stage pipeline: (1) Trajectory Synthesis with rigorous quality control for supervised fine-tuning (SFT), and (2) RL-based Agentic Refinement for visual citation, which leverages multi-step rollouts (retrieval and cropping) and citation-enhanced rewards to jointly optimize answer accuracy and citation precision.
Method
Wiki-Visa
FinRAGBench-V
MMLongBench-Doc
Ans. (%)
Cit. (%)
Ans. (%)
Cit. (%)
Ans. (%)
Cit. (%)
Proprietary Models
GPT-4o
31.40
3.40
21.23
19.91
25.51
10.80
Gemini-2.0-flash
39.60
0.08
20.44
6.31
25.56
2.83
Claude-4.5-sonnet
12.80
0.53
34.58
15.49
30.35
16.97
Qwen2.5-VL-3B-Instruct
Table 1: Main results of MCite-RL compared with baseline methods on Wiki-Visa, FinRAGBench-V, and MMLongBench-Doc. NOTE: the proprietary models are only tested under the setting of MCite-RL (w/o Training). Bold and underline denote the best and second-best performance, respectively. The blue-shaded rows represent our method and its variants.
Figure 3: Training dynamics comparison across four key metrics. The orange solid lines represent MCite-RL, while the blue solid lines denote the baseline VRAG-RL (w/o Citation Reward).
Configuration
Ans. (%)
Cit. (%)
MCite-RL (Full)
60.00
36.05
Reward Components
w/o Process-level Cit.
57.60
34.50
w/o Outcome-level Cit.
54.00
13.15
w/o All Cit. Rewards
54.20
20.56
w/o Ans. Reward
53.20
41.65
Table 2: Component-wise Ablation for Reward Function on Wiki-VISA (7B). We categorize components into reward items and training constraints to analyze their contributions.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Wiki-VISA
FinRAGBench-V
MMLongBench-Doc
GPT-4o
17.00
27.59
26.56
Gemini-2.0-flash
0.60
12.20
6.22
Claude-4.5-sonnet
2.20
28.15
26.79
Qwen2.5-VL-3B-Instruct
Vanilla RAG
1.20
0.29
2.63
ReAct
0.00
0.00
0.00
Appendix
Table 3: Answer-citation consistency (%) of MCite-RL compared with other methods. Higher scores indicate better semantic alignment between generated answers and cited regions.
Figure 4: This is a qualitative comparison showing the impact of the Citation-enhanced Reward. Without the reward (top), incorrect cropping leads to information loss and hallucinations; with the reward (bottom), the model precisely locates critical evidence to generate accurate answers and supporting citations.
Component
Instruction
Character Introduction
You are an intelligent assistant that performs search, visual grounding, and precise question answering. You must reason step by step and use tools when necessary. If the answer is uncertain, use the search tool. If a specific image region is needed, use the crop tool.
Available Tools
search : Retrieve relevant information using keywords or questions. crop : Crop a region from the image. answer : Provide the final answer with a visual citation (bounding box).
Workflow
Step 1: SEARCH - You MUST begin with a search. - Do NOT skip this step. Step 2: CROP - Progressively crop the region after search. - The first crop should be a large semantic region and must fully contain the ground-truth citation region. - Subsequent crops should become smaller and more focused. - The final crop should still contain the citation region and be close to, but not identical to, the final citation.
Iterative Refinement
After each step, evaluate whether the information is sufficient. If sufficient, continue refining the current region until it is precise enough to support the answer. If insufficient, perform a new search with refined keywords and restart refinement from a larger region.
Refinement Strategy
Follow a hierarchical zoom-in process: Large region → Intermediate region → Near-final region.
Constraints
- Do NOT over-crop irrelevant images. - Early crops MUST NOT match or approximate the final citation. - Each crop MUST be smaller and more focused than the previous one. - The final citation MUST EXACTLY match ground-truth coordinates.
Appendix
Table 4: Trajectory generation prompt used for synthesizing agentic SFT trajectories.
Criteria for Trajectory Quality Control
A trajectory is retained only if the MLLM:
(i) Successfully retrieves the correct image, ensuring relevant visual context is available;
(ii) Generates the correct answer, guaranteeing semantic correctness;
(iii) Predicts a valid citation box with high overlap with the ground-truth citation region;
(iv) Ensures the first crop fully contains the ground-truth citation box, providing a valid starting region for subsequent iterative focusing.
Appendix
Table 5: Filtering criteria for trajectory synthesis. A trajectory is retained only if it satisfies all the conditions.
Role
Trajectory Content (Reasoning, Action, and Observation)
User
Question: who played the preacher in the movie uncle drew
Model
<think> I need to find an image that identifies which actor played the character Preacher in the movie Uncle Drew. I’ll start by searching for an image of the cast or a relevant listing that mentions ’Preacher’ and the actor’s name. </think> Action: <search>Uncle Drew Preacher actor</search>
Environment
[System returns retrieved full-page screenshot]
Model
<think> The Wikipedia page likely contains the cast list where the character Preacher is mentioned. I will start with a large crop over the lower section to see if any cast or relevant lines mention ’Preacher’, then progressively narrow down. </think> Action: <bbox>[22, 895, 531, 1119]</bbox> (Coarse Crop)
Environment
[System returns coarse cropped image]
Model
<think> The first crop shows the ’Cast’ section… I need to zoom in further to the specific line mentioning ’Preacher’ to identify the actor who played that role. </think> Action: <bbox>[6, 257, 1164, 412]</bbox> (Fine Crop)
Appendix
Table 6: Synthesized Trajectory Data Sample. This table visualizes a complete multi-turn interaction where the model performs search and progressive cropping to answer the user query. Note how the bounding box coordinates refine from coarse to fine.
Principle
Guideline
Avoid (What to reject)
1. Completeness
The bounding box must fully encompass the visual evidence (text span or data row) required to answer the query.
Cutting off key context or truncated sentences.
2. Compactness
The box should be the smallest possible rectangle that encloses the evidence. It must exclude irrelevant surrounding sentences or whitespace.
Including the whole paragraph or page unnecessarily.
3. Table Logic
For structured data (tables/lists), annotate the specific row(s) containing both the entity and the value to preserve semantic meaning.
Selecting only the value cell without row headers.
4. Relevance
If the answer appears in multiple locations, select the single most informative region (e.g., the main definition rather than a summary).
Annotating multiple duplicate regions or ambiguous footnotes.
Appendix
Table 7: Annotation Principles for Visual Citation. The ground truth bounding box must balance completeness, compactness, and relevance.
Reward Component
Metric
Weight
Retrieval Reward
NDCG
0.15
Answer Reward
Exact Match
0.40
Citation Outcome Reward
IoU
0.35
Citation Process Reward
IoGT
0.10
Appendix
Table 8: Reward components, corresponding metrics, and weights used in RL optimization.
Hyperparameter
Value
Base Model
Qwen2.5-VL-7B-Instruct
Training Epochs
3
Learning Rate
1.0×10−5
LR Scheduler
Cosine (Warmup ratio 0.1)
Global Batch Size
16
Gradient Accumulation
2
Appendix
Table 9: Hyperparameters for SFT
Hyperparameter
Value
Algorithm
GRPO
Training Epochs
1
Actor Learning Rate
1.0×10−6
KL Coefficient
0.01
Global Train Batch Size
8
Max Prompt Length
8,192
Appendix
Table 10: Hyperparameters for RL
Dataset
Retrieval Corpus
QA Pairs
Wiki-VISA
12,387
500
FinRAGBench-V
3,011
1,398
MMLongBench-Doc
6,527
418
Appendix
Table 11: Data Statistics of the specific retrieval corpus size and validation set size for each dataset.
Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires external textual evidence beyond the image and the model's parametric knowledge. Existing multimodal RAG systems commonly rely on Top-K retrieval or reranking, which may return redundant passages and provide limited control over whether an answer update is sufficiently supported by the retrieved evidence. We propose CLIMB, a training-free inference-time framework for multimodal RAG. CLIMB first constructs a compact complementary evidence pool using an MMR-style objective that balances query relevance and passage-level redundancy. It then performs confidence-controlled refinement within this fixed pool: an R/E/C critic scores passages by relevance, evidence specificity, and cross-modal alignment, while an evidence-grounded confidence estimator accepts an updated answer only when the estimated confidence increases. This design provides a simple stopping criterion and reduces unnecessary refinement without modifying the underlying retriever or MLLM. Experiments on Encyclopedic-VQA and InfoSeek show that CLIMB consistently improves over retrieval-augmented multimodal baselines. Ablations further indicate that complementary pooling, critic-based scoring, and iterative confidence-controlled refinement each contribute to the final performance.
Retrieval-Augmented Generation (RAG) mitigates hallucinations in Multimodal Large Language Models (MLLMs), yet existing systems struggle with complex cross-modal reasoning. Flat vector retrieval often ignores structural dependencies, while current graph-based methods rely on costly ``translation-to-text'' pipelines that discard fine-grained visual information. To address these limitations, we propose \textbf{MG2-RAG}, a lightweight Multi-Granularity Graph RAG framework that jointly improves graph construction, modality fusion, and cross-modal retrieval. MG2-RAG constructs a hierarchical multimodal knowledge graph by combining lightweight textual parsing with entity-driven visual grounding, enabling textual entities and visual regions to be fused into unified multimodal nodes that preserve atomic evidence. Building on this representation, we introduce a multi-granularity graph retrieval mechanism that aggregates dense similarities and propagates relevance across the graph to support structured multi-hop reasoning. Extensive experiments across four representative multimodal tasks (i.e., retrieval, knowledge-based VQA, reasoning, and classification) demonstrate that MG2-RAG consistently achieves state-of-the-art performance while reducing graph construction overhead with an average 43.3× speedup and 23.9× cost reduction compared with advanced graph-based frameworks.
Sijun Dai, Qiang Huang, Xiaoxing You +1
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen) · School of Computer Science, Hangzhou Dianzi University
We introduce MiRAGE, an evaluation framework for retrieval-augmented generation (RAG) from multimodal sources. As audiovisual media becomes a more prevalent source of information online, RAG systems must integrate such media into generation. Yet, existing evaluation methods for RAG are largely text-centric and do not readily transfer to multimodal settings. MiRAGE is a claim-centric approach to multimodal RAG evaluation, consisting of InfoF1, which assesses factuality and information coverage, and CiteF1, which assesses citation support and completeness. We show that, when applied by humans, MiRAGE strongly aligns with extrinsic judgments of output quality. We additionally introduce an automatic implementation of MiRAGE and compare it to multimodal variants of three prominent text-centric RAG metrics---ALCE, ARGUE, and RAGAS---finding that MiRAGE outperforms all three on text while being the only one to generalize to multimodal sources. We release open-source implementations and outline evaluation methods for multimodal RAG.
Alexander Martin, William Walden, Reno Kriz +5
Johns Hopkins University · Human Language Technology Center of Excellence