cs.CVSep 25, 2026

MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning

Authors: Peipei Li, Shuhan Xia, Shengyang Liu, Zekun Li, Ran He

Organizations: Beijing University of Posts and Telecommunications · Minzu University of China · NLPR, Institute of automation, Chinese academy of science, Chinese Academy of Sciences

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Real-world multimodal misinformation often involves mixed forgery sources, requiring sample-specific detection strategies. Existing tool-augmented methods rely on predefined workflows or inference-time planning, limiting adaptability or increasing inference cost. To address this issue, we introduce MM-VeriAgent, which learns to verify mixed-source multimodal misinformation with tools. We first build MM-VeriTools, a specialized toolkit for misinformation detection agents. By benchmarking various candidate models and methods on the sub-tasks required by mixed-source detection, we select the strongest for textual, visual, and cross-modal forgery analysis and encapsulate them as callable tools with a unified interface. On top of this toolkit, we train the LVLM agent with reinforcement learning to teach it how to use these tools to better solve mixed-source detection. Since many of the tools are specialized models whose online execution at every rollout severely limits RL efficiency, we further introduce Tool-Execution Cache, which pre-executes candidate tool calls and reuses their cached outputs during training. This preserves multi-step rollouts while reducing online tool execution, largely improving the training efficiency.Experiments on MMFakeBench demonstrate substantial accuracy gains over the base model without explicit tool search at inference time. Ablation and efficiency analyses further validate the learned tool-use policy and show that Tool-Execution Cache reduces online tool executions during training.

Figures & tables

Figure 1: Overview of MM-VeriAgent. Given an image–text post x=(I,T)x=(I,T) , the policy performs a multi-turn rollout: at each turn it either emits a tool-calling action ata_{t} , which ends with a tool-specific stop token and invokes a tool from the MM-VeriTools toolkit (covering textual veracity, visual forensics, and cross-modal consistency), receiving the returned observation oto_{t} , or terminates the interaction and outputs the final verdict together with fine-grained source attribution and supporting evidence. The policy is optimized with GRPO over multi-turn rollouts.
MethodOrganizationAdaptiveSearch-freeLearnedCache
MMD-AgentFixed workflow✗✓✗✗
LRQ-FACTFixed workflow✗✓✗✗
DEFAMETest-time planning✓✗✗✗
T 2 AgentTest-time search✓✗✗✗
OursRL-learned policy✓✓✓✓
Table 1: Comparison of tool-augmented detectors for mixed-source multimodal misinformation detection. Cache denotes an offline tool-execution cache for training.
GroupModel / MethodRoleTypeAccSelected
TVDGPT-5-miniBase modelBase0.63–
GLiNER-medium ( Zaratiana et al. 2024 )Entity extractionAuxiliary0.71✓
Wikipedia searchKnowledge retrievalAuxiliary0.71✓
Google searchWeb retrievalAuxiliary0.75✓
Qwen3-30B ( Yang et al. 2025 )Text fact verificationDirect0.69✓
Faking Fake News ( Huang et al. 2023 )Fake news detectionDirect0.35✗
Table 2: Candidate tool pool and group-wise benchmarking of MM-VeriTools. Each auxiliary tool is attached to the base model, and direct tools are evaluated off the shelf. A candidate is selected only if it outperforms the base model (Acc in bold ).
Figure 2: Overview of cache-assisted training. Stage 1 constructs the Tool-Execution Cache offline via teacher rollouts, and Stage 2 performs GRPO training with tool calls served from the cache.
Figure 3: Cache hit rate and per-sample rollout time versus the number of teacher rollouts. Five rollouts yield a 100% hit rate and cut the rollout time from 40.3 to 6.5 seconds.
MethodModelBinary ClassificationMulti-Class Classification
ACCF1ACCF1
Proprietary MLLMs
Direct PromptingGemini-2.5-Flash78.878.460.560.2
Direct PromptingGemini-2.5-Pro86.783.269.869.5
Direct PromptingGPT-4o74.076.860.949.2
Direct PromptingGPT-5-mini78.676.153.450.6
Table 3: Main results on MMFakeBench. Proprietary and open-source MLLMs predict with a single forward pass without any tool, and specialized methods are executed with their original frameworks. All results are reported in percentage (%) and the best result in each metric is in bold .
ModelMethodBinaryMulti-Class
ACCF1ACCF1
Qwen2.5-VL-7BDirect Prompting72.853.934.325.0
MMD-Agent72.571.741.343.5
Ours70.681.026.026.8
GPT-5-miniDirect Prompting78.676.153.450.6
MMD-Agent82.179.665.265.5
Table 4: Effect of the toolset on untrained models. Prompted policies exploit the compact MMD-Agent toolset better than the extensive MM-VeriTools, which can even hurt the small model.
Figure 4: Effect of training with the toolset fixed to MM-VeriTools. Adding the same 2k samples as supervised data (SFT-10k) barely helps, whereas GRPO unlocks the toolkit.
VariantBinaryMulti-Class
ACCF1ACCF1
w/o TVD tools81.782.163.164.3
w/o VVD tools82.282.667.267.9
w/o CCD tools79.981.458.358.6
w/o evidence reward77.184.866.166.7
MM-VeriAgent83.285.170.471.5
Table 5: Ablation studies. Top: removing each category of verification tools (TVD: textual verification; VVD: visual forensics; CCD: cross-modal consistency). Bottom: removing the evidence-grounded reward. All results in %.
More in this topic

Multimodal Deception Detection

See all 19 papers

Explore similar work

CardsList
Jun 23, 2026

ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text--image framing errors. Existing benchmarks and methods remain poorly matched to this setting: they usually isolate short captions, single images, binary labels, or one manipulation source, while agentic verification remains costly under realistic evidence search. We present ReMMD, a realistic multilingual multi-image agentic verification framework for multimodal misinformation detection. ReMMD includes ReMMDBench, a real-world multimodal misinformation detection benchmark with 500 samples, 2,756 images, five monolingual languages, two cross-lingual settings, three text-length tiers, multi-image posts, five-way veracity labels, eight distortion labels, evidence provenance, and rationales. It also includes ReMMD-Agent, a persistent-memory verifier that decomposes posts into atomic points, builds a reusable evidence set, and predicts structured L1/L2/L3 outputs. Across proprietary systems, open LVLMs, MMD-Agent, and T2-Agent, ReMMD-Agent obtains the best five-way veracity performance, with 41.80% accuracy and 39.12% macro-F1 using GPT-5.2, while reducing cost by 17.5% relative to MMD-Agent and 79.9% relative to T2-Agent. The project is available at https://dang-ai.github.io/ReMMD.

Chenhao Dang, Dantong Zhu, Jun Yang +2

Shanghai Jiaotong University · Shanghai Artificial Intelligence Laboratory · Central South University +2

Multimodal AgentsMultimodal Model Evaluation
AskPDF
Feb 26, 2026

FactGuard: Agentic Video Misinformation Detection via Reinforcement Learning

Multimodal large language models (MLLMs) have substantially advanced video misinformation detection through unified multimodal reasoning, but they often rely on fixed-depth inference and place excessive trust in internally generated assumptions, particularly in scenarios where critical evidence is sparse, fragmented, or requires external verification. To address these limitations, we propose FactGuard, an agentic framework for video misinformation detection that formulates verification as an iterative reasoning process built upon MLLMs. FactGuard explicitly assesses task ambiguity and selectively invokes external tools to acquire critical evidence, enabling progressive refinement of reasoning trajectories. To further strengthen this capability, we introduce a two-stage training strategy that combines domain-specific agentic supervised fine-tuning with decision-aware reinforcement learning to optimize tool usage and calibrate risk-sensitive decision making. Extensive experiments on FakeSV, FakeTT, and FakeVV demonstrate FactGuard's state-of-the-art performance and validate its excellent robustness and generalization capacity.

Zehao Li, Hongwei Yu, Hao Jiang +7

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · University of Science and Technology Beijing, Beijing, China +1

Automated Fact-CheckingTool Use in VLMsVLM ReasoningRL for VLMs
AskPDF
Jul 26, 2026

Verification-Notebook Learning for Source-Aware Multimodal Misinformation Detection

Multimodal misinformation verification is challenging because misleading signals may come from different parts of a post and require different forms of evidence. LVLMs are well suited to this task, but their verification performance often depends on the inference procedure applied to each instance. Existing methods improve this procedure through stronger prompting, retrieval, or deliberation, but rarely retain the verification patterns learned from previous examples. We propose Verification-Notebook Learning (VNL), a non-parametric framework that learns an external verification procedure for a frozen LVLM before inference. VNL builds a compact notebook of decision principles, evidence cues, and recurring pitfalls from prior verification experience. The notebook remains fixed during inference and guides the verification of new examples. Rather than updating model parameters or storing demonstrations, VNL records learned knowledge in an artifact that can be inspected directly. Experiments show that VNL consistently outperforms a range of competitive baselines. Further analyses show that the Verification Notebook improves fine-grained source attribution while remaining compact and interpretable, providing an effective way to accumulate verification knowledge without model training.

Junyuan Tan

Independent Researcher

Multimodal Large Language Models
AskPDF