cs.CVOct 27, 2026

MVRCap: Towards Comprehensive Video Captioning via Hierarchical Multi-Dimensional Verifiable Rewards

Authors: Weihuang Lin, Xianing Chen, Zhibin Wang, Congzhi Zhang, Yiwei Ma, Jiayi Ji, Miao Chen, Jun Song, 

Organizations: Xiamen University · Alibaba Group

Abstract

Despite significant progress, current models struggle to generate comprehensive video captions, constrained primarily by coarse-grained supervision, which often fails to capture spatial-temporal dynamics and the underlying logical intent. To address this issue, we introduce MVRCap, a novel framework that reformulates video captioning as a reinforcement learning task steered by structured verification. Central to our approach is a Fine-Grained Verifiable Reward mechanism, serving as dense supervision to enforce rigorous visual-semantic alignment. Specifically, we construct this reward system anchored by three complementary pillars: a Hierarchical Concept Coverage Reward utilizing our newly constructed MVRCap-QA dataset to ensure multi-dimensional completeness and semantic depth, a Factuality Verification Reward to enhance factual consistency, and a Conciseness Constraint to maximize semantic density. Furthermore, we curate the high-quality MVRCap-SFT dataset, providing structured annotations across six key dimensions, to serve as a robust foundational initialization for the RL policy in this training pipeline. Extensive experiments demonstrate that MVRCap achieves state-of-the-art performance on authoritative benchmarks including VDC, CAPability, and VC4VG.

Figures & tables

Explore similar work

CardsList
  1. Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

    Jul 6, 2026Mingqi Gao, Hongyuan Dong, Yifei Chen +6Reward ModelingVideo Captioning

  2. CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning

    Jun 8, 2026Penghui Yang, Long Xing, Xiaoyi Dong +10Image CaptioningMultimodal Pretraining

  3. VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning

    May 27, 2026Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang +13Image CaptioningObject Hallucination in VLMs