Overview

Video captioning studies how to describe what happens in a clip, from a single concise sentence to dense, timestamped event descriptions and long-form narratives. Systems combine video-language models with temporal segmentation and grounding, frame selection, tracking or motion cues, and—in audiovisual settings—alignment between sounds, speakers, and visible events. Common test settings include short clips and instructional videos, using benchmarks such as MSR-VTT, VATEX, ActivityNet Captions, and YouCook2; newer evaluations also test factuality, temporal accuracy, entity consistency, and whether captions support downstream question answering. A central challenge is balancing completeness and fluent detail against hallucination, especially as videos grow longer, while standard text-overlap metrics often miss these qualities.

Latest papers

CardsList

71 papers7 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Adaptive Emotional Video Captioning via Affective Heterogeneous Graph Reasoning and Multi-task Joint Learning

    Jul 31, 2026Junbo Wang, Liangyu Fu, Yuke Li +2Video CaptioningEmotion Representation in Language Models

  2. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    Jul 30, 2026Tengfei Liu, Yang Shi, Yuran Wang +16Video Caption EvaluationVideo Captioning

  3. ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning

    Jul 23, 2026Debjyoti Das Adhikary, Aritra Hazra, Partha Pratim ChakrabartiVideo CaptioningVision-Language Grounding

  4. PercepCap: Video Captioner with Structured Spatio-Temporal Perception

    Jul 22, 2026Yifan Xu, Zihao Wang, Zhixiao Wang +6Video CaptioningTemporal Video Understanding

  5. VEGAS: Human-Aligned Video Caption Evaluation via Gaze

    Jul 9, 2026Shenghui Chen, Po-han Li, Ximeng Sun +5Video Caption EvaluationVideo Captioning

  6. Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

    Jul 6, 2026Mingqi Gao, Hongyuan Dong, Yifei Chen +6Reward ModelingVideo Captioning

  7. Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning

    Jul 3, 2026Wenzheng Zeng, Siyi Jiao, Chen Gao +2Temporal Video GroundingVideo Captioning

  8. Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning

    Jul 2, 2026Chen Zhao, Jiajun Ma, Qilong Huang +6Audio-Visual UnderstandingVideo Captioning

  9. MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

    Jun 28, 2026Weisong Liu, Haochen Wang, Kuan Gao +8Video CaptioningVideo-Language Models

  10. CineCap: Structured Reasoning with Spatio-Temporal Anchors for Cinematographic Video Captioning

    Jun 23, 2026Xinyu Mao, Yuhui Zeng, Xiaokun Liu +6Video CaptioningVideo-Language Models

  11. CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning

    Jun 8, 2026Penghui Yang, Long Xing, Xiaoyi Dong +10Image CaptioningMultimodal Pretraining

  12. OmniCap-IF: Benchmarking and Improving Instruction Following Abilities for Omni-Video Captioning

    Jun 7, 2026Jiahao Wang, An Ping, Yanghai Wang +13Video CaptioningMultimodal Generation

  13. Towards Accurate Emotion-Attributed Video Captioning via Fine-grained Emotion-Cause Pair Extraction

    Jun 7, 2026Weidong Chen, Cheng Ye, Zhendong Mao +3Relation ExtractionVideo Captioning

  14. PEEK: Picking Essential frames via Efficient Knowledge distillation

    May 29, 2026Killian Steunou, Anas Filali Razzouki, Khalil Guetari +2Efficient VLM InferenceVideo Captioning

  15. VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning

    May 27, 2026Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang +13Image CaptioningObject Hallucination in VLMs

  16. CodecCap: High-Fidelity Codec-Inspired Residual Modeling for Dense Video Captioning

    May 26, 2026Zihan Lin, Songhe Deng, Shuwei He +6Video Caption EvaluationVideo Captioning

  17. RoadTones: Tone Controllable Text Generation from Road Event Videos

    May 20, 2026Chirag Parikh, Siddhi Pravin Lipare, Ravi Kiran SarvadevabhatlaControllable Text GenerationVideo Captioning

  18. SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentary

    May 20, 2026Jingyi He, Yue Zhou, Long Bai +3Streaming Video UnderstandingSurgical Video Understanding

  19. DenseStep2M: A Scalable, Training-Free Pipeline for Dense Instructional Video Annotation

    Apr 29, 2026Mingji Ge, Qirui Chen, Zeqian Li +1Video UnderstandingLLM-Assisted Annotation

  20. DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding

    Apr 28, 2026Cennet Oguz, Yasser Hamidullah, Josef van Genabith +1Multimodal GroundingVideo Understanding

  21. One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition

    Apr 25, 2026Balaji Darur, Amanmeet Garg, Makarand TapaswiMultimodal GroundingVideo Understanding

  22. Building a Precise Video Language with Human-AI Oversight

    Apr 22, 2026Zhiqiu Lin, Chancharik Mitra, Siyuan Cen +13Human-in-the-Loop AnnotationVideo Captioning

  23. DiffVC: A Non-autoregressive Framework Based on Diffusion Model for Video Captioning

    Apr 9, 2026Junbo Wang, Liangyu Fu, Yuke Li +4Video CaptioningNon-Autoregressive Text Generation

  24. ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning

    Apr 9, 2026Daichi Yashima, Shuhei Kurita, Yusuke Oda +3Video CaptioningMultimodal Large Language Models

  25. A Benchmarking Methodology to Assess Open-Source Video Large Language Models in Automatic Captioning of News Videos

    Mar 29, 2026David Miranda Paredes, Jose M. Saavedra, Marcelo PizarroVideo Caption EvaluationVideo-Language Model Evaluation

  26. BFMD: A Full-Match Badminton Dense Dataset for Dense Shot Captioning

    Mar 26, 2026Ning Ding, Keisuke Fujii, Toru TamakiDense Video CaptioningSports Video Analysis