Video Captioning

Video captioning studies how to describe what happens in a clip, from a single concise sentence to dense, timestamped event descriptions and long-form narratives. Systems combine video-language models with temporal segmentation and grounding, frame selection, tracking or motion cues, and—in audiovisual settings—alignment between sounds, speakers, and visible events. Common test settings include short clips and instructional videos, using benchmarks such as MSR-VTT, VATEX, ActivityNet Captions, and YouCook2; newer evaluations also test factuality, temporal accuracy, entity consistency, and whether captions support downstream question answering. A central challenge is balancing completeness and fluent detail against hallucination, especially as videos grow longer, while standard text-overlap metrics often miss these qualities.