cs.CVNov 11, 2026

VTG-RAG: Long-Video Reasoning via Query-Adaptive Coarse-to-Fine Sampling and Temporal Graph Retrieval

Authors: Yao Lu, Yunhao Wang, Yaxin Gao, Feifei Teng, Chenxiao Liu, Zhe Tang, Jiaheng Wei, Zhaowei Zhu, 

Organizations: Institute of Cyberspace Security, Zhejiang University of Technology. · Binjiang Institute of Artificial Intelligence, Zhejiang University of Technology. · Hong Kong University of Science and Technology (Guangzhou).

Abstract

Long-video reasoning requires a system to localize sparse evidence over extended temporal sequences, preserve temporal structure, and generate answers despite substantial multimodal distractors. Existing video retrieval-augmented generation methods typically build indexes from uniformly sampled video segments. In long-video settings, this design readily introduces irrelevant context and is unreliable for questions with explicit temporal constraints. We propose VTG-RAG, a framework for long-video question answering that combines query-adaptive coarse-to-fine sampling with temporal graph retrieval. First, a two-stage sampling strategy selects core segments with high mean query relevance or high intra-segment relevance variance. The framework then integrates subtitles, speech transcripts, visual descriptions, entities, and relations from these segments into a temporally ordered multimodal graph. Finally, it performs temporally filtered, two-pass retrieval and reasoning across frame, text, and graph evidence spaces. A lot of experiments have proven the effectiveness of the VTG-RAG. Specifically, we evaluate the framework on Video-MME and LongVideoBench. On LongVideoBench, the reported Qwen2.5-VL-32B and 72B results improve from 48.16% to 50.56% and from 52.13% to 53.32%, respectively, with 24 answer-input frames. These correspond to gains of 2.40 and 1.19 percentage points. The ablation experiments of VTG-RAG further support that the contributions of each component are very important.

Figures & tables

Explore similar work

CardsList
  1. Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG

    Aug 24, 2026Zhe Jin, Zhimin Lin, Bin Zheng +2Graph Retrieval-Augmented GenerationLong-Video QA

  2. EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding

    Oct 6, 2026Yuhao Qin, Junbo Wang, Yuke Li +1Multimodal RAGVideo-Language Models

  3. Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

    Jun 11, 2026Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim +5Multimodal RAGEgocentric Video Understanding