Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG

Zhe Jin; Zhimin Lin; Bin Zheng; Junhua Fang; Huihua Yang

cs.CVAug 24, 2026

Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG

Authors: Zhe Jin, Zhimin Lin, Bin Zheng, Junhua Fang, Huihua Yang

Organizations: Beijing University of Posts and Telecommunications · Soochow University

Abstract

Graph-based retrieval-augmented generation (RAG) provides a scalable paradigm for long-video understanding, but existing systems typically inherit a fixed temporal granularity from video segmentation when constructing their retrieval index. We argue that this design unnecessarily couples indexing granularity with evidence granularity: coarse representations can often suffice for locating relevant temporal regions, while fine-grained evidence remains important for downstream reasoning. We propose Density-Aware Graph Construction (DAGC), a training-free approach that decouples a query-independent coarse retrieval index from the original fine-grained evidence space. DAGC constructs a compact, density-adaptive graph index by merging visually redundant neighboring chunks, while preserving mappings to the original temporal units. Retrieved coarse regions are subsequently expanded back to the original chunk granularity for fine-grained evidence refinement and answer generation. Experiments on MLVU, VideoMME, and LongVideoBench show that DAGC retains only about 40--50% of the original graph nodes and achieves $1.3$ -- $1.7\times$ end-to-end wall-clock acceleration while preserving approximately 99% of the original QA performance. The gains transfer across different LVLM backbones and video RAG pipelines, suggesting that long-video RAG need not maintain the same temporal granularity for indexing and evidence reasoning.

Figures & tables

Figure 1: Decoupling indexing and evidence granularity in long-video RAG. Existing designs either maintain fine temporal resolution throughout the pipeline or coarsen both indexing and evidence. DAGC instead uses a density-adaptive coarse index for retrieval while preserving fine-grained evidence for downstream reasoning.

Figure 2: Overview of Density-Aware Graph Construction (DAGC). DAGC separates the temporal granularity used for graph indexing from that used for downstream evidence reasoning. Visually redundant neighboring chunks are merged into density-adaptive coarse nodes for efficient indexing, while each node preserves the mapping $\mathcal{I}_{m}$ to its constituent original chunks. Retrieval first operates on the compact graph for coarse localization and then recovers the original chunks for fine-grained refinement and answer generation.

Model	MLVU			VideoMME			LVB			Avg. Accuracy	Performance Retention
Model	Accuracy	Retained	Wall Speedup	Accuracy	Retained	Wall Speedup	Accuracy	Retained	Wall Speedup	Avg. Accuracy	Performance Retention
Qwen2.5-VL-7B	69.0	–	–	70.1	–	–	59.4	–	–	66.2	–
+ Vgent	73.3	100%	1.0 $\times$	73.3	100%	1.0 $\times$	63.3	100%	1.0 $\times$	70.0	100%
+ DAGC	73.4	45%	1.3 $\times$	71.1	45%	1.4 $\times$	63.1	47%	1.6 $\times$	69.2	99%
Qwen2.5-VL-3B	65.0	–	–	67.0	–	–	56.3	–	–	62.8	–
+ Vgent	70.0	100%	1.0 $\times$	69.0	100%	1.0 $\times$	60.0	100%	1.0 $\times$	66.3	100%

Table 1: Main results on three long-video benchmarks. Accuracy denotes multiple-choice accuracy, Retained denotes the retained graph-node ratio relative to Vgent, and Wall Speedup denotes end-to-end wall-clock acceleration relative to the corresponding Vgent baseline. Performance Retention is computed from the average accuracy across the three benchmarks.

Model	Offline Time	Online Time	First-query Time
Qwen2.5-VL-7B	–	–	3.12
Qwen2.5-VL-7B + Vgent	24.14	4.12	28.26
Qwen2.5-VL-7B + DAGC	10.73	4.24	14.97

Table 2: Normalized runtime comparison using Qwen2.5-VL-7B. Offline Time denotes query-independent graph construction, Online Time denotes query-dependent inference after graph construction, and First-query Time includes both stages. All values are reported in seconds per minute of video.

Dataset	Vgent Accuracy	DAGC Accuracy	$\bm{\Delta}$ Accuracy	Retained	Wall Speedup
LVB	63.07	63.05	$-0.02$	47%	1.3 $\times$
MLVU	73.46	73.04	$-0.42$	47%	1.4 $\times$
VideoMME	68.18	65.63	$-2.55$	47%	1.4 $\times$

Table 3: Cross-model-family evaluation using InternVL3.5-8B. Retained denotes the retained graph-node ratio relative to Vgent, and Wall Speedup is measured relative to the corresponding InternVL3.5-8B + Vgent baseline.

Dataset	Common N	Baseline Accuracy	DAGC Accuracy	$\bm{\Delta}$ Accuracy	Retained	Offline Speedup	Wall Speedup
LVB	1,296	52.70%	54.17%	+1.47 pp	48.34%	1.377 $\times$	1.309 $\times$
MLVU	2,174	64.49%	62.88%	$-1.61$ pp	40.39%	1.695 $\times$	1.328 $\times$
VideoMME	2,683	65.97%	66.72%	+0.75 pp	45.10%	1.455 $\times$	1.308 $\times$

Table 4: Cross-framework evaluation after integrating DAGC into VideoRAG. Common N denotes the number of examples successfully evaluated by both the baseline and DAGC. Retained denotes the retained temporal indexing-unit ratio relative to the uncompressed VideoRAG baseline. Offline Speedup measures query-independent index construction, while Wall Speedup measures complete experimental wall-clock acceleration.

Variant	Retained	Accuracy	Wall Speedup
Qwen2.5-VL-7B	–	59.4	–
+ Vgent	100%	63.3	1.0 $\times$
+ DAGC w/o SN	100%	62.6	1.0 $\times$
+ DAGC w/o RR	47%	60.6	1.6 $\times$
+ DAGC w/o TE	47%	62.2	1.6 $\times$
+ DAGC	47%	63.1	1.6 $\times$

Table 5: Ablation of coarse indexing and fine-grained evidence recovery on LongVideoBench using Qwen2.5-VL-7B. SN denotes super-node compression, RR denotes recovery and reranking of original chunks, and TE denotes temporal expansion. Detailed category-level results are reported in Appendix A.6 .

Method	Accuracy	Retained
Random Merge	61.53	47%
Uniform Merge	62.28	47%
DAGC	63.07	47%

Table 6: Comparison of compression strategies on LongVideoBench using Qwen2.5-VL-7B under the same retained-node budget.

Task	DAGC	+ Event Boundary	$\bm{\Delta}$
Order	70.66	72.97	+2.32
Needle	82.25	81.69	-0.56
Count	60.68	57.77	-2.91
Weighted Overall	73.17	72.93	-0.24

Table 7: Effect of EfficientGEBD event-boundary constraints on the complete Order, Needle, and Count subsets (820 questions).

\Delta

denotes percentage-point change relative to DAGC.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Count	Ego	Needle	Order	PlotQA	Topic	Anomaly	Overall
Qwen2.5-VL-7B	42.3	60.0	79.4	66.7	75.1	86.4	73.0	69.0
Qwen2.5-VL-7B + Vgent	59.6	61.4	81.1	73.4	76.1	87.1	74.5	73.3
Qwen2.5-VL-7B + DAGC	60.7	62.0	82.3	70.7	76.6	87.1	74.0	73.4
Qwen2.5-VL-3B	32.3	52.9	78.2	56.2	71.5	88.1	76.0	65.0
Qwen2.5-VL-3B + Vgent	53.3	58.0	80.0	62.5	71.9	89.0	75.5	70.0
Qwen2.5-VL-3B + DAGC	52.4	58.2	78.6	64.0	72.1	88.3	75.5	69.9

Appendix

Table 8: Detailed results on MLVU across seven tasks. Count, Ego, Needle, Order, PlotQA, Topic, and Anomaly are the seven evaluated tasks. Overall denotes the aggregate accuracy across all seven tasks.

Model	Short	Medium	Long	Overall Accuracy	Wall Speedup
Qwen2.5-VL-7B + Vgent	78.76	72.63	68.33	73.25	1.0 $\times$
Qwen2.5-VL-7B + DAGC	78.67	72.41	62.22	71.11	1.4 $\times$
Qwen2.5-VL-3B + Vgent	75.33	68.55	63.66	69.00	1.0 $\times$
Qwen2.5-VL-3B + DAGC	74.23	65.22	59.11	66.19	1.3 $\times$
Qwen2-VL-7B + Vgent	76.43	69.55	63.22	69.74	1.0 $\times$
Qwen2-VL-7B + DAGC	75.55	66.55	59.77	67.30	1.4 $\times$

Appendix

Table 9: Detailed results on VideoMME across different video durations. Short, Medium, and Long denote the three duration subsets. Overall denotes the aggregate accuracy over all duration subsets. Wall Speedup denotes the relative wall-clock speedup over the corresponding Vgent baseline.

Model	E2O	SSS	T2E	S2O	SAA	TAA	S2A	SOS	T2A
Qwen2.5-VL-7B + Vgent	75.4	46.4	72.3	66.7	61.1	59.8	72.1	65.4	70.4
Qwen2.5-VL-7B + DAGC	75.4	44.8	69.2	68.1	58.3	58.5	70.5	65.4	67.9
Qwen2.5-VL-3B + Vgent	64.6	47.4	64.6	62.5	52.8	52.4	62.5	65.4	63.0
Qwen2.5-VL-3B + DAGC	64.6	43.8	66.2	63.9	54.2	53.7	62.5	64.2	64.2
Qwen2-VL-7B + Vgent	70.8	46.4	66.2	56.9	56.9	52.4	71.6	61.7	56.8
Qwen2-VL-7B + DAGC	70.8	47.4	66.2	55.6	52.8	53.7	72.7	63.0	59.3

Appendix

Table 10: Detailed category-level results on LongVideoBench, Part I. The table reports the first group of question-category accuracies.

Model	S2E	T3O	T2O	O3O	O2E	T3E	TOS	E3E	Overall Accuracy	Wall Speedup	Retained
Qwen2.5-VL-7B + Vgent	76.3	59.5	64.1	60.6	69.3	49.3	40.0	68.1	63.33	1.0 $\times$	–
Qwen2.5-VL-7B + DAGC	74.2	56.8	69.2	68.2	68.2	49.3	42.7	67.0	63.07	1.6 $\times$	47%
Qwen2.5-VL-3B + Vgent	77.4	62.2	55.1	54.5	67.8	54.8	44.0	67.0	60.00	1.0 $\times$	–
Qwen2.5-VL-3B + DAGC	75.3	59.5	59.0	62.1	70.5	52.1	42.7	68.1	60.6	1.3 $\times$	47%
Qwen2-VL-7B + Vgent	68.8	66.2	57.7	62.1	62.5	49.3	32.0	62.8	58.88	1.0 $\times$	–
Qwen2-VL-7B + DAGC	71.0	58.1	57.7	62.1	62.5	49.3	33.3	59.6	58.55	1.7 $\times$	47%

Appendix

Table 11: Detailed category-level results on LongVideoBench, Part II. The table reports the question-category accuracies and includes the overall accuracy, speedup, and retained-node ratio.

Method	Accuracy	Retained
Random Merge	61.53	47%
Uniform Merge	62.28	47%
DAGC	63.07	47%

Appendix

Table 12: Comparison of different compression strategies on LongVideoBench using Qwen2.5-VL-7B under the same retained-node budget.

Method	Accuracy	Graph Construction Time (s)	Wall Time (s)
DAGC	82.25 (292/355)	30,741	13,780
Hybrid Boundary DAGC	81.41 (289/355)	61,477	24,061

Appendix

Table 13: Comparison of DAGC with a hybrid boundary-aware variant on the complete MLVU Needle subset. The hybrid variant combines appearance, motion, and subtitle-semantic boundary signals. The accuracy difference is not statistically significant under a paired McNemar test (

p=0.25

). Graph Construction Time reports the cumulative compute time summed across all GPUs, whereas Wall Time denotes the actual elapsed wall-clock time.

Variant	E2O	SSS	T2E	S2O	SAA	TAA	S2A	SOS	T2A
Qwen2.5-VL-7B	70.8	46.4	66.2	55.6	62.8	53.7	72.7	63.0	59.3
+ Vgent	75.4	46.4	72.3	66.7	61.1	59.8	72.1	65.4	70.4
+ DAGC w/o SN	72.3	43.8	70.8	65.3	56.3	57.3	70.1	65.0	67.9
+ DAGC w/o RR	72.3	42.7	72.3	63.9	58.3	57.3	70.1	64.2	69.1
+ DAGC w/o TE	75.4	44.8	69.2	66.7	58.3	58.5	70.5	65.4	67.9
+ DAGC	75.4	44.8	69.2	68.1	58.3	58.5	70.5	65.4	67.9

Appendix

Table 14: Component ablation results on LongVideoBench, Part I. The table reports the first group of question-category accuracies. SN denotes super-node compression, RR denotes original-chunk recovery and reranking, and TE denotes temporal expansion.

Variant	S2E	T3O	T2O	O3O	O2E	T3E	TOS	E3E	Overall Accuracy	Wall Speedup	Retained
Qwen2.5-VL-7B	71.0	58.1	57.7	62.1	62.5	49.3	33.3	59.6	59.4	1.0 $\times$	–
+ Vgent	76.3	59.5	64.1	60.6	69.3	49.3	40.0	68.1	63.3	1.0 $\times$	100%
+ DAGC w/o SN	75.3	58.1	67.9	66.7	69.3	50.7	40.0	68.1	62.6	1.0 $\times$	100%
+ DAGC w/o RR	74.2	58.1	65.4	62.1	65.5	49.3	38.7	67.0	60.6	1.6 $\times$	47%
+ DAGC w/o TE	73.1	55.4	67.9	65.2	67.0	48.6	41.3	65.9	62.2	1.6 $\times$	47%
+ DAGC	74.2	56.8	69.2	68.2	68.2	49.3	42.7	67.0	63.1	1.6 $\times$	47%

Appendix

Table 15: Component ablation results on LongVideoBench, Part II. The table reports the remaining question-category accuracies together with overall accuracy, relative wall-clock speedup, and retained-node ratio. The full DAGC variant achieves the best overall accuracy among the compressed variants while preserving the efficiency advantage over Vgent.

Setting	E2O	SSS	T2E	S2O	SAA	TAA	S2A	SOS	T2A
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =12 (Default)	75.4	44.8	69.2	68.1	58.3	58.5	70.5	65.4	67.9
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =5	71.9	45.4	70.8	65.3	56.9	58.5	69.0	64.2	69.1
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =20	73.8	43.6	72.3	65.3	56.9	57.3	69.3	63.0	67.9
$\tau_{v}$ =0.95, $W$ =1, top- $k_{s}$ =12	72.3	43.8	70.8	65.3	56.3	57.3	70.1	65.0	67.9
$\tau_{v}$ =0.90, $W$ =3, top- $k_{s}$ =12	73.8	48.5	69.2	66.7	58.3	57.3	70.5	63.0	67.9
$\tau_{v}$ =0.95, $W$ =7, top- $k_{s}$ =12	72.3	45.4	70.8	65.3	58.3	57.3	70.5	64.2	70.4

Appendix

Table 16: Parameter sensitivity results on LongVideoBench, Part I. The table reports the first group of question-category accuracies under different adaptive graph compression settings.

Setting	S2E	T3O	T2O	O3O	O2E	T3E	TOS	E3E	Overall Accuracy	Wall Speedup	Retained
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =12 (Default)	74.2	56.8	69.2	68.2	68.2	49.3	42.7	67.0	63.1	1.6 $\times$	47%
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =5	74.2	58.1	65.4	66.7	68.2	49.3	45.3	68.1	62.7	1.6 $\times$	47%
$\tau_{v}$ =0.95, $W$ =3, top- $k_{s}$ =20	74.2	56.8	67.9	68.2	69.0	49.3	42.7	69.1	62.7	1.6 $\times$	47%
$\tau_{v}$ =0.95, $W$ =1, top- $k_{s}$ =12	75.3	58.1	67.9	66.7	69.3	50.7	40.0	68.1	62.6	1.0 $\times$	100%
$\tau_{v}$ =0.90, $W$ =3, top- $k_{s}$ =12	74.2	58.1	64.9	66.7	69.3	47.9	41.3	69.1	62.8	1.7 $\times$	39%
$\tau_{v}$ =0.95, $W$ =7, top- $k_{s}$ =12	74.2	58.1	65.4	66.7	68.2	49.3	42.7	67.0	62.5	1.7 $\times$	34%

Appendix

Table 17: Parameter sensitivity results on LongVideoBench, Part II. The table reports the remaining question-category accuracies together with overall accuracy, relative wall-clock speedup, and retained-node ratio. The default setting achieves the best overall accuracy while maintaining a clear efficiency gain.

Panel of Figure 3 — Figure 3: Effect of index-coarsening parameters on LongVideoBench. Bars show retained graph nodes, and lines show QA accuracy. (a) Higher $\tau_{v}$ yields more conservative merging and retains more nodes. (b) Larger super-node spans enable stronger coarsening but may reduce accuracy.

Graph Retrieval-Augmented Generation

Graph Retrieval-Augmented Generation connects language models to knowledge graphs to retrieve relational evidence for grounded, multi-hop answers.

See all 232 papers

Rethinking RAG in Long Videos: What to Retrieve and How to Use It?
Jun 11, 2026Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim +5Multimodal RAG Egocentric Video Understanding
KAIST, YH, Seoul, Republic of Korea · Qualcomm AI Research, Qualcomm Korea, YH, Seoul, Republic of Korea
Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios
May 7, 2026Peizheng Yan, Yu Zhao, Liang Xie +3Retrieval-Augmented Generation Streaming Video Understanding
Tianjin Key Lab of Intelligent Unmanned Swarm Tech & System Tianjin University Tianjin, China · Institute of Computing and Intelligence Harbin Institute of Technology (Shenzhen) Shenzhen, China · Tianjin Artificial Intelligence Innovation Center Tianjin, China +2
VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Apr 7, 2026Honghao Fu, Miao Xu, Yiwei Wang +3Multimodal RAG Long-Video Understanding
University of Queensland · University of California, Merced · Institute of Automation, CAS +1

Jun 11, 2026J/K move · Enter open · S save

Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim, Jihwan Bang+4

KAIST, YH, Seoul, Republic of Korea · Qualcomm AI Research, Qualcomm Korea, YH, Seoul, Republic of Korea

Multimodal RAG Egocentric Video Understanding RAG Evaluation Long-Video QA Video Retrieval Retrieval-Augmented Video Generation

Open paper AskPDF arXiv

Retrieval-augmented generation is moving beyond text into long, egocentric video, where systems must select query-relevant chunks across multiple modalities and temporal granularities. Yet progress in VideoRAG is limited by two gaps: existing benchmarks allow queries to be answered without the video, obscuring retrieval errors, and prior methods apply a single modality-granularity configuration per query, ignoring chunk-level variability. We address both by introducing V-RAGBench, a benchmark of

\langle

query, evidence chunk, answer

\rangle

triplets that enables faithful, decoupled evaluation of retrieval and generation, and CARVE, a simple method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk. Each chunk then enters the generator under its winning configuration selected during retrieval, yielding an interleaved evidence form where the chunk-level decision propagates across both stages. CARVE outperforms eight recent VideoRAG baselines, with the chunks supplied to the generator interleaving multiple configurations rather than sharing a single one, a behavior unattainable by query-level methods.

Abstract

Graph Retrieval-Augmented Generation

Explore similar work

Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios

VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG