Knowledge-Refined Dual Context-Aware Network for Partially Relevant Video Retrieval

Junkai Yang; Qirui Wang; Yaoqing Jin; Shuai Ma; Minghan Xu; Shanmin Pang

cs.CVMar 25, 2026

Knowledge-Refined Dual Context-Aware Network for Partially Relevant Video Retrieval

Authors: Junkai Yang, Qirui Wang, Yaoqing Jin, Shuai Ma, Minghan Xu, Shanmin Pang

Organizations: School of Software Engineering, Xi’an Jiaotong University · Faculty of Computer Science, Electrical Engineering and Information Technology, Universität Stuttgart

Abstract

Retrieving partially relevant segments from untrimmed videos remains difficult due to two persistent challenges: the mismatch in information density between text and video segments, and limited attention mechanisms that overlook semantic focus and event correlations. We present KDC-Net, a Knowledge-Refined Dual Context-Aware Network that tackles these issues from both textual and visual perspectives. On the text side, a Hierarchical Semantic Aggregation module captures and adaptively fuses multi-scale phrase cues to enrich query semantics. On the video side, a Dynamic Temporal Attention mechanism employs relative positional encoding and adaptive temporal windows to highlight key events with local temporal coherence. Additionally, a dynamic CLIP-based distillation strategy, enhanced with temporal-continuity-aware refinement, ensures segment-aware and objective-aligned knowledge transfer. Experiments on PRVR benchmarks show that KDC-Net consistently outperforms state-of-the-art methods, especially under low moment-to-video ratios.

Figures & tables

Fig. 1: The Overview diagram illustrates the task objectives of PRVR, and also shows that the Knowledge Refined Distillation strategy we proposed can effectively optimize the distilled signals obtained from the teacher model.

Fig. 2: Illustration of KDC-Net. It employs a distillation framework, the student model comprises two independent branches with no parameter sharing.

Model	ActivityNet Captions					TVR					TVR: M/V Intervals
	R@1	R@5	R@10	R@100	SumR	R@1	R@5	R@10	R@100	SumR	(0, 0.2]	(0.2, 0.4]	(0.4, 1]
CE [ liu2019use ]	5.5	19.1	29.9	71.1	125.6	3.7	12.8	20.1	64.5	101.1	104.1	149.5	165.5
W2VV++ [ li2019w2vv++ ]	5.4	18.7	29.7	68.8	122.6	5.0	14.7	21.7	61.8	103.2	99.0	140.7	161.3
VSE++ [ faghri2017vse++ ]	4.9	17.7	28.2	67.1	117.9	7.5	19.9	27.7	66.0	121.1	119.1	109.8	133.5
DE [ dong2019dual ]	5.6	18.8	29.4	67.8	121.7	7.6	20.1	28.1	67.6	123.4	123.2	139.7	161.3
DE++ [ dong2021dual ]	5.3	18.4	29.2	68.0	121.0	8.8	21.9	30.2	67.4	128.3	124.7	131.4	149.0

TABLE I: Performance comparison on ActivityNet Captions, TVR and Performance on Different M/V Intervals of TVR (only existing open-source models). All metrics are reported as percentages(%). R@K denotes Recall at rank K.

Modules			Metrics
HSA	DTA	KRD	R@1	R@5	R@10	R@100	SumR
			14.4	34.1	45.0	84.3	177.8
$\surd$			14.2	34.5	45.7	84.6	179.0
$\surd$	$\surd$		15.5	36.2	46.6	85.3	183.6
$\surd$		$\surd$	14.9	35.4	46.1	85.4	181.8
	$\surd$		15.2	35.8	46.0	85.1	182.1

TABLE II: Ablation studies of each component on TVR

Fig. 3: Ablation studies: (a) KRD window size ablation; (b) DTA parameters ablation; (c) $\delta$ and $\lambda$ parameters ablation.

Branch		ActivityNet					TVR
Inheritance	Exploration	R@1	R@5	R@10	R@100	SumR	R@1	R@5	R@10	R@100	SumR
	$\surd$	6.2	20.5	33.5	73.9	134.1	12.7	32.1	42.4	83.9	171.1
$\surd$		7.8	24.5	36.5	76.8	145.6	11.8	30.2	41.0	82.6	165.6
$\surd$	$\surd$	8.1	25.3	38.0	77.5	148.8	15.4	36.3	47.6	85.6	184.9

TABLE III: Ablation studies on each branch

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 470 papers

Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval
Apr 4, 2026Jun Li, Xuhang Lou, Jinpeng Wang +4Cross-Modal Retrieval Information Retrieval
Harbin Institute of Technology, Shenzhen · Tsinghua Shenzhen International Graduate School, Tsinghua University · Peng Cheng Laboratory
TF-PRVR: Training-Free Partially Relevant Video Retrieval
Oct 6, 2026Giyeol Kim, Chanho EomInformation Retrieval Vision-Language Retrieval
Chung-Ang University
Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval
Sep 4, 2026Hyun Seok Seong, Woojin Jun, SuBeen Lee +1Information Retrieval VLM Adaptation
Sungkyunkwan University

Apr 4, 2026J/K move · Enter open · S save

Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval

Jun Li, Xuhang Lou, Jinpeng Wang, Yuting Wang+3

Harbin Institute of Technology, Shenzhen · Tsinghua Shenzhen International Graduate School, Tsinghua University · Peng Cheng Laboratory

Cross-Modal Retrieval Information Retrieval Video Representation Learning Representation Learning

Open paper AskPDF arXiv

Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.

Abstract

Information Retrieval

Explore similar work

Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval

TF-PRVR: Training-Free Partially Relevant Video Retrieval

Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval