Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos that contain moments relevant to a text query. Since the target moment occupies only a portion of the video, PRVR requires retrieval based on fine-grained understanding beyond coarse video-level matching. However, existing methods often rely on frozen CLIP frame features, which lack temporal understanding. Even with recent progress in parameter-efficient CLIP adaptation, video-level predictions can still be supported by imprecise frame-level evidence. In this paper, we propose an Intrinsic Temporal Adaptation (ITA) framework for PRVR. First, our Backbone-Internal Temporal Adaptation allows the last few visual transformer layers to attend over groups of neighboring frames. This provides temporally aware frame embeddings while keeping CLIP frozen and training only adaptation parameters. Second, we introduce Affinity-Weighted Gradient Propagation to address the weakly supervised nature of PRVR, softly aggregating top-k frames based on text-frame affinities and propagating learning signals to multiple query-relevant frames. Our method achieves state-of-the-art performance on PRVR benchmarks, demonstrates robust cross-dataset transfer, and retrieves substantially more accurate frame-level evidence within ground-truth query-relevant moments. Our code is available at github.com/hynnsk/ITA.
Figures & tables
Figure 1: Comparison between ProPy and Ours. (a) Visualization of cosine similarities between each frame and its preceding frame. Rows and columns are video and frame indices, respectively. ProPy shows vertical stripe patterns across unrelated videos, while ours produces more video-specific transition patterns. (b) Strict frame-level top- k retrieval on ActivityNet Captions. Unlike video-level retrieval, we pool frames from all videos and test whether all top- k frames come from the ground-truth video. This evaluates whether video-level retrieval is supported by coherent frame-level evidence. ProPy rapidly degrades as k increases, whereas ours remains more robust. (c) Comparison of trainable parameters and inference-time GFLOPs of the video encoder.
Figure 2: Illustration of the proposed framework. Given sampled video frames, the CLIP visual encoder produces frame embeddings through Backbone-Internal Temporal Adaptation, where the last few visual transformer layers perform attention over grouped neighboring frames. The resulting frame embeddings are further refined by a lightweight global class-token attention layer. To address the weakly supervised nature of PRVR, we introduce Affinity-Weighted Gradient Propagation, which selects top- k frames according to text-frame affinities and propagates contrastive learning signals to multiple query-relevant frames. The whole framework is trained in a parameter-efficient manner by freezing the CLIP backbone and applying LoRA to the query-key-value projection layers.
Method
TVR
ActivityNet Captions
Charades-STA
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
MS-SL ( Dong et al., 2022 )
13.5
32.1
43.4
83.4
172.4
7.1
22.5
34.7
75.8
140.1
1.8
7.1
11.8
47.7
68.4
MS-SL++ ( Chen et al., 2025b )
13.6
33.1
44.2
83.5
174.5
7.0
23.1
35.2
75.8
141.1
1.8
7.6
12.0
48.4
69.7
PEAN ( Jiang et al., 2023 )
13.5
32.8
44.1
83.9
174.2
7.4
23.0
35.5
75.9
141.8
2.7
8.1
13.5
50.3
74.7
GMMFormer ( Wang et al., 2024b )
13.9
33.3
44.5
84.9
176.6
8.3
24.9
36.7
76.1
146.0
2.1
7.8
12.5
50.6
72.9
DL-DKD ( Dong et al., 2025 )
14.4
34.9
45.8
84.9
179.9
8.0
25.0
37.5
77.1
147.6
-
-
-
-
-
Table 1: Performance comparison on TVR, ActivityNet Captions, and Charades-STA. Rows highlighted in gray indicate the performance of methods leveraging ResNet152 + I3D + RoBERTa features.
Method
R@1
R@5
R@10
R@100
SumR
GMMFormer
18.2
43.7
56.7
92.5
211.1
MS-SL
20.4
46.7
60.7
94.6
222.5
Proto
22.6
48.8
61.3
93.9
226.6
MS-SL
21.2
50.0
61.0
94.1
226.4
GMMFormer
18.4
42.3
56.2
93.4
210.3
GMMFormer-V2
23.1
50.7
63.9
94.8
232.5
Table 2: Performance on QVHighlights val split. Rows highlighted in gray are results with CLIP-B/16 features.
Backbone
ProPy
ITA
CLIP-B/16
198.0
209.0
CLIP4Clip-B/32
183.6
196.1
InternVideo-MM-B/16
190.7
201.7
Table 3: Generalization across diverse backbones. We report the average SumR over four PRVR benchmarks.
PEA
GTA
TB
AW
TVR
ANet
Charades
QV
Avg.
✓
217.7
195.9
78.5
271.2
190.8
✓
✓
221.1
199.0
82.0
277.6
194.9
✓
✓
✓
223.2
199.6
83.9
277.9
196.2
✓
✓
219.6
199.5
81.7
280.8
195.4
✓
✓
✓
✓
224.7
201.4
87.4
281.9
198.9
Table 4: Ablation study on each component, reporting SumR. PEA, GTA, TB, and AW denote Parameter-Efficient Adaptation, Grouped Temporal Attention, Temporal Bias, and Affinity-Weighting, respectively.
Figure 3: Hyperparameter ablation on the number of selected frames k and the affinity temperature τ . For k , we fix τ=0.05 and vary k from top-1 to all-frame aggregation. For τ , we fix k=4 and vary τ from 0.01 to 1.0. We report average SumR over four datasets.
Figure 4: Ablation study on the group size for grouped temporal attention. GFLOPs denotes the video encoding cost. We report the average SumR over the four datasets.
Figure 5: Comparison between top- k and threshold-based frame selection. The orange dashed lines indicate our final model, while the blue curves show threshold-based selection using different fixed affinity thresholds (x-axis). The y-axis denotes SumR.
Method
QV
TVR
ANet
Charades
MS-SL
226.4
41.2
96.5
30.6
GMMFormer
210.3
38.1
91.1
28.7
AMDNet
221.5
37.2
86.7
30.4
HLFormer
220.5
39.2
92.2
29.6
MSC
220.7
40.5
94.2
27.8
ProPy
275.5
102.5
150.3
46.0
Table 5: Cross-dataset generalization results. All methods are trained only on QVHighlights and evaluated on the other datasets. QV denotes the source-domain evaluation and we report SumR for all results.
Figure 6: Qualitative comparison of frame-level retrieval. Given a text query, we first perform video-level retrieval and consider examples where both ProPy and our method retrieve the correct video. We then rank individual frames and visualize the top-5 retrieved frames for each method.
Figure 7: Visualization of frame-transition score for frames (4th-22nd frames). Orange and blue curves correspond to ProPy and Ours, respectively.
Dataset
Method
R@1
R@5
R@10
TVR
ProPy
6.95
16.26
21.63
ITA
11.90
23.95
31.37
ANet
ProPy
5.61
13.03
18.31
ITA
9.86
18.99
24.64
Charades
ProPy
0.67
2.80
4.14
ITA
2.12
4.70
6.75
Table 6: Frame-level retrieval using GT moments.
Method
Trainable
GFLOPs
GFLOPs
Avg.
Params (M) ↓
(video) ↓
(online) ↓
SumR
MS-SL
2.70
282.8
10.285
166.8
GMM-V2
30.11
284.7
6.475
172.0
MSC
30.24
284.0
6.289
169.8
ProPy
7.98
370.0
6.107
187.7
ITA
1.57
283.7
5.490
198.9
Table 7: Efficiency comparison with existing PRVR methods. GFLOPs (video) denotes the offline cost of encoding a single video. GFLOPs (online) denotes the online retrieval cost when visual embeddings of 10,000 database videos are pre-extracted and stored.
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos containing moments relevant to a given text query. Despite recent progress, existing PRVR methods suffer from two key limitations: a fixed video decomposition scheme that causes semantic dilution, and source-domain overfitting induced by task-specific training. In this paper, we propose TF-PRVR, the first training-free framework for PRVR. TF-PRVR leverages frozen vision-language features to construct video-specific hierarchical representations. It derives temporal semantic signals from frame-level features and applies frequency-based multi-scale analysis to identify adaptive temporal boundaries, producing hierarchical segments with coherent event-level semantics. Built on these segments, TF-PRVR constructs a unified multi-scale graph and propagates query relevance across temporally and semantically related nodes. A moment-aware scoring strategy then aggregates temporally aligned relevance across scales, emphasizing consistently supported moments while suppressing isolated false responses. Without task-specific training, TF-PRVR preserves the general-purpose alignment capability of pre-trained vision-language models and avoids dataset-specific overfitting. Extensive experiments demonstrate consistent performance across datasets with diverse visual and temporal characteristics, suggesting a practical direction for training-free PRVR.
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.
Jun Li, Xuhang Lou, Jinpeng Wang +4
Harbin Institute of Technology, Shenzhen · Tsinghua Shenzhen International Graduate School, Tsinghua University · Peng Cheng Laboratory
Retrieving partially relevant segments from untrimmed videos remains difficult due to two persistent challenges: the mismatch in information density between text and video segments, and limited attention mechanisms that overlook semantic focus and event correlations. We present KDC-Net, a Knowledge-Refined Dual Context-Aware Network that tackles these issues from both textual and visual perspectives. On the text side, a Hierarchical Semantic Aggregation module captures and adaptively fuses multi-scale phrase cues to enrich query semantics. On the video side, a Dynamic Temporal Attention mechanism employs relative positional encoding and adaptive temporal windows to highlight key events with local temporal coherence. Additionally, a dynamic CLIP-based distillation strategy, enhanced with temporal-continuity-aware refinement, ensures segment-aware and objective-aligned knowledge transfer. Experiments on PRVR benchmarks show that KDC-Net consistently outperforms state-of-the-art methods, especially under low moment-to-video ratios.
Junkai Yang, Qirui Wang, Yaoqing Jin +3
School of Software Engineering, Xi’an Jiaotong University · Faculty of Computer Science, Electrical Engineering and Information Technology, Universität Stuttgart