Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.
Figures & tables
Figure 1: (a) Limited contextual awareness causes spurious local spikes on a globally irrelevant “people boating” video, incorrectly outscoring the ground-truth “demonstrating the accordion through the camera” video which exhibits more overall relevance. (b) MIL considers only the closest pair, leading to sparse supervision where others remain undertrained. (c) DreamPRVR first imagines global registers through text-supervised diffusion, then concentrates on fine-grained learning, thereby realizing coarse-to-fine cross-modal alignment and jointly optimizing all video tokens to form a coherent embedding space.
Figure 2: Overview of Dream PRVR . ( a ) The query branch produces embedding q and samples q^ via TPS to supervise register generation. Video embeddings from a pre-trained model are first processed by a lightweight feature encoder and a probabilistic variational sampler (PVS) to produce the initial register rT , which is iteratively denoised via a truncated diffusion model to yield optimal registers r0 . r0 subsequently enhance frame- and clip-level representation learning, getting frame embeddings Vf and clip embeddings Vc . q learns a latent semantic structure through Ltssl and computes similarity scores Sf and Sc . ( b ) Textual Perturbation Sampler (TPS) models query uncertainty via controllable perturbations and samples q^ without trainable parameters. ( c ) Textual Semantic Structure Learning Ltssl employs Ldiv to diversify queries and Lqsp to align queries from the same video while contrasting across videos. ( d ) The asymmetric attention mask defines two cross-attention patterns, enabling full interactions for video tokens while constraining registers to video-only attention.
Model
ActivityNet Captions
Charades-STA
TVR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
Text-to-Video Retrieval (T2VR) Models
RIVRL [ 13 ]
5.2
18.0
28.2
66.4
117.8
1.6
5.6
9.4
37.7
54.3
9.4
23.4
32.2
70.6
135.6
DE++ [ 12 ]
5.3
18.4
29.2
68.0
121.0
1.7
5.6
9.6
37.1
54.1
8.8
21.9
30.2
67.4
128.3
CLIP4Clip [ 42 ]
5.9
19.3
30.4
71.6
127.3
1.8
6.5
10.9
44.2
63.4
9.9
24.3
34.3
72.5
141.0
Cap4Video [ 74 ]
6.3
20.4
30.9
72.6
130.2
1.9
6.7
11.3
45.0
65.0
10.3
26.4
36.8
74.0
147.5
Table 1: Retrieval performance comparison across three standard benchmarks. Models are ranked in ascending order of SumR scores on ActivityNet Captions. State-of-the-art performance is highlighted in bold , while “-” denotes unavailable results.
Model
Train time/epoch (ms)
Model parameters (M)
Inference time (ms)
Retrieval time (ms)
SumR
GMMFormer
26887
12.85
2876
3238
72.9
HLFormer
31463
28.43
3816
3655
78.7
GMMFormerV2
38004
30.79
3843
3688
78.2
Dream PRVR
33609
36.14
4001
3686
80.0
Table 2: Training and Inference Efficiency. Inference time denotes feature extraction for 1334 videos in the evaluation sets, while retrieval time accounts for the total time of query encoding, similarity computation and ranking. Dataset: Charades-STA.
ID
Model
ActivityNet Captions
Charades-STA
TVR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
(0)
Dream PRVR (full)
8.7
27.5
40.3
79.5
156.1
2.6
8.7
14.5
54.2
80.0
17.4
39.0
50.4
86.2
193.1
Efficacy of Register Generation Strategy
(1)
w/o registers
8.5
26.6
39.5
78.8
153.4
2.4
7.9
13.6
52.9
76.8
15.7
36.6
48.7
86.0
187.0
(2)
w/ AP
8.5
26.2
39.3
77.9
151.9
2.6
8.6
14.4
52.6
78.1
17.0
38.3
49.6
86.4
191.4
(3)
w/o DRE
8.2
25.7
38.7
78.0
150.6
2.4
8.5
14.8
52.6
78.3
16.8
38.2
49.6
86.3
190.8
Table 3: Ablation Study of Dream PRVR . The best scores are marked in bold .
Figure 3: The influence of the number of registers and the number of diffusion timesteps, with default settings marked in bold.
Figure 4: The t-SNE visualization of the learned textual space. Data points of the same color denote queries from the same video.
Figure 5: A qualitative case study of retrieval results. The same videos in Fig. 1 are selected for a better comparison.
Figure 6: The t-SNE visualization of the iterative generation process of global registers. T = 0 indicates the initialization while T = 10 represents the final results. 16 registers are trained for better clarity and data points of the same color correspond to the same video.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: (a) Illustration of the proposed Diffusion Register Estimator Block (DRE). Embed(t) , qT and c denote the temporal embedding, the latent embedding corrupted by t -step noise and the guided condition, respectively. (b) Condition generator for DRE.
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos containing moments relevant to a given text query. Despite recent progress, existing PRVR methods suffer from two key limitations: a fixed video decomposition scheme that causes semantic dilution, and source-domain overfitting induced by task-specific training. In this paper, we propose TF-PRVR, the first training-free framework for PRVR. TF-PRVR leverages frozen vision-language features to construct video-specific hierarchical representations. It derives temporal semantic signals from frame-level features and applies frequency-based multi-scale analysis to identify adaptive temporal boundaries, producing hierarchical segments with coherent event-level semantics. Built on these segments, TF-PRVR constructs a unified multi-scale graph and propagates query relevance across temporally and semantically related nodes. A moment-aware scoring strategy then aggregates temporally aligned relevance across scales, emphasizing consistently supported moments while suppressing isolated false responses. Without task-specific training, TF-PRVR preserves the general-purpose alignment capability of pre-trained vision-language models and avoids dataset-specific overfitting. Extensive experiments demonstrate consistent performance across datasets with diverse visual and temporal characteristics, suggesting a practical direction for training-free PRVR.
Partially Relevant Video Retrieval (PRVR) retrieves untrimmed videos when queries describe only short moments. Although recent methods improve local representations, uncertainty modeling, and global context, final ranking often still trusts the strongest local response; a coincidentally similar fragment can therefore produce an unsupported peak. We identify this failure as the query-agnostic concentration bottleneck and propose TRACE, a score-level evidence verification operator for PRVR. Given a query and global video registers, TRACE activates query-relevant registers, routes their support to frame-level evidence, and smoothly marginalizes alternative query-to-register-to-frame paths before localized temporal selection. Unlike representation-level feature fusion, TRACE uses this evidence only as a query-conditioned residual calibration of the original local score. On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks and improves the DreamPRVR backbone by 1.2, 1.1, and 1.5 points, respectively. Ablation, routing-corruption, hard-negative, and cross-backbone transfer analyses support the interpretation that the gains arise from query-conditioned evidence verification rather than a generic score offset.
Shuaiqi Cheng, Siyu You, Yanbi Wu +3
1The Hong Kong University of Science and Technology (Guangzhou) · University of Electronic Science and Technology of China · 3The Hong Kong University of Science and Technology
Retrieving partially relevant segments from untrimmed videos remains difficult due to two persistent challenges: the mismatch in information density between text and video segments, and limited attention mechanisms that overlook semantic focus and event correlations. We present KDC-Net, a Knowledge-Refined Dual Context-Aware Network that tackles these issues from both textual and visual perspectives. On the text side, a Hierarchical Semantic Aggregation module captures and adaptively fuses multi-scale phrase cues to enrich query semantics. On the video side, a Dynamic Temporal Attention mechanism employs relative positional encoding and adaptive temporal windows to highlight key events with local temporal coherence. Additionally, a dynamic CLIP-based distillation strategy, enhanced with temporal-continuity-aware refinement, ensures segment-aware and objective-aligned knowledge transfer. Experiments on PRVR benchmarks show that KDC-Net consistently outperforms state-of-the-art methods, especially under low moment-to-video ratios.
Junkai Yang, Qirui Wang, Yaoqing Jin +3
School of Software Engineering, Xi’an Jiaotong University · Faculty of Computer Science, Electrical Engineering and Information Technology, Universität Stuttgart