Partially Relevant Video Retrieval (PRVR) seeks to retrieve untrim-med videos containing a moment that matches a text query, without temporal annotations. The relevant moment may last only seconds within a video spanning several minutes, creating an extremely low signal-to-noise ratio that makes PRVR more challenging than standard full-video retrieval. This task presents two intertwined challenges: (1) signal dilution, where coarse global representations blur the brief relevant signal into the dominant irrelevant surroundings;(2) curvature rigidity, where embedding all videos in the same fixed-geometry space distorts representations for videos that range from flat atomic events to deep compositional hierarchies. Existing PRVR methods have improved moment selection and cross-modal matching, but they still typically encode all videos in a single fixed-curvature retrieval space, limiting their ability to model diverse video structures. To address both challenges, we propose CurvSpec, a framework that learns content-adaptive curvature for video retrieval representations rather than imposing a fixed geometric prior. CurvSpec processes features through parallel Euclidean and hyperbolic attention layers, with independently learned curvatures assigned to the hyperbolic layers, and a content-aware fusion mechanism routes each input to its most suitable geometric regime. To further suppress signal dilution, CurvSpec represents each video with semantic centroids whose number is determined by the video's content complexity, projects them onto the learned manifold, and matches each query against its nearest centroid by geodesic distance. Experiments on ActivityNet Captions, TVR, and Charades-STA demonstrate state-of-the-art retrieval performance.
Figures & tables
Figure 1 . Motivation and key ideas. (a) Videos exhibit diverse structures (Video 1 to 3, from flat to deep hierarchies). Neither Euclidean space nor a single fixed hyperbolic curvature accommodates all of them; CurvSpec routes each video to its optimal geometry. (b) Global pooling dilutes relevant signals into an ambiguous single vector; semantic centroids decompose the video into coherent regions, allowing the query to match the nearest relevant centroid free from irrelevant interference.
Figure 2 . Overview of CurvSpec. (a) Text queries and video frames/clips are encoded and processed by the CurvSpec block to yield enriched representations Zc and Zf . Clip features are further decomposed into semantic centroids, which, together with the query, are projected onto the Lorentz manifold for fine-grained partial relevance matching. (b) The CurvSpec block processes input through parallel Euclidean and Lorentz attention branches, each at a distinct geometry. A content-aware fusion mechanism adaptively combines the multi-geometry representations.
Method
ActivityNet Captions
TVR
Charades-STA
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
R@1
R@5
R@10
R@100
SumR
Text-to-Video Retrieval
HGR ( Chen et al., 2020 )
4.0
15.0
24.8
63.2
107.0
1.7
4.9
8.3
35.2
50.1
1.2
3.8
7.3
33.4
45.7
DE++ ( Dong et al., 2022b )
5.3
18.4
29.2
68.0
121.0
8.8
21.9
30.2
67.4
128.3
1.7
5.6
9.6
37.1
54.1
RIVRL ( Dong et al., 2022c )
5.2
18.0
28.2
66.4
117.8
9.4
23.4
32.2
70.6
135.6
1.6
5.6
9.4
37.7
54.3
CLIP4Clip ( Luo et al., 2022 )
5.9
19.3
30.4
71.6
127.3
9.9
24.3
34.3
72.5
141.0
1.8
6.5
10.9
44.2
63.4
Table 1. Retrieval performance on ActivityNet Captions, TVR, and Charades-STA. Bold and underline denote the best and second-best results, respectively. “–” indicates unavailable results.
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos containing moments relevant to a given text query. Despite recent progress, existing PRVR methods suffer from two key limitations: a fixed video decomposition scheme that causes semantic dilution, and source-domain overfitting induced by task-specific training. In this paper, we propose TF-PRVR, the first training-free framework for PRVR. TF-PRVR leverages frozen vision-language features to construct video-specific hierarchical representations. It derives temporal semantic signals from frame-level features and applies frequency-based multi-scale analysis to identify adaptive temporal boundaries, producing hierarchical segments with coherent event-level semantics. Built on these segments, TF-PRVR constructs a unified multi-scale graph and propagates query relevance across temporally and semantically related nodes. A moment-aware scoring strategy then aggregates temporally aligned relevance across scales, emphasizing consistently supported moments while suppressing isolated false responses. Without task-specific training, TF-PRVR preserves the general-purpose alignment capability of pre-trained vision-language models and avoids dataset-specific overfitting. Extensive experiments demonstrate consistent performance across datasets with diverse visual and temporal characteristics, suggesting a practical direction for training-free PRVR.
Partially Relevant Video Retrieval (PRVR) retrieves untrimmed videos when queries describe only short moments. Although recent methods improve local representations, uncertainty modeling, and global context, final ranking often still trusts the strongest local response; a coincidentally similar fragment can therefore produce an unsupported peak. We identify this failure as the query-agnostic concentration bottleneck and propose TRACE, a score-level evidence verification operator for PRVR. Given a query and global video registers, TRACE activates query-relevant registers, routes their support to frame-level evidence, and smoothly marginalizes alternative query-to-register-to-frame paths before localized temporal selection. Unlike representation-level feature fusion, TRACE uses this evidence only as a query-conditioned residual calibration of the original local score. On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks and improves the DreamPRVR backbone by 1.2, 1.1, and 1.5 points, respectively. Ablation, routing-corruption, hard-negative, and cross-backbone transfer analyses support the interpretation that the gains arise from query-conditioned evidence verification rather than a generic score offset.
Shuaiqi Cheng, Siyu You, Yanbi Wu +3
1The Hong Kong University of Science and Technology (Guangzhou) · University of Electronic Science and Technology of China · 3The Hong Kong University of Science and Technology
Partially relevant video retrieval aims to retrieve untrimmed videos using text queries that describe only partial content. However, the inherent asymmetry between brief queries and rich video content inevitably introduces uncertainty into the retrieval process. In this setting, vague queries often induce semantic ambiguity across videos, a challenge that is further exacerbated by the sparse temporal supervision within videos, which fails to provide sufficient matching evidence. To address this, we propose Holmes, a hierarchical evidential learning framework that aggregates multi-granular cross-modal evidence to quantify and model uncertainty explicitly. At the inter-video level, similarity scores are interpreted as evidential support and modeled via a Dirichlet distribution. Based on the proposed three-fold principle, we perform fine-grained query identification, which then guides query-adaptive calibrated learning. At the intra-video level, to accumulate denser evidence, we formulate a soft query-clip alignment via flexible optimal transport with an adaptive dustbin, which alleviates sparse temporal supervision while suppressing spurious local responses. Extensive experiments demonstrate that Holmes outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/ICML26-Holmes.
Jun Li, Peifeng Lai, Xuhang Lou +5
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · Harbin Institute of Technology, Shenzhen, China