Given a compact semantic scene graph, long-term indoor video relocalization estimates a map-frame trajectory after lighting and furniture changes. Visual methods rely on appearance and become unreliable under these changes; localizing one frame at a time from object classes and geometry instead leaves sparse, ambiguous evidence. We introduce VideoReloc, whose adaptive clips use odometry to gather spatial evidence until object and motion criteria are met, adapting query length to the observed scene. Its run-level decision rechecks conflicting placements using evidence accumulated across connected clips, stabilizing the trajectory beyond adjacent-clip tracking. Hypothesis-first registration proposes poses from object triplets and verifies each using clip-wide object centers and box surfaces. Orientation-aware refinement uses box faces, gravity and wall directions to resolve ambiguity in camera orientation and refine the full pose. This reframes sparse-map relocalization as verification of spatially extended video queries, moving discriminative support from stored appearance to temporal context and permitting a 100 kB map of class-labelled boxes. On RIO10 and ReplicaCAD, the all-frame localization success rate at 1 m/10∘ is 73.5% and 61.1% under causal evaluation, rising to 90.6% and 74.8% with clip closure. The evaluated per-frame scene coordinate regressors reach up to 47.6% and 49.8%, respectively, with maps of 12.6-42 MB. Project page: https://videoreloc.github.io
Figures & tables
Fig. 2: Overview of VideoReloc. Per-clip graph construction produces Lc from RGB-D odometry and depth-backed instance fusion. Clip candidates generation combines previous-clip prediction, triangle retrieval, frame-pose anchors and orientation fallback; box-surface consistency verifies them. The back-end applies orientation-aware refinement, the run-level decision and the clip-rigid pose graph; final frame poses satisfy Ti=ScVi .
RIO10, 10 re-scans, 34,415 frames
ReplicaCAD, 18 tours, 72,000 frames
Method
Map size ↓
Pos./rot. ↓
R@1 m/10 ∘ ↑
0.25 m/5 ∘ ↑
Pos./rot. ↓
R@1 m/10 ∘ ↑
0.25 m/5 ∘ ↑
HLoc [ 5 ]
1–2 GB
0 12 cm / 0 3.9 ∘
49.8
41.3
00 4 cm / 0 0.6 ∘
48.4
44.5
ACE-G [ 4 ]
12.6 MB
0 36 cm / 11.7 ∘
45.5
25.7
0 52 cm / 0 8.2 ∘
49.8
37.1
R-SCoRe [ 3 ]
42 MB
0 40 cm / 12.1 ∘
47.6
38.5
0 72 cm / 0 9.4 ∘
48.6
42.5
MSG-Loc [ 13 ]
1.5 MB
161 cm / 49.3 ∘
0 1.2
0 0.1
265 cm / 27.7 ∘
11.3
0 1.3
FPFH–ICP †
6–13 MB
0 11 cm / 0 3.4 ∘
22.4
21.7
00 1 cm / 0 0.2 ∘
10.0
0 9.9
TABLE I: Comparison of recall and pose errors across relocalization methods and evaluation variants on RIO10 and ReplicaCAD.
Illumination
Rearrangement
Method
day
evening
night
ΔL
unchanged
rearranged
ΔR
kept
HLoc
55.5
51.9
37.7
−17.7
75.9
42.9
−33.0
56 %
ACE-G
53.1
49.6
46.8
−6.3
78.4
44.1
−34.3
56 %
R-SCoRe
54.2
50.3
41.5
−12.6
74.7
43.4
−31.3
58 %
MSG-Loc
12.2
12.0
9.6
−2.6
33.0
6.9
−26.1
21 %
FPFH–ICP †
10.1
9.1
8.9
−1.2
40.1
3.2
−36.8
8 %
TABLE II: Recall at 1 m/10 ∘ under illumination and furniture changes in ReplicaCAD.
Table 4
Fig. 3: RIO10 qualitative results. Grey boxes are map objects; trajectory colour continuously encodes position error (green: 0 m; yellow: 0.5 m; red: ≥ 1 m). VideoReloc uses estimates with clip closure. Lower plots show log-scale rolling median translation/rotation errors; dashed lines mark 1 m/10 ∘ .
Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.
Zhaopeng Cui, Jiarui Hu, Jingbo Liu +7
1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group
Relative camera pose is a basic geometric cue for visual re-localization and scene understanding. When using images alone, visual evidence may not sufficiently support reliable estimation of 3D-consistent motion. The challenge is especially acute in unfamiliar and ever-changing environments for real-time applications, where a system must infer spatial structure under tight time constraints from what it sees rather than rely on scene-specific reconstruction or training. This raises a fundamental question: how can live 3D camera motion be estimated accurately for re-localization in unseen environments while drawing on enough scene context to resolve pose ambiguity? We introduce MultiLoc, a multi-view-guided relative pose regressor trained at scale to achieve spatial and geometric consistent representations for robust visual re-localization in unseen scenarios with high inference speed. Specifically, MultiLoc creates a minimal 3D-sub-scene representation of the environment and efficiently transforms it into 3D spatially and geometrically consistent features in a single forward pass, enabling precise pose estimates with high inference speed gains. Across diverse indoor, outdoor, and in-the-wild visual re-localization benchmarks---Indoor6, Cambridge Landmarks and WaySpots---MultiLoc consistently outperforms state-of-the-art relative pose regression methods. We also highlight that MultiLoc, a pose regressor, also performs competitively with inference-heavy and structure-based approaches while retaining sub-second inference and generalizing to unseen scenes. Given a small posed support set, it also surpasses relative pose regression, feature-matching, and non-regression methods on several relative camera pose benchmarks. Code will be released.
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.