VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments

Jingyi Xu; Zhangshuo Qi; Zhongmiao Yan; Xuyu Gao; Qianyun Jiao; Songpengcheng Xia; Xieyuanli Chen; Ling Pei

cs.CVFeb 23, 2026

VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments

Authors: Jingyi Xu, Zhangshuo Qi, Zhongmiao Yan, Xuyu Gao, Qianyun Jiao, Songpengcheng Xia, Xieyuanli Chen, Ling Pei

Organizations: Shanghai Jiao Tong University. · Beijing Institute of Technology. · National University of Defense Technology.

Abstract

In autonomous driving, robust place recognition is critical for global localization and loop closure detection. While inter-modality fusion of camera and LiDAR data in multimodal place recognition (MPR) has shown promise in overcoming the limitations of unimodal counterparts, existing MPR methods basically attend to hand-crafted fusion strategies and heavily parameterized backbones that require costly retraining. To address this, we propose VGGT-MPR, a multimodal place recognition framework that adopts the Visual Geometry Grounded Transformer (VGGT) as a unified geometric engine for both global retrieval and re-ranking. In the global retrieval stage, VGGT extracts geometrically-rich visual embeddings through prior depth-aware and point map supervision, and densifies sparse LiDAR point clouds with predicted depth maps to improve structural representation. This enhances the discriminative ability of fused multimodal features and produces global descriptors for fast retrieval. Beyond global retrieval, we design a training-free re-ranking mechanism that exploits VGGT's cross-view keypoint-tracking capability. By combining mask-guided keypoint extraction with confidence-aware correspondence scoring, our proposed re-ranking mechanism effectively refines retrieval results without additional parameter optimization. Extensive experiments on large-scale autonomous driving benchmarks and our self-collected data demonstrate that VGGT-MPR achieves state-of-the-art performance, exhibiting strong robustness to severe environmental changes, viewpoint shifts, and occlusions. Our code and data will be made publicly available.

Figures & tables

Fig. 1: From VGGT to VGGT-MPR. VGGT is reinterpreted as a unified geometry-centric foundation to address modality-specific limitations in multimodal place recognition. It enhances visual representations with implicit structural awareness, densifies sparse LiDAR observations via depth estimation for global retrieval, and provides reliable cross-view point tracking for training-free re-ranking.

Fig. 2: Overview of our proposed VGGT-MPR. Capitalizing on the spatial perception capabilities of VGGT, the pipeline integrates a global retrieval module (GRM) and a re-ranking mechanism (RRM). The GRM fuses multimodal inputs (camera images and LiDAR point clouds) to generate global descriptors for database indexing and retrieval. Subsequently, the RRM refines the retrieved candidates to improve place recognition accuracy.

Approach	Modality	BS Split				SON Split				SQ Split
Approach	Modality	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20
PatchNetVLAD	V	83.41	89.96	94.21	96.75	95.34	97.97	98.82	90.01	86.92	88.94	89.42	91.30
CricaVPR	V	87.10	97.34	99.30	99.86	93.94	98.74	98.99	99.75	87.80	91.46	92.68	94.21
CVTNet	L	86.09	94.96	96.88	97.36	98.04	98.16	99.45	99.45	98.93	99.40	99.75	100.00
OverlapMamba	L	74.58	88.28	93.29	96.64	86.87	97.47	98.48	98.74	95.73	99.39	99.70	100.00
MinkLoc++	V+L	76.72	92.57	96.07	97.19	89.39	98.74	99.49	100.00	89.94	96.04	97.87	98.48

TABLE I: Performance comparison on the nuScenes dataset

Fig. 3: Our proposed re-ranking mechanism in VGGT-MPR. Given a query image and a corresponding candidate from the top- $k$ matches (e.g., candidates A or B ), we perform mask-guided keypoint extraction (a) and confidence-aware correspondence scoring (b) to calculate the score for the input candidate. The top- $k$ candidates are ultimately re-ranked based on their respective correspondence scores to enhance recognition accuracy.

Fig. 4: The unmanned ground vehicle (UGV) and collection trajectory for our self-collected data.

Fig. 5: Visualization of retrieval results. The first column shows the current query image, the second and third columns show the top- $1$ places retrieved by LCPR and our method, respectively. The value in the bottom-right corner of each retrieved image indicates its distance to the query position.

Approach	Modality	Query: 2012-06-15 Database: 2012-01-08				Query: 2013-02-23 Database: 2012-01-08				Query: 2013-04-05 Database: 2012-01-08
Approach	Modality	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20
PatchNetVLAD	V	59.27	65.09	67.23	68.69	53.22	58.91	61.71	64.01	55.72	62.03	64.93	67.80
CricaVPR	V	63.33	71.78	75.57	78.69	53.92	63.28	68.13	73.43	54.24	62.09	66.54	71.34
CVTNet	L	84.14	88.94	90.92	93.01	84.90	89.81	91.77	93.30	82.25	87.62	89.67	91.80
OverlapMamba	L	63.71	75.52	79.97	84.48	62.69	74.26	78.24	82.60	58.48	69.86	75.02	79.39
MinkLoc++	V+L	42.77	51.89	56.24	60.52	35.44	47.06	53.58	61.40	36.38	56.25	64.76	72.48

TABLE II: Performance comparison on the NCLT dataset

Approach	00			02			05
Approach	AR@1	AR@5	AR@10	AR@1	AR@5	AR@10	AR@1	AR@5	AR@10
MinkLoc++	77.84	81.36	82.61	68.96	73.73	74.33	61.23	65.26	66.84
LCPR	51.82	70.80	78.07	35.52	54.93	62.99	40.18	56.84	62.99
EINet	70.15	78.04	78.64	59.61	61.02	69.37	53.49	59.96	64.05
VGGT-MPR	82.69	83.41	84.54	71.94	74.55	75.84	66.84	67.19	68.37

TABLE III: Performance comparison on the KITTI dataset

Approach	AR@1	AR@5	AR@10	AR@20
MinkLoc++	62.67	72.45	77.28	80.26
LCPR	57.01	69.55	76.74	80.93
EINet	70.44	81.48	85.23	87.80
GSPR	47.88	65.31	74.76	78.35
VGGT-MPR	76.05	87.52	92.59	94.17

TABLE IV: Performance comparison on our self-collected data

Vision Input	LiDAR Input	AR@1	AR@5	AR@10	AR@20
✗	✓	89.12	98.84	99.32	99.32
✓	✗	94.01	99.10	99.54	99.77
✓	✓	98.28	99.76	99.93	99.93

TABLE V: Ablation study on input modalities

Depth Densification	Visual Embedding Extraction	AR@1	AR@5	AR@10	AR@20
✗	✗	92.51	98.32	99.42	99.73
✓	✗	95.84	99.10	99.56	99.79
✗	✓	97.91	99.68	99.89	99.90
✓	✓	98.28	99.76	99.93	99.93

TABLE VI: Ablation study on VGGT functions in GRM

Approach	AR@1	AR@5	AR@10	AR@20
w/o RRM (nuScenes)	97.21	99.42	99.78	99.84
w/ RRM (nuScenes)	98.28	99.76	99.93	99.93
w/o RRM (NCLT)	85.22	91.60	94.34	96.92
w/ RRM (NCLT)	86.29	93.03	94.83	97.53
w/o RRM (KITTI)	81.93	83.18	83.98	84.89
w/ RRM (KITTI)	82.69	83.41	84.54	85.45

TABLE VII: Ablation study on the re-ranking mechanism

Fig. 6: Visualization of retrieval results on the self-collected dataset. The first two columns present the query LiDAR point clouds and camera image, while the last two columns display the corresponding retrieved matches.

Fig. 7: Qualitative comparison of top- $1$ retrieval results before and after re-ranking. Colored lines indicate keypoint tracking correspondences between the query and the candidate image, where the color denotes the tracking confidence from VGGT, i.e., high confidence ( $>0.75$ ) in green, medium-high confidence ( $0.50$ – $0.75$ ) in yellow, medium-low confidence ( $0.25$ – $0.50$ ) in purple, and low confidence ( $<0.25$ ) in blue. The correspondence score and real distance are displayed in the bottom-left and bottom-right corners of each query-candidate pair.

Number	AR@1	AR@5	AR@10	AR@20
1	97.21	-	-	-
5	97.76	99.44	-	-
10	97.96	99.69	99.70	-
15	97.73	99.76	99.82	-
20	97.78	99.60	99.86	99.88
25	97.80	99.65	99.86	99.88

TABLE VIII: Ablation study on the number of retrieved candidates

Explore similar work

Aug 10, 2026

GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition

Cross-modal place recognition (CMPR) aims to identify the same location across heterogeneous sensing modalities, such as vision and LiDAR. Existing methods commonly bridge the modality gap using complex alignment modules, multi-stage training, or full fine-tuning of pretrained backbones. In this work, we revisit CMPR from the perspective of geometric consistency and propose GeoUniPR, a unified and concise geometry-consistent framework. GeoUniPR reduces cross-modal discrepancy at the representation level by projecting LiDAR point clouds into the camera perspective to construct Geometry-Consistent depth image views (DIV), which establish direct RGB-LiDAR correspondence. We further augment DIV with native LiDAR cues, including intensity and surface-normal information, yielding a multi-channel geometric representation that improves structural consistency. Based on this representation, GeoUniPR learns a unified embedding space using two modality-specific ViT-based encoders with identical architectures, trained through parameter-efficient adaptation without auxiliary alignment modules, multi-stage training, or full backbone fine-tuning. In addition, we introduce Spatially-Consistent InfoNCE (SC-InfoNCE), a CMPR-specific contrastive objective that suppresses distance-induced false negatives under spatial continuity. Extensive experiments on KITTI and KITTI-360 demonstrate that GeoUniPR achieves state-of-the-art (SOTA) performance in both same-modal and cross-modal place recognition, with strong cross-dataset generalization.

Wonbong Kim, Jiatong Xiao, Rui Li +5

School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China +1

Cross-Modal Representation Learning LiDAR Place Recognition Visual Place Recognition Geometric Representation Learning Multimodal Contrastive Learning

Ask PDF

Jun 13, 2026

G2IA: Geometry-Guided Instance-Aware Retrieval and Refinement for Cross-Modal Place Recognition

Cross-modal place recognition (CMPR) enables camera-only robots to localize against pre-built LiDAR maps in autonomous navigation scenarios. This image-to-point-cloud setting is challenged by two coupled ambiguities: the modality gap between perspective RGB appearance and sparse metric geometry, and perceptual aliasing among urban places with similar roads, facades, intersections, and object arrangements. Instead of treating CMPR as a single global descriptor matching problem, we argue that reliable retrieval requires both geometry-aware representation alignment and fine-grained candidate verification. In this paper, we propose G2IA, a geometry-guided instance-aware framework for image-to-point-cloud place recognition. In the retrieval stage, visual geometry priors from VGGT and instance features are integrated to construct place descriptors that are more compatible with LiDAR-derived map representations. In the refinement stage, the retrieved candidates are re-ranked by explicitly verifying whether local instance shapes and their relative spatial layouts are consistent across modalities. Experiments on public benchmarks demonstrate that G2IA consistently improves image-to-point-cloud place recognition under different localization thresholds, and exhibits strong cross-dataset generalization.

Xianyun Jiao, Jingyi Xu, Zhongmiao Yan +2

Shanghai Jiao Tong University · National University of Defense Technology

LiDAR Place Recognition Visual Place Recognition Cross-Modal Feature Matching Map-Based Localization

Ask PDF

Apr 6, 2026

MPTF-Net: Multi-view Pyramid Transformer Fusion Network for LiDAR-based Place Recognition

LiDAR-based place recognition (LPR) is essential for global localization and loop-closure detection in large-scale SLAM systems. Existing methods typically construct global descriptors from Range Images or BEV representations for matching. BEV is widely adopted due to its explicit 2D spatial layout encoding and efficient retrieval. However, conventional BEV representations rely on simple statistical aggregation, which fails to capture fine-grained geometric structures, leading to performance degradation in complex or repetitive environments. To address this, we propose MPTF-Net, a novel multi-view multi-scale pyramid Transformer fusion network. Our core contribution is a multi-channel NDT-based BEV encoding that explicitly models local geometric complexity and intensity distributions via Normal Distribution Transform, providing a noise-resilient structural prior. To effectively integrate these features, we develop a customized pyramid Transformer module that captures cross-view interactive correlations between Range Image Views (RIV) and NDT-BEV at multiple spatial scales. Extensive experiments on the nuScenes, KITTI and NCLT datasets demonstrate that MPTF-Net achieves state-of-the-art performance, specifically attaining a Recall@1 of 96.31% on the nuScenes Boston split while maintaining an inference latency of only 10.02 ms, making it highly suitable for real-time autonomous unmanned systems.

Shuyuan Li, Zihang Wang, Xieyuanli Chen +5

LiDAR Place Recognition Feature Fusion Multi-Scale Feature Fusion

Ask PDF