VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments

Jingyi Xu; Zhangshuo Qi; Zhongmiao Yan; Xuyu Gao; Qianyun Jiao; Songpengcheng Xia; Xieyuanli Chen; Ling Pei

cs.CVFeb 23, 2026

VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments

Authors: Jingyi Xu, Zhangshuo Qi, Zhongmiao Yan, Xuyu Gao, Qianyun Jiao, Songpengcheng Xia, Xieyuanli Chen, Ling Pei

Organizations: Shanghai Jiao Tong University. · Beijing Institute of Technology. · National University of Defense Technology.

Abstract

In autonomous driving, robust place recognition is critical for global localization and loop closure detection. While inter-modality fusion of camera and LiDAR data in multimodal place recognition (MPR) has shown promise in overcoming the limitations of unimodal counterparts, existing MPR methods basically attend to hand-crafted fusion strategies and heavily parameterized backbones that require costly retraining. To address this, we propose VGGT-MPR, a multimodal place recognition framework that adopts the Visual Geometry Grounded Transformer (VGGT) as a unified geometric engine for both global retrieval and re-ranking. In the global retrieval stage, VGGT extracts geometrically-rich visual embeddings through prior depth-aware and point map supervision, and densifies sparse LiDAR point clouds with predicted depth maps to improve structural representation. This enhances the discriminative ability of fused multimodal features and produces global descriptors for fast retrieval. Beyond global retrieval, we design a training-free re-ranking mechanism that exploits VGGT's cross-view keypoint-tracking capability. By combining mask-guided keypoint extraction with confidence-aware correspondence scoring, our proposed re-ranking mechanism effectively refines retrieval results without additional parameter optimization. Extensive experiments on large-scale autonomous driving benchmarks and our self-collected data demonstrate that VGGT-MPR achieves state-of-the-art performance, exhibiting strong robustness to severe environmental changes, viewpoint shifts, and occlusions. Our code and data will be made publicly available.

Figures & tables

Fig. 1: From VGGT to VGGT-MPR. VGGT is reinterpreted as a unified geometry-centric foundation to address modality-specific limitations in multimodal place recognition. It enhances visual representations with implicit structural awareness, densifies sparse LiDAR observations via depth estimation for global retrieval, and provides reliable cross-view point tracking for training-free re-ranking.

Fig. 2: Overview of our proposed VGGT-MPR. Capitalizing on the spatial perception capabilities of VGGT, the pipeline integrates a global retrieval module (GRM) and a re-ranking mechanism (RRM). The GRM fuses multimodal inputs (camera images and LiDAR point clouds) to generate global descriptors for database indexing and retrieval. Subsequently, the RRM refines the retrieved candidates to improve place recognition accuracy.

Approach	Modality	BS Split				SON Split				SQ Split
Approach	Modality	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20
PatchNetVLAD	V	83.41	89.96	94.21	96.75	95.34	97.97	98.82	90.01	86.92	88.94	89.42	91.30
CricaVPR	V	87.10	97.34	99.30	99.86	93.94	98.74	98.99	99.75	87.80	91.46	92.68	94.21
CVTNet	L	86.09	94.96	96.88	97.36	98.04	98.16	99.45	99.45	98.93	99.40	99.75	100.00
OverlapMamba	L	74.58	88.28	93.29	96.64	86.87	97.47	98.48	98.74	95.73	99.39	99.70	100.00
MinkLoc++	V+L	76.72	92.57	96.07	97.19	89.39	98.74	99.49	100.00	89.94	96.04	97.87	98.48

TABLE I: Performance comparison on the nuScenes dataset

Fig. 3: Our proposed re-ranking mechanism in VGGT-MPR. Given a query image and a corresponding candidate from the top- $k$ matches (e.g., candidates A or B ), we perform mask-guided keypoint extraction (a) and confidence-aware correspondence scoring (b) to calculate the score for the input candidate. The top- $k$ candidates are ultimately re-ranked based on their respective correspondence scores to enhance recognition accuracy.

Fig. 4: The unmanned ground vehicle (UGV) and collection trajectory for our self-collected data.

Fig. 5: Visualization of retrieval results. The first column shows the current query image, the second and third columns show the top- $1$ places retrieved by LCPR and our method, respectively. The value in the bottom-right corner of each retrieved image indicates its distance to the query position.

Approach	Modality	Query: 2012-06-15 Database: 2012-01-08				Query: 2013-02-23 Database: 2012-01-08				Query: 2013-04-05 Database: 2012-01-08
Approach	Modality	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20	AR@1	AR@5	AR@10	AR@20
PatchNetVLAD	V	59.27	65.09	67.23	68.69	53.22	58.91	61.71	64.01	55.72	62.03	64.93	67.80
CricaVPR	V	63.33	71.78	75.57	78.69	53.92	63.28	68.13	73.43	54.24	62.09	66.54	71.34
CVTNet	L	84.14	88.94	90.92	93.01	84.90	89.81	91.77	93.30	82.25	87.62	89.67	91.80
OverlapMamba	L	63.71	75.52	79.97	84.48	62.69	74.26	78.24	82.60	58.48	69.86	75.02	79.39
MinkLoc++	V+L	42.77	51.89	56.24	60.52	35.44	47.06	53.58	61.40	36.38	56.25	64.76	72.48

TABLE II: Performance comparison on the NCLT dataset

Approach	00			02			05
Approach	AR@1	AR@5	AR@10	AR@1	AR@5	AR@10	AR@1	AR@5	AR@10
MinkLoc++	77.84	81.36	82.61	68.96	73.73	74.33	61.23	65.26	66.84
LCPR	51.82	70.80	78.07	35.52	54.93	62.99	40.18	56.84	62.99
EINet	70.15	78.04	78.64	59.61	61.02	69.37	53.49	59.96	64.05
VGGT-MPR	82.69	83.41	84.54	71.94	74.55	75.84	66.84	67.19	68.37

TABLE III: Performance comparison on the KITTI dataset

Approach	AR@1	AR@5	AR@10	AR@20
MinkLoc++	62.67	72.45	77.28	80.26
LCPR	57.01	69.55	76.74	80.93
EINet	70.44	81.48	85.23	87.80
GSPR	47.88	65.31	74.76	78.35
VGGT-MPR	76.05	87.52	92.59	94.17

TABLE IV: Performance comparison on our self-collected data

Vision Input	LiDAR Input	AR@1	AR@5	AR@10	AR@20
✗	✓	89.12	98.84	99.32	99.32
✓	✗	94.01	99.10	99.54	99.77
✓	✓	98.28	99.76	99.93	99.93

TABLE V: Ablation study on input modalities

Depth Densification	Visual Embedding Extraction	AR@1	AR@5	AR@10	AR@20
✗	✗	92.51	98.32	99.42	99.73
✓	✗	95.84	99.10	99.56	99.79
✗	✓	97.91	99.68	99.89	99.90
✓	✓	98.28	99.76	99.93	99.93

TABLE VI: Ablation study on VGGT functions in GRM

Approach	AR@1	AR@5	AR@10	AR@20
w/o RRM (nuScenes)	97.21	99.42	99.78	99.84
w/ RRM (nuScenes)	98.28	99.76	99.93	99.93
w/o RRM (NCLT)	85.22	91.60	94.34	96.92
w/ RRM (NCLT)	86.29	93.03	94.83	97.53
w/o RRM (KITTI)	81.93	83.18	83.98	84.89
w/ RRM (KITTI)	82.69	83.41	84.54	85.45

TABLE VII: Ablation study on the re-ranking mechanism

Fig. 6: Visualization of retrieval results on the self-collected dataset. The first two columns present the query LiDAR point clouds and camera image, while the last two columns display the corresponding retrieved matches.

Fig. 7: Qualitative comparison of top- $1$ retrieval results before and after re-ranking. Colored lines indicate keypoint tracking correspondences between the query and the candidate image, where the color denotes the tracking confidence from VGGT, i.e., high confidence ( $>0.75$ ) in green, medium-high confidence ( $0.50$ – $0.75$ ) in yellow, medium-low confidence ( $0.25$ – $0.50$ ) in purple, and low confidence ( $<0.25$ ) in blue. The correspondence score and real distance are displayed in the bottom-left and bottom-right corners of each query-candidate pair.

Number	AR@1	AR@5	AR@10	AR@20
1	97.21	-	-	-
5	97.76	99.44	-	-
10	97.96	99.69	99.70	-
15	97.73	99.76	99.82	-
20	97.78	99.60	99.86	99.88
25	97.80	99.65	99.86	99.88

TABLE VIII: Ablation study on the number of retrieved candidates

Explore similar work

GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition
Aug 10, 2026Wonbong Kim, Jiatong Xiao, Rui Li +5Cross-Modal Representation Learning LiDAR Place Recognition
School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China +1
G2IA: Geometry-Guided Instance-Aware Retrieval and Refinement for Cross-Modal Place Recognition
Jun 13, 2026Xianyun Jiao, Jingyi Xu, Zhongmiao Yan +2LiDAR Place Recognition Visual Place Recognition
Shanghai Jiao Tong University · National University of Defense Technology
MPTF-Net: Multi-view Pyramid Transformer Fusion Network for LiDAR-based Place Recognition
Apr 6, 2026Shuyuan Li, Zihang Wang, Xieyuanli Chen +5LiDAR Place Recognition Feature Fusion

Aug 10, 2026J/K move · Enter open · S save

GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition

Wonbong Kim, Jiatong Xiao, Rui Li, Xufei Wang+4

School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China+1

Cross-Modal Representation Learning LiDAR Place Recognition Visual Place Recognition Geometric Representation Learning Multimodal Contrastive Learning

Open paper AskPDF arXiv

Cross-modal place recognition (CMPR) aims to identify the same location across heterogeneous sensing modalities, such as vision and LiDAR. Existing methods commonly bridge the modality gap using complex alignment modules, multi-stage training, or full fine-tuning of pretrained backbones. In this work, we revisit CMPR from the perspective of geometric consistency and propose GeoUniPR, a unified and concise geometry-consistent framework. GeoUniPR reduces cross-modal discrepancy at the representation level by projecting LiDAR point clouds into the camera perspective to construct Geometry-Consistent depth image views (DIV), which establish direct RGB-LiDAR correspondence. We further augment DIV with native LiDAR cues, including intensity and surface-normal information, yielding a multi-channel geometric representation that improves structural consistency. Based on this representation, GeoUniPR learns a unified embedding space using two modality-specific ViT-based encoders with identical architectures, trained through parameter-efficient adaptation without auxiliary alignment modules, multi-stage training, or full backbone fine-tuning. In addition, we introduce Spatially-Consistent InfoNCE (SC-InfoNCE), a CMPR-specific contrastive objective that suppresses distance-induced false negatives under spatial continuity. Extensive experiments on KITTI and KITTI-360 demonstrate that GeoUniPR achieves state-of-the-art (SOTA) performance in both same-modal and cross-modal place recognition, with strong cross-dataset generalization.

Abstract

Explore similar work

GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition

G2IA: Geometry-Guided Instance-Aware Retrieval and Refinement for Cross-Modal Place Recognition

MPTF-Net: Multi-view Pyramid Transformer Fusion Network for LiDAR-based Place Recognition