Riemannian and Symplectic Geometry for Hierarchical Text-Driven Place Recognition

Tianyi Shang; Zhenyu Li

cs.CVApr 2, 2026

Riemannian and Symplectic Geometry for Hierarchical Text-Driven Place Recognition

Authors: Tianyi Shang, Zhenyu Li

Abstract

Text-to-point-cloud localization enables robots to understand spatial positions through natural language descriptions, which is crucial for human-robot collaboration in applications such as autonomous driving and last-mile delivery. However, existing methods employ pooled global descriptors for similarity retrieval, which suffer from severe information loss and fail to capture discriminative scene structures. To address these issues, we propose SympLoc, a novel coarse-to-fine localization framework with multi-level alignment in the coarse stage. Different from previous methods that rely solely on global descriptors, our coarse stage consists of three complementary alignment levels: 1) Instance-level alignment establishes direct correspondence between individual object instances in point clouds and textual hints through Riemannian self-attention in hyperbolic space; 2) Relation-level alignment explicitly models pairwise spatial relationships between objects using the Information-Symplectic Relation Encoder (ISRE), which reformulates relation features through Fisher-Rao metric and Hamiltonian dynamics for uncertainty-aware geometrically consistent propagation; 3) Global-level alignment synthesizes discriminative global descriptors via the Spectral Manifold Transform (SMT) that extracts structural invariants through graph spectral analysis. This hierarchical alignment strategy progressively captures fine-grained to coarse-grained scene semantics, enabling robust cross-modal retrieval. Extensive experiments on the KITTI360Pose dataset demonstrate that SympLoc achieves a 19% improvement in Top-1 recall@10m compared to existing state-of-the-art approaches.

Figures & tables

Fig. 1: Overview of SympLoc, a coarse-to-fine framework for language-guided 3D point cloud localization. The framework comprises three complementary alignment modules in coarse stage: 1) Relation Level Alignment with the Information-Symplectic Relation Encoder (ISRE), which models pairwise spatial relationships through differential geometry; 2) Global Level Alignment with the Spectral Manifold Transform (SMT), which extracts structural invariants via Chebyshev spectral filtering; and 3) Instance Level Alignment with the Riemannian Instance Enhancer (RIE), which embeds features into hyperbolic space to capture hierarchical scene structures. The coarse stage retrieves the most relevant submap via multi-branch similarity aggregation, while the fine stage predicts the precise spatial location through cascaded cross-attention and MLP regression.

Methods	Localization Recall ( $\epsilon<5/10/15m$ ) $\uparrow$
Methods	Validation Set			Test Set
	k = 1	k = 5	k = 10	k = 1	k = 5	k = 10
Text2Pos ( CVPR’22 ) [ 2 ]	0.14/0.25/0.31	0.36/0.55/0.61	0.48/0.68/0.74	0.13/0.20/0.30	0.33/0.42/0.49	0.43/0.61/0.65
RET ( AAAI’23 ) [ 3 ]	0.19/0.30/0.37	0.44/0.62/0.67	0.52/0.72/0.78	0.16/0.25/0.29	0.35/0.51/0.56	0.46/0.65/0.71
Text2Loc ( CVPR’24 ) [ 4 ]	0.37/0.57/0.63	0.68/0.85/0.87	0.77/0.91/0.93	0.33/0.48/0.52	0.60/0.75/0.78	0.70/0.84/0.86
IFRP-T2P ( ACM MM’24 ) [ 31 ]	0.23/0.45/0.53	0.53/0.70/0.81	0.64/0.86/0.89	0.22/0.40/0.46	0.47/0.68/0.73	0.58/0.78/0.82

TABLE I: A comprehensive comparison between SympLoc and existing SOTA methods on the KITTI360Pose dataset. (Best results are bolded , and second-best results are underlined .)

Methods	Submap Retrieval Recall $\uparrow$
	Validation Set			Test Set
	k = 1	k = 3	k = 5	k = 1	k = 3	k = 5
Text2Pos	0.14	0.28	0.37	0.12	0.25	0.33
RET	0.18	0.34	0.44	0.15	0.29	0.37
Text2Loc	0.31	0.54	0.64	0.28	0.49	0.58
IFRP-T2P	0.24	0.46	0.57	0.23	0.39	0.48

TABLE II: Text to point cloud submap retrieval accuracy of SympLoc and SOTA methods in the coarse stage on the KITTI360Pose dataset. SympLoc (global) denotes the variant of our method that only utilizes the global branch in the coarse stage.

Methods			Retrieval Recall $\uparrow$
Relation	Instance	Global	k=1	k=3	k=5
✓			0.46	0.70	0.78
	✓		0.48	0.72	0.79
		✓	0.39	0.62	0.71
✓	✓		0.50	0.74	0.81
✓		✓	0.48	0.72	0.80

TABLE III: Ablation study investigating the contributions of the three independent branches, Relation, Instance, and Global—within the coarse retrieval framework.

Methods	Submap Retrieval Recall $\uparrow$
	Validation Set			Test Set
	k = 1	k = 3	k = 5	k = 1	k = 3	k = 5
w/o RIE	0.52	0.78	0.86	0.49	0.73	0.80
w/o ISRE	0.53	0.79	0.87	0.49	0.71	0.81
w/o SMT	0.51	0.77	0.85	0.48	0.72	0.79
SympLoc	0.54	0.81	0.89	0.51	0.75	0.82

TABLE IV: Ablation study on core innovations of SympLoc. We independently remove RIE, ISRE, and SMT to verify their contributions to the coarse-stage text-to-point cloud submap retrieval on the test set.

Fig. 2: In this visualization analysis experiment, we present the textual descriptions, the point clouds of the actual location (ground truth), and the top K retrieved point-cloud submaps. We mark the retrieved position in each submap with a star symbol. If the Euclidean distance between the retrieved coordinates and the actual coordinates is within 15 meters, we consider this retrieval correct and highlight the submap with a green box. Incorrect submaps are marked with red boxes.

Robot Localization

Robot localization estimates a robot’s position and orientation from onboard sensors, maps, and observations of its surroundings.

See all 177 papers

Nov 19, 2025

Text2Loc++: Generalizing 3D Point Cloud Localization from Natural Language

We tackle the problem of localizing 3D point cloud submaps using complex and diverse natural language descriptions, and present Text2Loc++, a novel neural network designed for effective cross-modal alignment between language and point clouds in a coarse-to-fine localization pipeline. To support benchmarking, we introduce a new city-scale dataset covering both color and non-color point clouds from diverse urban scenes, and organize location descriptions into three levels of linguistic complexity. In the global place recognition stage, Text2Loc++ combines a pretrained language model with a Hierarchical Transformer with Max Pooling (HTM) for sentence-level semantics, and employs an attention-based point cloud encoder for spatial understanding. We further propose Masked Instance Training (MIT) to filter out non-aligned objects and improve multimodal robustness. To enhance the embedding space, we introduce Modality-aware Contrastive Learning (MCL), incorporating cross-modal, submap-, text-, and instance-level losses. In the fine localization stage, we completely remove explicit text-instance matching and design a lightweight yet powerful framework based on Prototype-based Map Cloning (PMC) and a Cascaded Cross-Attention Transformer (CCAT). Extensive experiments on the KITTI360Pose dataset show that Text2Loc++ outperforms existing methods by up to 15%. In addition, the proposed model exhibits robust generalization when evaluated on the new dataset, effectively handling complex linguistic expressions and a wide variety of urban environments.

Yan Xia, Letian Shi, Yilin Di +2

the School of Artificial Intelligence and Data Science, University of Science and Technology of China, 230026 Hefei, China · Technical University of Munich, 80333 Munich, Germany · the Visual Geometry Group, University of Oxford, Oxford OX1 2JD, United Kingdom

3D VLMs Camera Pose Estimation

Ask PDF

Oct 7, 2026

PARC-Loc: Text-to-Point-Cloud Localization with Partial Assignment and Relational Consistency

Text-to-point-cloud localization estimates a position in a city-scale 3D map from descriptions of surrounding objects. Existing coarse-to-fine methods retrieve submaps using aggregate learned compatibility and then localize within a selected submap. However, repetitive or similar urban objects can inflate the embedding similarity between the query and multiple submaps, even when the instance layout within a submap violates the query description. Meanwhile, query-relevant instances often span submap boundaries, leaving the retrieved submap with incomplete contextual evidence. We term these failure modes layout-inconsistent aliasing and boundary evidence incompleteness, respectively. To address them, we propose PARC-Loc, a coarse-to-fine localization framework built on Partial Assignment with Relational Consistency (PARC). PARC jointly models hint-object compatibility and pairwise spatial relations, allowing unmatched elements while favoring assignments consistent with the queried layout. At the coarse stage, its candidate-level assessment complements neural similarity for layout-consistent submap selection. At the fine stage, the context is expanded with query-relevant instances from adjacent submaps, while PARC yields object-level matching weights that guide cross-modal attention. Extensive experiments on KITTI360Pose and CityLoc show that PARC-Loc outperforms conventional coarse-to-fine baselines. On KITTI360Pose, our method improves Top-1 localization recall at 5 m from 0.50 to 0.67, achieving a 34% relative gain over the strongest baseline.

Shengkai Ma, Zhenyu Hou, Weihua Cao

Map-Based Localization 3D Visual Grounding

Ask PDF

Mar 10, 2026

VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models

Text-to-point-cloud (T2P) localization aims to infer precise spatial positions within 3D point cloud maps from natural language descriptions, reflecting how humans perceive and communicate spatial layouts through language. However, existing methods largely rely on shallow text-point cloud correspondence without effective spatial reasoning, limiting their accuracy in complex environments. To address this limitation, we propose VLM-Loc, a framework that leverages the spatial reasoning capability of large vision-language models (VLMs) for T2P localization. Specifically, we transform point clouds into bird's-eye-view (BEV) images and scene graphs that jointly encode geometric and semantic context, providing structured inputs for the VLM to learn cross-modal representations bridging linguistic and spatial semantics. On top of these representations, we introduce a partial node assignment mechanism that explicitly associates textual cues with scene graph nodes, enabling interpretable spatial reasoning for accurate localization. To facilitate systematic evaluation across diverse scenes, we present CityLoc, a benchmark built from multi-source point clouds for fine-grained T2P localization. Experiments on CityLoc demonstrate VLM-Loc achieves superior accuracy and robustness compared to state-of-the-art methods. Our code, model, and dataset are available at repository.

Shuhao Kang, Youqi Liao, Peijie Wang +5

VCIP, CS, Nankai University · Wuhan University · CASIA +6

Map-Based Localization 3D VLMs

Ask PDF

Abstract

Robot Localization

Explore similar work

Text2Loc++: Generalizing 3D Point Cloud Localization from Natural Language

PARC-Loc: Text-to-Point-Cloud Localization with Partial Assignment and Relational Consistency

VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models