Authors: Youqi Liao, Shuhao Kang, Jingyu Xu, Olaf Wysocki, Yan Xia, Jianping Li, Zhen Dong, Bisheng Yang, Xieyuanli Chen
Organizations: Y. Liao, Z. Dong and B. Yang are with Wuhan University. · S. Kang is with the Technical University of Munich. · J. Xu is with the Institute of Artificial Intelligence (TeleAI), China Telecom, China. · Olaf Wysocki is with the University of Cambridge, CV4DT, UK. · Y. Xia is with the University of Science and Technology of China, China. · J. Li is with Nanyang Technological University. · X. Chen is with the national university of defense technology, China.
Natural language provides an intuitive way to express spatial intent in geospatial applications. While existing localization methods often rely on dense point cloud maps or high-resolution imagery, OpenStreetMap (OSM) offers a compact and freely available map representation that encodes rich semantic and structural information, making it well-suited for large-scale localization. However, text-to-OSM (T2O) localization remains largely unexplored. In this paper, we formulate the T2O localization task, which aims to estimate accurate 2D positions in urban environments from textual scene descriptions without relying on geometric observations or GNSS-based initial location. To support the proposed task, we introduce TOL, a large-scale benchmark spanning multiple continents and diverse urban environments. TOL contains approximately 121K textual queries paired with OSM map tiles and covers about 316 km of road trajectories across Boston, Karlsruhe, and Singapore. We further propose TOLoc, a coarse-to-fine localization framework that explicitly models the semantics of surrounding objects and their directional information. In the coarse stage, direction-aware features are extracted from both textual descriptions and OSM tiles to construct global descriptors, which are used to retrieve candidate locations for the query. In the fine stage, the query text and top-1 retrieved tile are jointly processed, where a dedicated alignment module fuses the textual descriptor and local map features to regress the 2-DoF pose. Experimental results demonstrate that TOLoc achieves strong localization performance, outperforming the best existing method by 6.53%, 9.93%, and 8.32% at 5 m, 10 m, and 25 m thresholds, respectively, and shows strong generalization to unseen environments. Dataset, code and models will be publicly available at: https://github.com/WHU-USI3DV/TOL.
Figures & tables
Fig. 1: Overview and motivation of text-to-OSM localization. (a) Given a textual query, the system retrieves the most relevant OSM tile and estimates the final 2-DoF position. (b) Compared with point cloud map-based localization, OSM provides a lightweight map representation. Unlike existing text-driven retrieval methods that only return candidate tiles, our method further performs pose estimation for meter-level localization.
Type
Element
Areas
building, parking, playground, grass, park, forest, water
Ways
fence, wall, hedge, kerb, cycleway, path, road, busway, tree row
Nodes
parking entrance, street lamp, junction, traffic signal, stop sign, give way sign, bus stop, stop area, crossing, gate, bollard, gas station, bicycle parking, charging station, shop, restaurant, bar, vending machine, pharmacy, tree, stone, ATM, toilets, water fountain, bench, waste basket, post box, artwork, recycling station, clock, fire hydrant, pole, street cabinet
TABLE I: Details of OSM elements.
Fig. 2: Illustration of rasterized OSM tiles with area, way, and node channels.
Fig. 3: Pipeline of TOLoc. Given a query text T and an OSM database O={Oj}j=1Z , TOLoc performs coarse-to-fine localization. It first learns text–map correspondences to retrieve the top- K candidate tiles, and then aligns the text descriptor dT with the feature map FO of the top-1 tile to predict the final 2-DoF position. The proposed TOA module fuses local OSM features and textual cues through self-attention and cross-attention for fine-grained localization.
Method
Backbone
η=25 m
η=10 m
R@1
R@5
R@10
R@1
R@5
R@10
GOTPR
GNN
11.53
22.91
29.91
6.39
14.36
18.91
CT2Loc
CLIP-B32
13.92
31.28
38.87
6.77
17.60
24.31
CLIP-B16
18.24
35.34
44.50
8.75
20.02
27.63
TOLoc
CLIP-B32
24.80
37.61
45.00
12.11
23.78
28.95
CLIP-B16
26.29
40.06
45.93
15.87
26.79
32.21
TABLE II: Place recognition results on the TOL-N set.
Method
Backbone
00
02
03
04
05
06
07
08
09
10
18
GOTPR
GNN
2.11
4.16
30.20
4.94
9.28
5.21
12.91
7.04
4.33
11.27
7.60
CT2Loc
CLIP-B32
4.54
5.64
30.79
5.75
11.60
7.62
9.41
10.56
6.33
12.79
13.19
CLIP-B16
6.21
5.46
32.28
6.81
12.00
7.52
9.00
12.66
10.77
15.43
12.87
TOLoc
CLIP-B32
6.22
7.86
31.98
9.44
13.43
10.53
14.01
12.11
11.07
21.81
19.44
CLIP-B16
10.12
9.84
36.63
10.22
13.81
11.28
16.99
17.26
11.91
23.93
16.67
SigLIP-224
13.26
9.83
35.35
10.41
14.37
11.04
16.44
16.19
13.92
26.07
20.01
TABLE III: Cross-scene generalization of place recognition on the TOL-K360 set, measured by R@1 under the 25 m threshold.
Method
PE
SR ↑
LE ↓
@5m
@10m
@25m
@5%
@10%
@25%
GOTPR
×
2.42
6.39
11.53
8.42
17.97
178.03
CT2Loc-B32
×
2.16
7.32
15.35
8.67
13.51
111.95
CT2Loc-B16
×
2.19
8.75
18.23
7.17
11.02
76.67
TOLoc-C-B32
×
3.74
12.11
24.80
5.99
8.62
26.16
TOLoc-C-B16
×
5.19
15.87
26.29
4.94
6.92
19.40
TABLE IV: Localization results on the TOL-N set.
Method
PE
Seq 00
Seq 02
Seq 03
Seq 04
Seq 05
Seq 06
Seq 07
Seq 08
Seq 09
Seq 10
Seq 18
GOTPR
×
0.88
2.57
15.45
2.26
4.12
2.41
7.13
2.77
1.54
4.60
2.51
CT2Loc-C-B32
1.32
2.86
12.87
2.15
4.80
3.35
3.74
4.70
2.12
5.09
4.15
CT2Loc-C-B16
2.33
2.51
15.25
2.67
4.59
3.15
3.36
5.40
3.40
5.22
4.12
TOLoc-C-B32
2.37
4.46
13.86
3.97
4.16
5.86
5.71
6.16
4.40
8.23
7.49
TOLoc-C-B16
4.20
5.96
18.91
4.86
6.18
7.01
7.96
9.70
4.64
9.48
6.13
TOLoc-S-B224
6.42
6.02
20.30
5.17
6.85
7.07
7.20
9.41
6.56
12.69
7.17
TABLE V: Cross-scene generalization of localization on the TOL-K360 set under the 10 m threshold.
Fig. 4: Qualitative results on the TOL benchmark. In columns 3–5, green boxes indicate correct retrievals (error ≤ 25 m), while red boxes indicate failures. In columns 2 and 6, black dots {\color[rgb]{0,0,0}\bullet} denote ground-truth query locations and red dots {\color[rgb]{1,0,0}\bullet} denote the estimated positions. Retrieval and localization errors (in meters) are reported in the lower-left corner.
Monocular re-localization enables robots to estimate camera poses from visual observations. However, many existing methods rely on dense maps or large reference image databases, which face scalability limitations and privacy risks. OpenStreetMap (OSM), as a lightweight privacy-preserving map, offers semantic and geometric information with global scalability. Nonetheless, OSM localization remains challenging due to cross-modal discrepancies between natural images and OSM, as well as the high cost of global map-based localization. In this paper, we propose an uncertainty-aware hierarchical search framework with semantic alignment for localization in OSM. First, object-centric DINO-ViT tokens are exploited to reduce the semantic gap between ground-view observations and OSM vectors. Second, global dense matching is decomposed into coarse FFT correlation and uncertainty-controlled local refinement. Extensive experiments demonstrate that our method significantly improves localization accuracy and speed. When trained on a single dataset, the 3∘ orientation recall of our method even outperforms the 5∘ recall of state-of-the-art methods.
Yuchen Zou, Xiao Hu, Lihuang Fang +1
School of Automation Science and Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · International Digital Economy Academy, Guangdong, Shenzhen 510085, China · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen 518055, China
Text-to-point-cloud (T2P) localization aims to infer precise spatial positions within 3D point cloud maps from natural language descriptions, reflecting how humans perceive and communicate spatial layouts through language. However, existing methods largely rely on shallow text-point cloud correspondence without effective spatial reasoning, limiting their accuracy in complex environments. To address this limitation, we propose VLM-Loc, a framework that leverages the spatial reasoning capability of large vision-language models (VLMs) for T2P localization. Specifically, we transform point clouds into bird's-eye-view (BEV) images and scene graphs that jointly encode geometric and semantic context, providing structured inputs for the VLM to learn cross-modal representations bridging linguistic and spatial semantics. On top of these representations, we introduce a partial node assignment mechanism that explicitly associates textual cues with scene graph nodes, enabling interpretable spatial reasoning for accurate localization. To facilitate systematic evaluation across diverse scenes, we present CityLoc, a benchmark built from multi-source point clouds for fine-grained T2P localization. Experiments on CityLoc demonstrate VLM-Loc achieves superior accuracy and robustness compared to state-of-the-art methods. Our code, model, and dataset are available at repository.
Shuhao Kang, Youqi Liao, Peijie Wang +5
VCIP, CS, Nankai University · Wuhan University · CASIA +6
Language-based 3D localization retrieves the point-cloud submap containing a target position from descriptions of nearby objects and their spatial relations. Existing methods typically compress queries and submaps into global descriptors, potentially obscuring object-level semantics and cross-description spatial coherence. We propose Position-Conditioned Evidence Localization (PosEviLoc), a query-position-aware framework for coarse text-to-point-cloud localization. Instead of relying on global matching, PosEviLoc evaluates each candidate submap using explicit semantic and spatial evidence. It models direction as a relation jointly determined by an object position and a hypothetical query position. The resulting Query-Position Spatial Evidence Field (QSEF) measures the fraction of query descriptions supported at each hypothetical position, explicitly capturing their agreement without using the ground-truth query pose to construct the evidence field. A Multi-Level Evidence Readout (MER) summarizes this evidence in a compact representation, which a lightweight MLP converts into a retrieval score. Across five benchmarks, PosEviLoc outperforms MNCL by an average of 17 percentage points in Recall@1. When used as a plug-and-play reranker, it improves MNCL by an average of 16 percentage points. Moreover, PosEviLoc introduces substantially fewer parameters and achieves faster inference speed than existing methods.
Tianyi Shang, Yike Shi, Zhenyu Li
Purdue University · Xi’an Jiaotong-Liverpool University · Qilu University of Technology (Shandong Academy of Sciences)