TOL: Textual Localization with OpenStreetMap

Youqi Liao; Shuhao Kang; Jingyu Xu; Olaf Wysocki; Yan Xia; Jianping Li; Zhen Dong; Bisheng Yang

cs.CVApr 2, 2026

TOL: Textual Localization with OpenStreetMap

Authors: Youqi Liao, Shuhao Kang, Jingyu Xu, Olaf Wysocki, Yan Xia, Jianping Li, Zhen Dong, Bisheng Yang,

Organizations: Y. Liao, Z. Dong and B. Yang are with Wuhan University. · S. Kang is with the Technical University of Munich. · J. Xu is with the Institute of Artificial Intelligence (TeleAI), China Telecom, China. · Olaf Wysocki is with the University of Cambridge, CV4DT, UK. ·

Abstract

Natural language provides an intuitive way to express spatial intent in geospatial applications. While existing localization methods often rely on dense point cloud maps or high-resolution imagery, OpenStreetMap (OSM) offers a compact and freely available map representation that encodes rich semantic and structural information, making it well-suited for large-scale localization. However, text-to-OSM (T2O) localization remains largely unexplored. In this paper, we formulate the T2O localization task, which aims to estimate accurate 2D positions in urban environments from textual scene descriptions without relying on geometric observations or GNSS-based initial location. To support the proposed task, we introduce TOL, a large-scale benchmark spanning multiple continents and diverse urban environments. TOL contains approximately 121K textual queries paired with OSM map tiles and covers about 316 km of road trajectories across Boston, Karlsruhe, and Singapore. We further propose TOLoc, a coarse-to-fine localization framework that explicitly models the semantics of surrounding objects and their directional information. In the coarse stage, direction-aware features are extracted from both textual descriptions and OSM tiles to construct global descriptors, which are used to retrieve candidate locations for the query. In the fine stage, the query text and top-1 retrieved tile are jointly processed, where a dedicated alignment module fuses the textual descriptor and local map features to regress the 2-DoF pose. Experimental results demonstrate that TOLoc achieves strong localization performance, outperforming the best existing method by 6.53%, 9.93%, and 8.32% at 5 m, 10 m, and 25 m thresholds, respectively, and shows strong generalization to unseen environments. Dataset, code and models will be publicly available at: https://github.com/WHU-USI3DV/TOL.

Figures & tables

Fig. 1: Overview and motivation of text-to-OSM localization. (a) Given a textual query, the system retrieves the most relevant OSM tile and estimates the final 2-DoF position. (b) Compared with point cloud map-based localization, OSM provides a lightweight map representation. Unlike existing text-driven retrieval methods that only return candidate tiles, our method further performs pose estimation for meter-level localization.

Type	Element
Areas	building, parking, playground, grass, park, forest, water
Ways	fence, wall, hedge, kerb, cycleway, path, road, busway, tree row
Nodes	parking entrance, street lamp, junction, traffic signal, stop sign, give way sign, bus stop, stop area, crossing, gate, bollard, gas station, bicycle parking, charging station, shop, restaurant, bar, vending machine, pharmacy, tree, stone, ATM, toilets, water fountain, bench, waste basket, post box, artwork, recycling station, clock, fire hydrant, pole, street cabinet

TABLE I: Details of OSM elements.

Fig. 2: Illustration of rasterized OSM tiles with area, way, and node channels.

Fig. 3: Pipeline of TOLoc. Given a query text $\mathcal{T}$ and an OSM database $\mathbb{O}=\{\mathcal{O}_{j}\}_{j=1}^{Z}$ , TOLoc performs coarse-to-fine localization. It first learns text–map correspondences to retrieve the top- $K$ candidate tiles, and then aligns the text descriptor $\boldsymbol{d}_{\mathcal{T}}$ with the feature map $\boldsymbol{F}_{\mathcal{O}}$ of the top-1 tile to predict the final 2-DoF position. The proposed TOA module fuses local OSM features and textual cues through self-attention and cross-attention for fine-grained localization.

Method	Backbone	$\eta=25$ m			$\eta=10$ m
Method	Backbone	R@1	R@5	R@10	R@1	R@5	R@10
GOTPR	GNN	11.53	22.91	29.91	6.39	14.36	18.91
CT2Loc	CLIP-B32	13.92	31.28	38.87	6.77	17.60	24.31
CT2Loc	CLIP-B16	18.24	35.34	44.50	8.75	20.02	27.63
TOLoc	CLIP-B32	24.80	37.61	45.00	12.11	23.78	28.95
	CLIP-B16	26.29	40.06	45.93	15.87	26.79	32.21

TABLE II: Place recognition results on the TOL-N set.

Method	Backbone	00	02	03	04	05	06	07	08	09	10	18
GOTPR	GNN	2.11	4.16	30.20	4.94	9.28	5.21	12.91	7.04	4.33	11.27	7.60
CT2Loc	CLIP-B32	4.54	5.64	30.79	5.75	11.60	7.62	9.41	10.56	6.33	12.79	13.19
CT2Loc	CLIP-B16	6.21	5.46	32.28	6.81	12.00	7.52	9.00	12.66	10.77	15.43	12.87
TOLoc	CLIP-B32	6.22	7.86	31.98	9.44	13.43	10.53	14.01	12.11	11.07	21.81	19.44
	CLIP-B16	10.12	9.84	36.63	10.22	13.81	11.28	16.99	17.26	11.91	23.93	16.67
	SigLIP-224	13.26	9.83	35.35	10.41	14.37	11.04	16.44	16.19	13.92	26.07	20.01

TABLE III: Cross-scene generalization of place recognition on the TOL-K360 set, measured by R@1 under the 25 m threshold.

Method	PE	SR $\uparrow$			LE $\downarrow$
Method	PE	@5m	@10m	@25m	$@5\%$	$@10\%$	$@25\%$
GOTPR	×	2.42	6.39	11.53	8.42	17.97	178.03
CT2Loc-B32	×	2.16	7.32	15.35	8.67	13.51	111.95
CT2Loc-B16	×	2.19	8.75	18.23	7.17	11.02	76.67
TOLoc-C-B32	×	3.74	12.11	24.80	5.99	8.62	26.16
TOLoc-C-B16	×	5.19	15.87	26.29	4.94	6.92	19.40

TABLE IV: Localization results on the TOL-N set.

Method	PE	Seq 00	Seq 02	Seq 03	Seq 04	Seq 05	Seq 06	Seq 07	Seq 08	Seq 09	Seq 10	Seq 18
GOTPR	×	0.88	2.57	15.45	2.26	4.12	2.41	7.13	2.77	1.54	4.60	2.51
CT2Loc-C-B32		1.32	2.86	12.87	2.15	4.80	3.35	3.74	4.70	2.12	5.09	4.15
CT2Loc-C-B16		2.33	2.51	15.25	2.67	4.59	3.15	3.36	5.40	3.40	5.22	4.12
TOLoc-C-B32		2.37	4.46	13.86	3.97	4.16	5.86	5.71	6.16	4.40	8.23	7.49
TOLoc-C-B16		4.20	5.96	18.91	4.86	6.18	7.01	7.96	9.70	4.64	9.48	6.13
TOLoc-S-B224		6.42	6.02	20.30	5.17	6.85	7.07	7.20	9.41	6.56	12.69	7.17

TABLE V: Cross-scene generalization of localization on the TOL-K360 set under the 10 m threshold.

Fig. 4: Qualitative results on the TOL benchmark. In columns 3–5, green boxes indicate correct retrievals (error $\leq$ 25 m), while red boxes indicate failures. In columns 2 and 6, black dots ${\color[rgb]{0,0,0}\bullet}$ denote ground-truth query locations and red dots ${\color[rgb]{1,0,0}\bullet}$ denote the estimated positions. Retrieval and localization errors (in meters) are reported in the lower-left corner.

Text offset	R@1	R@5	R@10
$1/12S$	53.55	73.33	82.29
$1/6S$	40.53	60.23	70.06
$1/3S$	28.83	49.75	59.94
$1/2S$	21.59	36.10	43.92

TABLE VI: Ablation study on text offset range.

Map-Based Localization

Map-Based Localization estimates a device or robot’s position and orientation by matching sensor observations to stored maps.

See all 106 papers

Uncertainty-Aware Hierarchical Re-Localization in OpenStreetMap via Semantic Alignment
Mar 2, 2026Yuchen Zou, Xiao Hu, Lihuang Fang +1Map-Based Localization
School of Automation Science and Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · International Digital Economy Academy, Guangdong, Shenzhen 510085, China · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen 518055, China
VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models
Mar 10, 2026Shuhao Kang, Youqi Liao, Peijie Wang +5Map-Based Localization 3D VLMs
VCIP, CS, Nankai University · Wuhan University · CASIA +6
PosEviLoc: Position-Conditioned Spatial Evidence for Language-Based 3D Localization
Sep 20, 2026Tianyi Shang, Yike Shi, Zhenyu LiMap-Based Localization 3D Visual Grounding
Purdue University · Xi’an Jiaotong-Liverpool University · Qilu University of Technology (Shandong Academy of Sciences)

Mar 2, 2026J/K move · Enter open · S save

Uncertainty-Aware Hierarchical Re-Localization in OpenStreetMap via Semantic Alignment

Yuchen Zou, Xiao Hu, Lihuang Fang, Yuqing Tang

School of Automation Science and Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · International Digital Economy Academy, Guangdong, Shenzhen 510085, China · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen 518055, China

Map-Based Localization

Open paper AskPDF arXiv

Monocular re-localization enables robots to estimate camera poses from visual observations. However, many existing methods rely on dense maps or large reference image databases, which face scalability limitations and privacy risks. OpenStreetMap (OSM), as a lightweight privacy-preserving map, offers semantic and geometric information with global scalability. Nonetheless, OSM localization remains challenging due to cross-modal discrepancies between natural images and OSM, as well as the high cost of global map-based localization. In this paper, we propose an uncertainty-aware hierarchical search framework with semantic alignment for localization in OSM. First, object-centric DINO-ViT tokens are exploited to reduce the semantic gap between ground-view observations and OSM vectors. Second, global dense matching is decomposed into coarse FFT correlation and uncertainty-controlled local refinement. Extensive experiments demonstrate that our method significantly improves localization accuracy and speed. When trained on a single dataset, the 3

^\circ

orientation recall of our method even outperforms the 5

^\circ

recall of state-of-the-art methods.

Abstract

Map-Based Localization

Explore similar work

Uncertainty-Aware Hierarchical Re-Localization in OpenStreetMap via Semantic Alignment

VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models

PosEviLoc: Position-Conditioned Spatial Evidence for Language-Based 3D Localization