PlaceSeek: Human-Centered Geospatial Retrieval of Urban Outdoor Places via Semantic Grounding and Affective Alignment

Ziqi Cui; Shangyu Lou

cs.CVAug 25, 2026

PlaceSeek: Human-Centered Geospatial Retrieval of Urban Outdoor Places via Semantic Grounding and Affective Alignment

Authors: Ziqi Cui, Shangyu Lou

Organizations: University of California, Santa Barbara & San Diego State University California, USA

Abstract

People search for urban outdoor places not only by category or function, but also by what activities a place can support and how it is perceived. Existing geospatial retrieval remains largely POIcentric and metadata-driven, making it difficult to satisfy openended, affective, or activity-oriented needs. We present PlaceSeek, a human-centered outdoor place retrieval framework that maps natural-language queries to geolocated street-view imagery. PlaceSeek introduces an intent-aware retrieval mechanism that decomposes user queries into functional and affective sub-intents. A Semantic Grounding Module verifies whether candidate street-view results contain the physical evidence needed to support the intended activity, while an Affective Alignment Module re-ranks physically valid candidates using a LoRA-adapted vision-language model trained on human urban perception judgments. We evaluate PlaceSeek on 31,956 street-view locations in Milan across 10 naturallanguage queries annotated by five human evaluators. PlaceSeek achieves 88.0% Precision@5, a mean match score of 3.39/4.0, and 0.920 nDCG@5, outperforming CLIP, fine-tuned CLIP, SigLIP, and a VQA-based baseline. Ablation results show that physical grounding is essential for retrieval validity, while affective alignment improves ranking quality among physically valid candidates. These findings highlight that complex urban spatial queries require modeling both verifiable visual evidence and human perceptual preferences. PlaceSeek provides a potential framework for human-centered nextgeneration geospatial retrieval systems.

Figures & tables

Figure 1. From POI-centric search to human-centered geolocation retrieval.

Figure 2. PlaceSeek pipeline. A user natural-language query is decomposed by an LLM parser into physical anchor words and affective need terms. The Semantic Grounding Module (SGM) performs coarse-to-fine image retrieval grounded in physical evidence; the Affective Alignment Module (AAM) then re-ranks the candidate set by perceptual alignment.

Component	Parsed output
Query	“A safe , quiet place with lots of greenery where people can read outdoors .”
Physical evidence terms	greenery (must; explicit; vegetation); bench / public seating (must; inferred from activity; affordance).
Inference rule	Reading outdoors implies a sittable affordance; benches or public seating provide visible street-view evidence for outdoor reading.
Affective terms	safe (positive; explicit; safety); quiet (positive; explicit; quietness).

Table 1. Example intent decomposition for query AO2. The parser converts an activity-oriented request into decomposed physical and affective intent terms.

Model	Liv. $\uparrow$	Beauty $\uparrow$	Boring $\uparrow$	Depr. $\uparrow$	Safe $\uparrow$	Wealth $\uparrow$	Avg. $\uparrow$
EVA-CLIP L/14	55.0	51.2	45.8	52.0	51.8	47.8	50.6
SigLIP SO400M	55.1	56.3	46.9	48.3	55.9	56.1	53.1
OpenCLIP B/32	57.6	53.0	45.9	47.2	55.6	56.3	52.6
OpenCLIP L/14	55.6	55.7	49.2	53.9	56.3	52.9	52.8
OCLIP L/14 (ours)	64.6	68.6	60.7	65.9	65.4	68.6	65.7

Table 2. Pairwise win rates (%) across six perceptual dimensions. Best per column in bold .

ID	Category	Query Text
AO1	Activity-oriented	I want to find a walkable street surrounded by vintage-style historic buildings.
AO2	Activity-oriented	A safe and quiet place with greenery where people can read outdoors.
AO3	Activity-oriented	A tree-lined shaded path suitable for jogging.
MI1	Mixed-intent	I’m looking for a romantic public square with a visible statue or sculpture.
MI2	Mixed-intent	I’m looking for a lively waterside place in the city.
MI3	Mixed-intent	I want to find a modern-looking urban place with tall buildings around.

Table 3. The ten test queries used in the end-to-end evaluation, grouped by query type.

Label dimension	Fleiss’ $\kappa$	Unanimous agreement (%)
Overall match	0.522	51.7
Physical match	0.631	64.1
Affective match	0.430	44.4

Table 4. Inter-annotator agreement on 903 candidate–query pairs (

n=5

annotators). Binary labels treat scores

\leq 2

as non-match and

\geq 3

as match.

	Mean match (1–4) $\uparrow$			Precision@ $k$ (%) $\uparrow$			nDCG@ $k$ $\uparrow$
Method	@5	@10	@20	@5	@10	@20	@5	@10	@20
CLIP	2.43	2.49	2.47	46.0	51.0	50.5	0.650	0.668	0.695
FT-CLIP	2.75	2.81	2.61	62.0	64.0	56.0	0.774	0.743	0.736
SigLIP	2.92	2.78	2.79	72.0	65.0	66.0	0.825	0.825	0.816
VQA (Qwen3)	2.93	2.81	2.71	74.0	66.0	61.5	0.884	0.867	0.830
PlaceSeek (ours)	3.39	3.36	3.37	88.0	89.0	89.5	0.920	0.917	0.910

Table 5. End-to-end retrieval performance on 10 test queries (mean over all queries). Best result per column in bold .

Figure 3. Cumulative precision ( $\uparrow$ ) curves averaged over all 10 queries. PlaceSeek maintains the highest precision across the top-20 ranked results, while baselines degrade more rapidly.

Method	Overall	Physical	Affective
CLIP	2.43	2.72	2.46
FT-CLIP	2.75	2.92	2.87
SigLIP	2.92	3.34	2.97
VQA (Qwen3)	2.93	3.09	2.93
PlaceSeek (ours)	3.39	3.63	3.41

Table 6. Top-5 mean match scores decomposed into overall, physical, and affective dimensions (1–4 scale). Best result per column in bold .

	@5		@10		@20
Method	Prec. $\uparrow$	nDCG $\uparrow$	Prec. $\uparrow$	nDCG $\uparrow$	Prec. $\uparrow$	nDCG $\uparrow$
w/o SGM	38.0	0.466	43.0	0.507	46.5	0.591
w/o AAM	88.0	0.914	83.0	0.906	74.5	0.894
Full PlaceSeek	88.0	0.920	89.0	0.917	89.5	0.910

Table 7. Ablation study. Best result per column in bold .

Method	P@5 (%) $\uparrow$	P@10 (%) $\uparrow$	P@20 (%) $\uparrow$
CLIP only	73.3	76.7	72.8
CLIP + GroundingDINO	80.0	80.0	77.8
CLIP+GroundingDINO+MLLM	88.9	87.8	87.2

Table 8. Retrieval precision of SGM stages, averaged over nine urban element types. Best results per column in bold .

Figure 4. Per-query Precision@5 ( $\uparrow$ ) heatmap for the five main methods across all 10 test queries. Darker color indicates higher Precision@5. PlaceSeek (bottom row) achieves perfect precision on six tasks and remains the most stable method overall.

Figure 5. Geolocated top-5 retrieval results on the Milan map for three queries (MI2, MI3, AO1). Retrieved locations show spatial coherence with Milan’s urban structure, including waterside, modern-area, and historic-street patterns. An interactive version covering all query tasks is available at https://placeseek-map-production-51f3.up.railway.app .

Figure 6. Top-5 retrieval results for query AO2 (a safe, quiet place with lots of greenery where people can read outdoors.).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Vis. ID	Query intent	Physical prompt-bank families	Affective prompt-bank families
AO1	Walkable historic street	Sidewalk / pedestrian pavement (must); heritage or vintage building facade (must); low-traffic street (less-better); ornate architectural details.	Walkable / comfortable walking environment; vintage-style and historic atmosphere; charming, beautiful, and pleasant street character.
AO2	Safe quiet green reading place	Greenery (must); bench / public seating (must; inferred from reading outdoors).	Safe public place; quiet outdoor place; calm and peaceful reading atmosphere.
AO3	Tree-lined shaded jogging path	Tree canopy (must); shaded path; wide pavement / broad jogging path (must).	Comfortable jogging route; pleasant shaded walkway; pleasant outdoor setting.
MI1	Romantic square with statue or sculpture	Statue / sculpture (must); open paved square or public plaza.	Romantic public square; beautiful / scenic plaza; inviting and pleasant outdoor atmosphere.
MI2	Lively waterside place	Waterfront / river / canal / water edge (must); outdoor crowd or people gathering; waterside promenade or boardwalk.	Lively, energetic, vibrant waterside atmosphere; pleasant promenade or gathering place.
MI3	Modern place with tall buildings	Skyscrapers / high-rise buildings (must); modern skyline.	Modern streetscape; modern architecture; contemporary urban design and modern cityscape.

Appendix

Table 9. Summary of generated prompt-bank outputs for all evaluation queries. AO2 is shown in detail in Table 1 .

Panel of Figure 7 — Figure 7. Per-query top-5 component scores by method. Left: mean physical match score. Right: mean affective match score.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 470 papers

May 11, 2026

Much of Geospatial Web Search Is Beyond Traditional GIS

Web search queries concern place far more often than existing labelling schemes suggest, yet the landscape of geospatial web search queries - what people ask of place, and how often - remains poorly characterised at scale. We apply dense sentence embeddings, a lightweight SetFit classifier, and density-based clustering to the full MS MARCO corpus of 1.01 million real Bing queries without prior filtering for toponyms or spatial keywords, identifying 181,827 geospatial queries (18.0%), nearly threefold the 6.17% labelled as Location in the original annotations. The resulting taxonomy of 88 query categories reveals that geospatial web search is dominated by transactional and practical lookups: costs and prices alone account for 15.3% of geospatial queries, nearly twice the size of the entire physical geography theme. Much of this activity - costs, opening hours, contact details, weather, travel recommendations - falls outside the scope of what traditional GIS and knowledge graphs are built to serve. The categories vary substantially in the kind of answer they admit, from deterministic lookups answerable from spatial databases or knowledge graphs to evaluative or temporally volatile queries that require generative or real-time systems. We discuss implications for hybrid retrieval architectures and for benchmarks of geographic reasoning in large language models. We openly release the labelled dataset, classifier, and taxonomy.

Ilya Ilyankou, Stefano Cavazzi, James Haworth

SpaceTimeLab, Department of Civil, Environmental, and Geomatic Engineering, UCL, London, UK · Ordnance Survey, Southampton, UK

Information Retrieval

Ask PDF

Apr 13, 2026

Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steering

CLIP retrieval is typically framed as a pointwise similarity problem in a shared embedding space. While CLIP achieves strong global cross-modal alignment, many retrieval failures arise from local geometric inconsistencies: nearby items are incorrectly ordered, leading to systematic confusions (e.g., pentagon vs. hexagon) and produces diffuse, weakly controlled result sets. Prior work largely optimizes for point wise relevance or finetuning to mitigate these problems. We instead view retrieval as a problem of neighborhood alignment. Our work introduces (1) neighborhood-level re-ranking via Hungarian matching, which rewards structural consistency; (2) query-conditioned local steering, where directions derived from contrastive neighborhoods around the query reshape retrieval. We show that these techniques improve retrieval performance on attribute-binding and compositional retrieval tasks. Together, these methods operate on local neighborhoods but serve different roles: re-ranking rewards alignment whereas local steering controls neighborhood structure. This shows that retrieval quality and controllability depend critically on local structure, which can be exploited at inference time without retraining.

Nirmalendu Prakash, Narmeen Fatimah Oozeer, Xin Su +8

Singapore University of Technology and Design · Martian · Thoughtworks +2

Cross-Modal Retrieval Cross-Modal Alignment Multimodal Reranking Composed Image Retrieval

Ask PDF

Jun 2, 2026

Pinpoint: Grounded Worldwide Image Geolocation via Cross-Source Retrieval and Reranking

Image geolocation aims to estimate where a photograph was taken from its visual content. At worldwide scale, this remains challenging because visual evidence is often ambiguous, diverse, and unevenly distributed. Prior work has typically treated geolocation of ordinary internet photos and street-view imagery as separate tasks, despite their complementary strengths: internet photos better match the appearance distribution of user-captured queries, while street-view imagery provides denser, geographically grounded coverage. We present Pinpoint, a retrieve-and-rerank architecture that combines both sources in a coarse-to-fine pipeline. A contrastive image-GPS embedder is trained on both user-uploaded Flickr photos and street-view imagery, learning a shared image-GPS embedding space that is used to retrieve candidate locations. An attention-based reranker then rescores retrieved candidates by combining candidate-level visual and GPS features with cross-source evidence from nearby locations to ground the prediction. Unlike recent prior work, Pinpoint does not rely on multimodal large-language models, making inference faster and more reproducible. Pinpoint achieves state-of-the-art results across all metrics on standard benchmarks for internet photos (IM2GPS3k and YFCC4k) and street-view imagery (OSV-5M).

Nika Chuzhoy, Brian Hu, Amit A. Arora +2

Virtualitics

Cross-View Geo-Localization Geospatial Representation Learning Visual Place Recognition Image Retrieval

Ask PDF

Abstract

Information Retrieval

Explore similar work

Much of Geospatial Web Search Is Beyond Traditional GIS

Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steering

Pinpoint: Grounded Worldwide Image Geolocation via Cross-Source Retrieval and Reranking