Online discourse about urban issues - walkability, cycling infrastructure, public transit, housing density, and street safety - is voluminous but unstructured, and existing city-evaluation tools capture none of it. We present a pipeline and conversational system that combines geographic entity resolution, topic modeling, sentiment analysis, and Retrieval-Augmented Generation (RAG) over 22,788 chunks of YouTube transcripts and comments spanning 309 North American cities. Beyond the system itself, our contribution is a set of measurements about what happens when standard NLP components meet short, informal, geographically ambiguous text. A Twitter-tuned RoBERTa classifier outperforms a VADER lexicon baseline by 12.6 macro-F1 points (0.589 vs. 0.464; McNemar p = 0.0001), but both models collapse on the neutral class, which dominates urbanist comment traffic; annotators disagree on the same class (Cohen's kappa = 0.53). Dense retrieval beats a TF-IDF baseline at every cutoff (P@5 0.790 vs. 0.560), and video-level relevance proxies understate chunk-level precision by a wide margin (0.660 vs. 0.94 under human rating). For groundedness evaluation, we find BERTScore unusable when a multi-sentence generated summary is compared against a single short comment - scores are nearly flat regardless of relevance - and show that ROUGE-1-based groundedness is a paraphrase-driven lower bound rather than a hallucination rate. These findings generalize beyond the urbanist domain to any RAG system built over short user-generated documents.
Figures & tables
Figure 1: System architecture: preprocessing (data cleaning and topic-relevance classification) followed by geographic entity resolution, topic modeling, sentiment classification, document indexing, retrieval, and response generation, with the specific method used at each stage.
Table 1: City sentiment profiles for the eight highest-coverage cities: three highest-weighted topic dimensions (normalized proportion of that city’s aggregated topic weight) and normalized sentiment share. † New York and Manhattan are resolved as distinct entities and are reported separately; ‡ “London” is matched against the US/Canada gazetteer and is not disambiguated from London, UK.
Table 3: Per-class F1 and recall on the validation set.
Method
P@5
R@5
P@10
R@10
Dense (FAISS)
0.790
0.230
0.660
0.372
TF-IDF baseline
0.560
0.183
0.545
0.302
Table 4: Retrieval Precision@ k and Recall@ k , dense (FAISS) vs. TF-IDF baseline, averaged across 20 held-out queries. Judgments are video-level.
Metric
Score
Groundedness rate (ROUGE-1 ≥ 0.15)
23.3% [10.0–35.0]
Mean / median ROUGE-1 recall
0.139 / 0.135
Mean / median cosine similarity
0.730 / 0.727
Query-side city detection accuracy
100.0% (20/20)
Feature detection accuracy
80.0% (16/20)
Table 5: Groundedness across 20 held-out queries. The groundedness rate is the mean over three generation runs, with the observed range in brackets; continuous metrics are from a single run. City accuracy is query-side city detection; feature accuracy is detection of at least one expected aspect category.
Query
ROUGE-1
Cosine
Grounded
What do people say about road safety in Charlotte?
0.206
0.777
✓
What is housing like in Montreal?
0.172
0.716
✓
Are there good bike lanes in Seattle?
0.169
0.778
✓
How dense is housing in Montreal?
0.166
0.722
✓
How do people feel about car dependency in Houston?
0.155
0.721
✓
How is cycling infrastructure in Portland?
0.153
0.751
✓
Table 6: Per-query ROUGE-1 recall and sentence-embedding cosine similarity, from the highest-scoring of three generation runs. ✓ marks queries meeting the ROUGE-1 threshold of 0.15.
Modern cities rely on an increasing number of digital services to operate, but residents' daily needs are still difficult to meet. Services are fragmented and have little interoperability, placing a heavy operational burden on users. Existing digital platforms, urban foundation models, and intelligent assistants each address only isolated aspects of an urban task. But they struggle to reliably convert complex natural-language requests into executable cross-system workflows. We propose Urban-Agent, a tool-augmented agent framework for cross-system urban tasks. It couples the cognitive and reasoning capabilities of a large language model with a tool-set supporting code execution, API calls, and Model Context Protocol. Through one adaptive closed loop, it clarifies missing information before acting, grounds tool use in live observations, and aligns the final response with observed evidence and task constraints. To address the evaluation gap, we introduce Urban-Eval, a benchmark specifically designed for cross-system urban request. Unlike prior benchmarks that assess either general tool use or urban knowledge and reasoning, Urban-Eval evaluates both task results and execution quality, including required tool coverage, dependency validity, and evidence traceability. Experimental results indicate that Urban-Agent reaches a 71% task success rate, 10 points above the strongest baseline. This lead holds across GPT-5-mini, Gemini-2.5-flash, DeepSeek-V4-flash, and Qwen3-235B-A22B.
People search for urban outdoor places not only by category or function, but also by what activities a place can support and how it is perceived. Existing geospatial retrieval remains largely POIcentric and metadata-driven, making it difficult to satisfy openended, affective, or activity-oriented needs. We present PlaceSeek, a human-centered outdoor place retrieval framework that maps natural-language queries to geolocated street-view imagery. PlaceSeek introduces an intent-aware retrieval mechanism that decomposes user queries into functional and affective sub-intents. A Semantic Grounding Module verifies whether candidate street-view results contain the physical evidence needed to support the intended activity, while an Affective Alignment Module re-ranks physically valid candidates using a LoRA-adapted vision-language model trained on human urban perception judgments. We evaluate PlaceSeek on 31,956 street-view locations in Milan across 10 naturallanguage queries annotated by five human evaluators. PlaceSeek achieves 88.0% Precision@5, a mean match score of 3.39/4.0, and 0.920 nDCG@5, outperforming CLIP, fine-tuned CLIP, SigLIP, and a VQA-based baseline. Ablation results show that physical grounding is essential for retrieval validity, while affective alignment improves ranking quality among physically valid candidates. These findings highlight that complex urban spatial queries require modeling both verifiable visual evidence and human perceptual preferences. PlaceSeek provides a potential framework for human-centered nextgeneration geospatial retrieval systems.
Ziqi Cui, Shangyu Lou
University of California, Santa Barbara & San Diego State University California, USA
Urban mobility is naturally expressed both as trajectories in space and as natural-language descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry-centric, while language-centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travel-intent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input-output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources.