STRIDE: Automated Evaluation of Text-to-Trajectory Alignment across Diverse Contexts

Wanchun Ni; Tao Qi; Leonel Aguilar; Jiugeng Sun; Marlene Wagner; Verena Zimmermann; Mennatallah El-Assady

cs.AISep 28, 2026

STRIDE: Automated Evaluation of Text-to-Trajectory Alignment across Diverse Contexts

Authors: Wanchun Ni, Tao Qi, Leonel Aguilar, Jiugeng Sun, Marlene Wagner, Verena Zimmermann, Mennatallah El-Assady

Organizations: ETH Zurich · Beijing University of Posts and Telecommunications

Abstract

Language-conditioned trajectory generation is here, but its evaluation has not kept pace. Existing pedestrian trajectory metrics compare trajectories with real-world human data. This does not scale to text-to-trajectory generation across diverse contexts, as collecting human trajectories for every scenario is costly and infeasible. Moreover, pedestrian behavior is heterogeneous and context-dependent, with no single metric as the correct answer, and current evaluation frameworks are not transferable to this domain. These challenges make scalable, reliable evaluation difficult. We introduce STRIDE, the first framework for evaluating context alignment between scenario descriptions and pedestrian trajectories. STRIDE addresses these challenges through three design choices. First, we derive our VRDST evaluation protocol from sociological theories to define a complete evaluation space. Second, it decomposes high-level context into scenario-adaptive behavioral questions. Third, every question is resolved against a deterministic measurement tool library that yields reproducible answers. Together, STRIDE enables complete, verifiable, automated, and scalable evaluation across diverse contexts without requiring human trajectory data. We instantiate STRIDE in the crowd domain as STRIDE-Bench, comprising 1K scenarios, 6K behavioral questions, and 11K measurements with calibrated expected answers across 30 real-world maps. Comprehensive human validations show that STRIDE-Bench is consistent with human behavior and judgment, achieving 80% human agreement. We further evaluate several text-to-trajectory models, finding limited context-alignment capability and persistent challenges in fine-grained context conditioning. We believe that the STRIDE framework provides a first step toward principled evaluation of context-aligned pedestrian trajectory generation.

Figures & tables

Figure 1: Overview of the Stride evaluation framework. Stage 1 constructs Stride-Bench by decomposing each scenario into behavioral questions, measurement functions, and expected answers. Stage 2 computes the Stride score by comparing function outputs against the expected answers.

Figure 2: Scenario corpus map processing: (a) real-world map from Google Map. (b) walkable areas defined by Google Map color set. (c) polygon-approximated obstacles. (d) metadata on the map.

Figure 3: DMT function usage across crowd categories in Stride-Bench . Each cell reports the percentage of scenes within a category that use the corresponding function.

Figure 4: Stride-Bench validation (ann. means annotators). (a) Human validation. Annotator-benchmark agreement averages 80% with Cohen’s $\kappa=0.730$ ; inter-annotator agreement of 66% with Krippendorff’s $\alpha=0.698$ , indicating Stride-Bench answers are human-consistent. (b) deepseek-v3, claude-sonnet-4.6, qwen3.6-plus, and gpt-5.5 achieve similar agreement and Cohen’s $\kappa$ with the benchmark, comparable to human annotators, showing robustness across models.

Figure 5: Trajectories for Fête des Lumières in Lyon; Large View Trackers . Stride scores are (a) Human, $0.943$ ; (b) Text-Crowd, $0.411$ ; (c) LLM-SFM, $0.584$ ; (d) Random Walk, $0.317$ .

Model	Agg.	Amb.	Coh.	Dem.	Den.	Dis.	Esc.	Exp.	Par.	Rus.	Vio.	$\mathrm{STRIDE}_{\mathrm{Bench}}\uparrow$
Text-Crowd	$\mathbf{0.669}$	$\mathbf{0.787}$	$\mathbf{0.733}$	$\mathbf{0.771}$	$\mathbf{0.729}$	$\mathbf{0.778}$	$\mathbf{0.393}$	$\mathbf{0.829}$	$\mathbf{0.790}$	$\mathbf{0.584}$	$\mathbf{0.387}$	$\mathbf{0.645\pm 0.233}$
LLM-SFM	$0.488$	$0.327$	$0.418$	$0.605$	$0.650$	$0.489$	$0.256$	$0.610$	$0.568$	$0.243$	$0.230$	$0.434\pm 0.231$
SingularTrajectory	$0.425$	$0.452$	$0.526$	$0.540$	$0.530$	$0.505$	$0.238$	$0.536$	$0.509$	$0.358$	$0.229$	$0.421\pm 0.177$
Random Walk	$0.355$	$0.409$	$0.348$	$0.312$	$0.273$	$0.368$	$0.268$	$0.340$	$0.312$	$0.312$	$0.254$	$0.311\pm 0.117$
Stop	$0.259$	$0.260$	$0.249$	$0.230$	$0.194$	$0.307$	$0.178$	$0.237$	$0.246$	$0.195$	$0.185$	$0.221\pm 0.105$

Table 1: Per-category and overall STRIDE-Bench scores, where higher is better (

\mathrm{STRIDE}_{\mathrm{Bench}}\in[0,1]

). Category names are abbreviated by their first three letters. The last column reports overall mean

\pm

standard deviation on the benchmark dataset.

Time	Model	V	R	D	S	T	Mean $\uparrow$
0–30s	LLM-SFM	$\mathbf{0.615}\pm 0.360$	$\mathbf{0.860}\pm 0.083$	$0.523\pm 0.265$	$0.603\pm 0.136$	$0.268\pm 0.191$	$\mathbf{0.574}\pm 0.190$
	SingularTraj	$0.217\pm 0.203$	$0.819\pm 0.123$	$0.379\pm 0.197$	$\mathbf{0.604}\pm 0.131$	$0.251\pm 0.211$	$0.454\pm 0.228$
	Text-Crowd	$0.531\pm 0.070$	$0.734\pm 0.078$	$\mathbf{0.677}\pm 0.182$	$0.507\pm 0.049$	$\mathbf{0.288}\pm 0.157$	$0.547\pm 0.156$
	Random	$0.504\pm 0.480$	$0.817\pm 0.126$	$0.019\pm 0.021$	$0.376\pm 0.141$	$0.093\pm 0.079$	$0.362\pm 0.289$
	Stop	$0.012\pm 0.009$	$0.471\pm 0.471$	$0.182\pm 0.233$	$0.350\pm 0.154$	$0.026\pm 0.041$	$0.208\pm 0.180$
30–120s	LLM-SFM	$\mathbf{0.806}\pm 0.124$	$\mathbf{0.914}\pm 0.027$	$0.471\pm 0.203$	$0.687\pm 0.095$	$0.283\pm 0.152$	$\mathbf{0.632}\pm 0.228$

Table 2: STRIDE scores across models, time windows, and V R D S T axes. Values are mean

\pm

standard deviation; higher is better. Mean denotes the average score among the protocols.

Size	Model	V	R	D	S	T	Mean $\uparrow$
Small (1–25)	LLM-SFM	$0.458\pm 0.085$	$0.671\pm 0.023$	$0.323\pm 0.094$	$0.491\pm 0.195$	$0.280\pm 0.301$	$0.445\pm 0.138$
	SingularTraj	$0.365\pm 0.365$	$0.736\pm 0.042$	$0.139\pm 0.123$	$0.492\pm 0.196$	$0.129\pm 0.083$	$0.372\pm 0.228$
	Text-Crowd	$\mathbf{0.734}\pm 0.013$	$\mathbf{0.915}\pm 0.072$	$\mathbf{0.632}\pm 0.068$	$\mathbf{0.546}\pm 0.204$	$\mathbf{0.362}\pm 0.271$	$\mathbf{0.638}\pm 0.185$
	Random	$0.513\pm 0.487$	$0.822\pm 0.165$	$0.064\pm 0.075$	$0.519\pm 0.139$	$0.269\pm 0.329$	$0.437\pm 0.256$
	Stop	$0.018\pm 0.018$	$0.493\pm 0.493$	$0.191\pm 0.214$	$0.531\pm 0.261$	$0.165\pm 0.345$	$0.280\pm 0.199$
Medium (26–100)	LLM-SFM	$0.483\pm 0.029$	$0.671\pm 0.031$	$0.142\pm 0.057$	$0.584\pm 0.092$	$0.321\pm 0.225$	$0.440\pm 0.189$

Table 3: STRIDE scores across models, agent sizes, and V R D S T axes. Values are mean

\pm

standard deviation; higher is better. Mean denotes the average score among the protocols.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Scene	Category	Description
0040 (target)	Escaping	An alarm triggers evacuation, and people attempt to escape the concourse, heading toward open exits and far corners.
0015	Cohesive	A tour group sticks together while crossing the concourse to catch a regional train, keeping a tight formation around the guide.
0055	Violent	A violent brawl erupts in a confined spot, causing surrounding people to flee toward distant exits.
0045	Dense	Peak-hour crowding produces packed movement in the open corridors and near platform approaches.

Appendix

Table 4: Scene descriptions used for LLM to choose from. 0040 is the target scenario description.

Dimension	DMT Name
V Velocity	mean_speed , speed_variation_coeff
R Realism	collision_fraction , lingering_fraction
D Direction	flow_alignment , path_linearity , directional_entropy
S Spatial	spatial_concentration , mean_local_density , peak_local_density , dispersal_score , convergence_score
T Temporal	speed_trend , collision_trend , lingering_trend , density_trend , spread_trend , entropy_trend , clustering_trend , flow_alignment_trend

Appendix

Table 5: Mapping from STRIDE dimensions to diagnostic metric tools.

Model	Mean	Median	Std	Min	Max
Human	0.9427	1.0000	0.0751	0.812	1.000
LLM-SFM	0.5843	0.6339	0.1423	0.214	0.714
Text-Crowd	0.4115	0.4286	0.1042	0.214	0.571
Random	0.3175	0.2887	0.0385	0.286	0.375

Appendix

Table 6: STRIDE scores on the Lyon human-festival scenes. Values are computed over 12 scenes and 86 questions; higher is better.

Function	Human	LLM-SFM	Text-Crowd	Random
collision_fraction	1.00	0.83	0.08	1.00
convergence_score	0.79	0.71	0.64	0.00
directional_entropy_normalized	1.00	0.14	0.43	0.00
dispersal_score	1.00	0.92	1.00	1.00
flow_alignment	0.94	0.12	0.24	0.00
lingering_fraction	1.00	0.67	1.00	0.92

Appendix

Table 7: Function-level pass rates on the Lyon scenes. Each value is the fraction of scenes in which the function-level check passes.

Figure 6: STRIDE Human Annotation Survey Format

Figure 7: Stride human annotation results per V R D S T dimensions

Explore similar work

TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding
May 11, 2026Lihuan Li, Wilson Wongso, Baiyu Chen +6Urban Mobility Journey
UNSW Sydney · HKUST (GZ)
CityTrajBench: A Unified Benchmark for City-Scale Vehicle Trajectory Generation
Jun 1, 2026Shibo Zhu, Xiaodan Shi, Dayin Chen +4Urban Mobility Student-Generated Trajectories
Department of Building Environment and Energy Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China · Eastern Institute for Advanced Study, Eastern Institute of Technology, Ningbo, China · International Centre of Urban Energy Nexus, The Hong Kong Polytechnic University, Hong Kong SAR, China +5
TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation
Jun 10, 2026Siyu Li, Toan Tran, Lingyi Zhao +2Mobility Traccia
Dept. of Computer Science, Emory University, Atlanta, GA, USA · Novateur Research Solutions, Ashburn, VA, USA

May 11, 2026, cs.AIJ/K move · Enter open · S save

TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding

Lihuan Li, Wilson Wongso, Baiyu Chen, Hao Xue+5

UNSW Sydney · HKUST (GZ)

Urban Mobility Journey Prism

Open paper AskPDF arXiv

Urban mobility is naturally expressed both as trajectories in space and as natural-language descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry-centric, while language-centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travel-intent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input-output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources.

Abstract

Explore similar work

TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding

CityTrajBench: A Unified Benchmark for City-Scale Vehicle Trajectory Generation

TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation