InsTraj: Instructing Diffusion Models with Travel Intentions to Generate Real-world Trajectories

Yuanshao Zhu; Yuxuan Liang; Xiangyu Zhao; Liang Han; Xinwei Fang; Xun Zhou; Xuetao Wei; James Jianqiao Yu

cs.AIApr 5, 2026

InsTraj: Instructing Diffusion Models with Travel Intentions to Generate Real-world Trajectories

Authors: Yuanshao Zhu, Yuxuan Liang, Xiangyu Zhao, Liang Han, Xinwei Fang, Xun Zhou, Xuetao Wei, James Jianqiao Yu

Organizations: Southern University of Science and Technology · City University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou) · Harbin Institute of Technology (Shenzhen) ·

Abstract

The generation of realistic and controllable GPS trajectories is a fundamental task for applications in urban planning, mobility simulation, and privacy-preserving data sharing. However, existing methods face a two-fold challenge: they lack the deep semantic understanding to interpret complex user travel intent, and struggle to handle complex constraints while maintaining the realistic diversity inherent in human behavior. To resolve this, we introduce InsTraj, a novel framework that instructs diffusion models to generate high-fidelity trajectories directly from natural language descriptions. Specifically, InsTraj first utilizes a powerful large language model to decipher unstructured travel intentions formed in natural language, thereby creating rich semantic blueprints and bridging the representation gap between intentions and trajectories. Subsequently, we proposed a multimodal trajectory diffusion transformer that can integrate semantic guidance to generate high-fidelity and instruction-faithful trajectories that adhere to fine-grained user intent. Comprehensive experiments on real-world datasets demonstrate that InsTraj significantly outperforms state-of-the-art methods in generating trajectories that are realistic, diverse, and semantically faithful to the input instructions.

Figures & tables

Figure 1. Comparison of the proposed InsTraj method with existing trajectory generation or navigation methods. The InsTraj can effectively generate trajectories that are faithful to human travel intentions and exhibit diversity.

Figure 2. Overview of the InsTraj framework. It distills travel intentions from trajectory data, encodes them via LLM into semantic blueprints, and employs MT-DiT with Joint Attention to generate instruction-faithful trajectories.

Dataset	Method	Density $\downarrow$	Trip $\downarrow$	Length $\downarrow$
Chengdu	VAE	0.0139	0.0502	0.0368
	TrajGAN	0.0137	0.0488	0.0329
	DP-TrajGAN	0.0127	0.0438	0.0234
	Diffwave	0.0145	0.0253	0.0315
	DiffTraj	0.0055	0.0154	0.0169
	ControlTraj	0.0039	0.0106	0.0117

Table 1. Quantitative comparison of spatio-temporal fidelity.

Panel of Figure 3 — Figure 3. Visualization of the geographical distribution and attributes of generated trajectories.

Dataset	Method	FCR(%) $\uparrow$		TIS $\uparrow$
Dataset	Method	S	D	TIS $\uparrow$
Chengdu	DiffTraj	75.14	74.81	0.351
	ControlTraj	75.54	74.61	0.350
	InsTraj (Bert)	75.40	75.62	0.348
	InsTraj (DiT)	82.83	81.37	0.369
	InsTraj	83.03	82.54	0.371

Table 2. Analysis of instruction faithfulness and the contribution of key components.

Panel of Figure 4 — Figure 4. Performance and efficiency analysis.

Panel of Figure 5 — Figure 5. Visualizing spatio-temporal mobility patterns discovered by InsTraj under various instructions guided.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Functional Category	Example OSM Tags
Residential	building=residential, landuse=residential
Commercial	shop=*, building=retail
Food	amenity=restaurant, amenity=cafe
Transportation	highway=bus stop, railway=station
Education	building=school, amenity=university
Recreational	leisure=stadium, leisure=playground

Appendix

Table 3. Mapping schema from OSM tags to functional categories.

Parameter	Setting value	Refer range
Input Length	200	120 $\sim$ 256
Patch size	4	1 $\sim$ 8
Diffusion Steps	500	300 $\sim$ 500
Sampling steps	100	50 $\sim$ 500
$\beta$ (linear schedule)	0.0001 $\sim$ 0.02	–
Embedding dim	384	$\geq$ 128

Appendix

Table 4. General hyperparameters setting for InsTraj.

Dataset	Trajetory Number	Area Number	Function Type
Chengdu	$1\,095\,463$	$50\times 50$	19
Xi’an	$1\,083\,784$	$50\times 50$	17

Appendix

Table 5. Statistics of two trajectory datasets.

Panel of Figure 6 — Figure 6. Origin trajectory and function profile of Chengdu.

Panel of Figure 7 — Figure 7. Origin trajectory and function profile of Xi’an.

Panel of Figure 8 — Figure 8. Generated trajectory comparison of Chengdu.

Panel of Figure 9 — Figure 9. Generated trajectory comparison of Xi’an.

Data Ratio	Density $\downarrow$	Trip $\downarrow$	Length $\downarrow$
5%	0.0268 / 0.0214	0.0395 / 0.0258	0.0358 / 0.0198
10%	0.0201 / 0.0185	0.0283 / 0.0244	0.0337 / 0.0182
50%	0.0115 / 0.0097	0.0145 / 0.0131	0.0209 / 0.0165
100%	0.0093	0.0110	0.0131

Appendix

Table 6. Cross-city transfer performance (scratch/transfer).

Instruction Type	FCR-S $\uparrow$	FCR-D $\uparrow$	TIS $\uparrow$
Template-only	83.35	82.68	0.369
Template (Synonym replaced)	80.18	79.42	0.327
LLM-refined (Ours)	83.03	82.54	0.371

Appendix

Table 7. Impact of instruction quality on faithfulness.

Instruction Type	FCR-S $\uparrow$	FCR-D $\uparrow$	TIS $\uparrow$
Coarse-grained	–	68.76	0.254
Medium-grained	75.32	73.92	0.336
Fine-grained	83.03	82.54	0.371

Appendix

Table 8. Impact of instruction granularity on faithfulness.

Dataset	Origin Accuracy	Destination Accuracy
Chengdu	98.3	96.7
Xi’an	99.2	97.1

Appendix

Table 9. Quantitative accuracy of OD constraint satisfaction.

Trajectory Generation

Trajectory generation creates plausible, goal-directed paths and motion sequences for people, vehicles, and robots.

See all 182 papers

From GPS Points to Travel Patterns: Flexible and Semantic Trajectory Generation with LLMs
May 28, 2026Silin Zhou, Chenhao Wang, Yuntao Wen +3Trajectory Generation Intelligent Transportation Systems
University of Electronic Science and Technology of China Chengdu, China · King Abdullah University of Science and Technology Thuwal, Saudi Arabia
TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation
May 11, 2026Wilson Wongso, Lihuan Li, Arian Prabowo +4Trajectory Generation Synthetic Data Generation
University of New South Wales · Hong Kong University of Science and Technology (Guangzhou)
TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation
Jun 10, 2026Siyu Li, Toan Tran, Lingyi Zhao +2Trajectory Generation Human Behavior Simulation
Dept. of Computer Science, Emory University, Atlanta, GA, USA · Novateur Research Solutions, Ashburn, VA, USA

May 28, 2026J/K move · Enter open · S save

From GPS Points to Travel Patterns: Flexible and Semantic Trajectory Generation with LLMs

Silin Zhou, Chenhao Wang, Yuntao Wen, Shuo Shang+2

University of Electronic Science and Technology of China Chengdu, China · King Abdullah University of Science and Technology Thuwal, Saudi Arabia

Trajectory Generation Intelligent Transportation Systems Synthetic Data Generation

Open paper AskPDF arXiv

Urban trajectories play a crucial role in modeling urban dynamics and supporting various smart city applications. However, privacy concerns restrict access to large-scale and high-quality trajectory datasets. Trajectory generation provides a promising alternative by synthesizing realistic data to mitigate privacy risks. However, existing methods fail to explicitly capture travel patterns and can only generate fixed-length trajectories under a single condition. To address these limitations, we propose HTP, which Hierarchically generates Travel patterns first and then generates GPS Points by using large language models (LLMs), rather than directly generating GPS points. We first design a trajectory-specific residual quantization variational autoencoder (RQ-VAE) that quantizes micro-level GPS trajectories into compact, macro-level travel pattern tokens in a coarse-to-fine manner. These tokens capture rich segment spatial irregularities, such as point density variations caused by traffic conditions. Then, we extend the LLM vocabulary with travel pattern tokens to align trajectory representations with the LLM input, and apply supervised fine-tuning (SFT) to align the LLM with the trajectory generation task, enabling generation of travel pattern sequences under various conditions. Extensive experiments on two real-world datasets show that HTP outperforms the strongest baseline by an average of 29.78% in terms of generation quality. Our code is available at https://github.com/slzhou-xy/HTP.

Abstract

Trajectory Generation

Explore similar work

From GPS Points to Travel Patterns: Flexible and Semantic Trajectory Generation with LLMs

TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation

TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation