cs.AISep 29, 2026

SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling

Authors: Yunan Lu, Shuang Xie, Meghna Allamudi, Mingyu Zhao, Han Li, Lingyun Wang, Zhou Yu

Organizations: Shopify · Columbia University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Virtual clients offer a cost-effective approach to support applications such as A/B testing, recommender system development, and interface evaluation. However, building them requires access to large-scale, semantically faithful, fine-grained online user trajectories. These data are difficult to obtain because proprietary logs are subject to privacy restrictions and small businesses often lack sufficient traffic. Consequently, existing public datasets either abstract away fine-grained user interaction details or preserve rich context but remain platform-specific and small-scale. To address this gap, we propose SimTrace, a framework that generates faithful, fine-grained synthetic multimodal clickstreams through a computer-use client agent that is grounded in real user trajectories and the given web environment. SimTrace anonymizes real interactions and constructs a simulated twin of the given web environment, then uses both to generate synthetic interaction trajectories. Each action is paired with its corresponding web observations and user context, yielding a shareable alternative to confidential logs for developing computer-use agent-style virtual clients. We apply SimTrace to an e-commerce setting and evaluate both its fidelity and downstream utility. SimTrace outperforms competing baselines on 7 out of 8 fidelity metrics. Models trained on synthetic data achieve performance comparable to those trained on real data on downstream tasks such as purchase prediction and recommendation. For next action prediction task, augmenting real data with synthetic data further improves accuracy by 11.0% relative to training on real data alone. We release SimTrace as an open-source package to facilitate research on online user behavior modeling.

Figures & tables

Figure 1: Overview of SimTrace and the resulting dataset. SimTrace transforms real activity logs into representative, anonymized behavior trajectories, constructs a twin sandbox environment of the given website, and uses LLM-based teacher agent to generate synthetic interaction sessions. Each resulting session contains user context and a sequence of timestamped steps, where each step pairs a structured action record with the corresponding webpage HTML and screenshot. See Appendix C.2 for a complete data example.
CategorySemantic ActionDefinition
 searchThe user enters a query in the search bar.
Domain-detailThe user visits a product page or interacts with product attributes.
specificaddThe user adds a product to the cart.
ActionsremoveThe user removes a product from the cart.
 checkoutThe user proceeds to purchase using Buy Now or Checkout .
 gotoThe user navigates to a new non-product-detail page, e.g. applying a filter, changing the sort order, or following a navigation link.
Table 1: Taxonomy for transforming raw clickstream events in e-commerce domain.
Table 3
 Outcome-levelSequence-levelSemantic-level
MethodOutcome JSD ↓\downarrow [0,1][0,1]Action Freq. JSD ↓\downarrow [0,1][0,1]Transition Matrix L1\bm{L_{1}} Distance ↓\downarrow [0,1][0,1]Trajectory Levenshtein ↓\downarrow [0,1][0,1]Product Coherence Gap ↓\downarrow [0,1][0,1]Product Diversity Ratio ↓\downarrow [0,∞)[0,\infty)Product Alignment ↑\uparrow [−1,1][-1,1]Rationale Alignment ↑\uparrow [−1,1][-1,1]
Prompting-based methods
Persona2.94×10−12.94{\times}10^{-1}5.40×10−25.40{\times}10^{-2}4.30×10−14.30{\times}10^{-1}6.74×10−16.74{\times}10^{-1}3.92×10−23.92{\times}10^{-2}2.84×10−12.84{\times}10^{-1}5.75×10−15.75{\times}10^{-1}6.10×10−16.10{\times}10^{-1}
Trajectory8.56×10−38.56{\times}10^{-3}1.10×10−21.10{\times}10^{-2}2.34×10−12.34{\times}10^{-1}2.18×10−12.18{\times}10^{-1}6.86×10−26.86{\times}10^{-2}7.74×10−2\mathbf{7.74{\times}10^{-2}}5.32×10−15.32{\times}10^{-1}6.52×10−16.52{\times}10^{-1}
Trajectory + Persona3.36×10−33.36{\times}10^{-3}9.73×10−39.73{\times}10^{-3}2.39×10−12.39{\times}10^{-1}2.85×10−12.85{\times}10^{-1}6.85×10−26.85{\times}10^{-2}9.15×10−29.15{\times}10^{-2}5.43×10−15.43{\times}10^{-1}6.54×10−16.54{\times}10^{-1}
Agentic methods
Table 4: Fidelity comparison between different generation methods on gemini-3-flash model. The best results are shown in bold . Statistically significant improvements ( p<0.05p<0.05 ) over all other baselines are marked with ∗ .
MethodMetricTRTRTSTRΔ\Delta95% CI
Purchase prediction
SFT (Qwen3.5-4B)F1 ↑\uparrow64.3665.52+1.16+1.16[−3.00, +5.00][-3.00,\,+5.00]
Session-based recommendation
RAIN (Graph-based)MRR@5 ↑\uparrow12.4712.58+0.11+0.11[−1.59, +1.70][-1.59,\,+1.70]
HR@5 ↑\uparrow21.2520.85−0.40-0.40[−3.20, +2.27][-3.20,\,+2.27]
MRR@10 ↑\uparrow13.2013.64+0.44+0.44[−1.24, +2.03][-1.24,\,+2.03]
Table 5: Downstream performance of models trained on real (TRTR) and synthetic (TSTR) sessions and evaluated on the same held-out real test set. Δ\Delta is TSTR-TRTR. All 95% CIs for Δ\Delta include zero, indicating no statistically significant differences between two training conditions.
Figure 2: Next action prediction performance of Qwen3.5-9B on the OPeRA test set. Augmenting real with synthetic data improves all metrics, with RL achieving the best overall performance.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

 Outcome-levelSequence-levelSemantic-level
MethodOutcome JSD ↓\downarrow [0,1][0,1]Action Freq. JSD ↓\downarrow [0,1][0,1]Transition Matrix L1\bm{L_{1}} Distance ↓\downarrow [0,1][0,1]Trajectory Levenshtein ↓\downarrow [0,1][0,1]Product Coherence Gap ↓\downarrow [0,1][0,1]Product Diversity Ratio ↓\downarrow [0,∞)[0,\infty)Product Alignment ↑\uparrow [−1,1][-1,1]Rationale Alignment ↑\uparrow [−1,1][-1,1]
Prompting-based methods
Persona1.94×10−11.94{\times}10^{-1}4.39×10−24.39{\times}10^{-2}3.82×10−13.82{\times}10^{-1}6.08×10−16.08{\times}10^{-1}6.06×10−26.06{\times}10^{-2}1.03×10−11.03{\times}10^{-1}5.97×10−15.97{\times}10^{-1}6.47×10−1\mathbf{6.47{\times}10^{-1}}
Trajectory3.98×10−33.98{\times}10^{-3}1.65×10−21.65{\times}10^{-2}2.28×10−12.28{\times}10^{-1}2.59×10−12.59{\times}10^{-1}7.80×10−27.80{\times}10^{-2}6.45×10−26.45{\times}10^{-2}5.41×10−15.41{\times}10^{-1}6.38×10−16.38{\times}10^{-1}
Trajectory + Persona1.20×10−31.20{\times}10^{-3}2.03×10−22.03{\times}10^{-2}2.38×10−12.38{\times}10^{-1}2.80×10−12.80{\times}10^{-1}8.00×10−28.00{\times}10^{-2}7.14×10−27.14{\times}10^{-2}5.89×10−15.89{\times}10^{-1}6.34×10−16.34{\times}10^{-1}
Agentic methods
Appendix
Table 6: Fidelity comparison between different generation methods on the gpt-5.6-sol model. The best results are shown in bold . Statistically significant improvements ( p<0.05p<0.05 ) over all other baselines are marked with ∗ .

Explore similar work

CardsList
Sep 23, 2026cs.AI

TRACER: Trajectory-Aligned Learning for Multi-Turn User Simulation

Faithful user simulation is fundamental to building, evaluating, and improving interactive AI at scale. Yet current simulators often produce plausible individual responses without reproducing the intent evolution and outcomes observed in real interactions. We propose TRACER, a multi-turn user simulator that models evolving user intent and aligns simulated trajectories with real ones. TRACER is trained in two stages: supervised fine-tuning on real user dialogues, followed by multi-turn reinforcement learning. The RL stage combines hierarchical outcome- and trajectory-level rewards with deviation-aware advantage modulation, jointly addressing reward sparsity and credit assignment challenges in long dialogues. On real customer-service sessions organized into reference cohorts, TRACER-7B surpasses the strongest baseline by 11.4 conversion F1 points, while outperforming all baselines on group-level conversion-rate error and semantic trajectory distance and generalizing to out-of-distribution scenarios. In human Turing tests, annotators identified TRACER conversations at near-chance accuracy. Building on this simulator, we further introduce the Dynamic Marketing Benchmark, which jointly evaluates persuasion and response quality via simulated interactions, revealing that higher response quality does not necessarily correspond to higher conversion rates.

Geng Chen, Ruotong Pan, Zhirui Yang +9

Peking University · Kuaishou Technology · Xi’an Jiaotong University +3

User SimulationMulti-Turn InteractionsPersuasionMulti-Turn Reinforcement LearningStyle
AskPDF
Jun 10, 2026cs.AI

TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation

Human mobility data is important for transportation, urban planning, and epidemic control, but large-scale trajectory collection is often costly and privacy-constrained, motivating realistic synthetic trajectory generation. Existing LLM-based generators typically rely on either prompt engineering, which preserves zero-shot reasoning but lacks fine-grained spatiotemporal grounding, or trajectory-level fine-tuning, which improves statistical precision but incurs substantial computational cost and may weaken general reasoning. We propose TrajGenAgent, a semantic-aware hierarchical LLM-agent framework for human mobility trajectory generation without model fine-tuning. TrajGenAgent uses a two-stage orchestrator-worker design: an LLM first synthesizes an individual- and weekday-conditioned activity chain from historical evidence via in-context learning, and a deterministic workflow then grounds each activity into a complete visit using personalized POI retrieval, distance-aware location selection, kinematics-aware travel-time propagation, and LLM-based duration estimation. To evaluate realism beyond aggregate spatiotemporal statistics, we introduce an anomaly-detection-based evaluation framework using two complementary detectors to assess behavioral and semantic plausibility. Experiments on benchmark and large-scale simulation datasets show that TrajGenAgent improves spatiotemporal fidelity, semantic coherence, and individual-specific behavioral realism over representative neural and LLM-based baselines, while avoiding parameter updates.

Siyu Li, Toan Tran, Lingyi Zhao +2

Dept. of Computer Science, Emory University, Atlanta, GA, USA · Novateur Research Solutions, Ashburn, VA, USA

MobilityTracciaHuman Motion GenerationLarge Language Model Agents
AskPDF
Jul 1, 2026cs.HC

SenseWalk: Agent-Based Semantic Trajectory Simulation Powered by Large Language Models in Zoned Environments

Semantic trajectory analysis has recently emerged as an approach for modeling human movement by capturing implicit patterns and behaviors through semantic information (e.g., visitors' profiles and goals) beyond raw spatial paths to better understand why people move in certain ways. However, analyzing semantic trajectories in real-world scenarios remains challenging, as collecting high-quality data is costly and often lacks rich semantic information. Meanwhile, existing simulation tools require substantial technical expertise, which makes them difficult for practitioners to adopt. To address these limitations, the paper proposes SenseWalk{SenseWalk}, an interactive system that supports simulating semantic trajectories by LLM-powered agents. We develop a simulation workflow that combines LLMs and the social force model to balance physical plausibility and semantic coherence. A user-friendly interface is designed to facilitate users in customizing the simulation configuration and analyzing simulation outputs. We also conduct a quantitative experiment to evaluate the effectiveness of our simulation workflow, and a user study (n=12) to assess the usefulness and efficiency of our system.

Ziyue Lin, Xinhang Xie, Kangyi Wang +1

School of Data Science, Fudan University · Shanghai, China

Microscopic Traffic SimulationsSocial DynamicsDynamic Environments
AskPDF