cs.AISep 29, 2026

SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling

Authors: Yunan Lu, Shuang Xie, Meghna Allamudi, Mingyu Zhao, Han Li, Lingyun Wang, Zhou Yu

Organizations: Shopify · Columbia University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Virtual clients offer a cost-effective approach to support applications such as A/B testing, recommender system development, and interface evaluation. However, building them requires access to large-scale, semantically faithful, fine-grained online user trajectories. These data are difficult to obtain because proprietary logs are subject to privacy restrictions and small businesses often lack sufficient traffic. Consequently, existing public datasets either abstract away fine-grained user interaction details or preserve rich context but remain platform-specific and small-scale. To address this gap, we propose SimTrace, a framework that generates faithful, fine-grained synthetic multimodal clickstreams through a computer-use client agent that is grounded in real user trajectories and the given web environment. SimTrace anonymizes real interactions and constructs a simulated twin of the given web environment, then uses both to generate synthetic interaction trajectories. Each action is paired with its corresponding web observations and user context, yielding a shareable alternative to confidential logs for developing computer-use agent-style virtual clients. We apply SimTrace to an e-commerce setting and evaluate both its fidelity and downstream utility. SimTrace outperforms competing baselines on 7 out of 8 fidelity metrics. Models trained on synthetic data achieve performance comparable to those trained on real data on downstream tasks such as purchase prediction and recommendation. For next action prediction task, augmenting real data with synthetic data further improves accuracy by 11.0% relative to training on real data alone. We release SimTrace as an open-source package to facilitate research on online user behavior modeling.

Figures & tables

Figure 1: Overview of SimTrace and the resulting dataset. SimTrace transforms real activity logs into representative, anonymized behavior trajectories, constructs a twin sandbox environment of the given website, and uses LLM-based teacher agent to generate synthetic interaction sessions. Each resulting session contains user context and a sequence of timestamped steps, where each step pairs a structured action record with the corresponding webpage HTML and screenshot. See Appendix C.2 for a complete data example.
CategorySemantic ActionDefinition
 searchThe user enters a query in the search bar.
Domain-detailThe user visits a product page or interacts with product attributes.
specificaddThe user adds a product to the cart.
ActionsremoveThe user removes a product from the cart.
 checkoutThe user proceeds to purchase using Buy Now or Checkout .
 gotoThe user navigates to a new non-product-detail page, e.g. applying a filter, changing the sort order, or following a navigation link.
Table 1: Taxonomy for transforming raw clickstream events in e-commerce domain.
Table 3
 Outcome-levelSequence-levelSemantic-level
MethodOutcome JSD ↓\downarrow [0,1][0,1]Action Freq. JSD ↓\downarrow [0,1][0,1]Transition Matrix L1\bm{L_{1}} Distance ↓\downarrow [0,1][0,1]Trajectory Levenshtein ↓\downarrow [0,1][0,1]Product Coherence Gap ↓\downarrow [0,1][0,1]Product Diversity Ratio ↓\downarrow [0,∞)[0,\infty)Product Alignment ↑\uparrow [−1,1][-1,1]Rationale Alignment ↑\uparrow [−1,1][-1,1]
Prompting-based methods
Persona2.94×10−12.94{\times}10^{-1}5.40×10−25.40{\times}10^{-2}4.30×10−14.30{\times}10^{-1}6.74×10−16.74{\times}10^{-1}3.92×10−23.92{\times}10^{-2}2.84×10−12.84{\times}10^{-1}5.75×10−15.75{\times}10^{-1}6.10×10−16.10{\times}10^{-1}
Trajectory8.56×10−38.56{\times}10^{-3}1.10×10−21.10{\times}10^{-2}2.34×10−12.34{\times}10^{-1}2.18×10−12.18{\times}10^{-1}6.86×10−26.86{\times}10^{-2}7.74×10−2\mathbf{7.74{\times}10^{-2}}5.32×10−15.32{\times}10^{-1}6.52×10−16.52{\times}10^{-1}
Trajectory + Persona3.36×10−33.36{\times}10^{-3}9.73×10−39.73{\times}10^{-3}2.39×10−12.39{\times}10^{-1}2.85×10−12.85{\times}10^{-1}6.85×10−26.85{\times}10^{-2}9.15×10−29.15{\times}10^{-2}5.43×10−15.43{\times}10^{-1}6.54×10−16.54{\times}10^{-1}
Agentic methods
Table 4: Fidelity comparison between different generation methods on gemini-3-flash model. The best results are shown in bold . Statistically significant improvements ( p<0.05p<0.05 ) over all other baselines are marked with ∗ .
MethodMetricTRTRTSTRΔ\Delta95% CI
Purchase prediction
SFT (Qwen3.5-4B)F1 ↑\uparrow64.3665.52+1.16+1.16[−3.00, +5.00][-3.00,\,+5.00]
Session-based recommendation
RAIN (Graph-based)MRR@5 ↑\uparrow12.4712.58+0.11+0.11[−1.59, +1.70][-1.59,\,+1.70]
HR@5 ↑\uparrow21.2520.85−0.40-0.40[−3.20, +2.27][-3.20,\,+2.27]
MRR@10 ↑\uparrow13.2013.64+0.44+0.44[−1.24, +2.03][-1.24,\,+2.03]
Table 5: Downstream performance of models trained on real (TRTR) and synthetic (TSTR) sessions and evaluated on the same held-out real test set. Δ\Delta is TSTR-TRTR. All 95% CIs for Δ\Delta include zero, indicating no statistically significant differences between two training conditions.
Figure 2: Next action prediction performance of Qwen3.5-9B on the OPeRA test set. Augmenting real with synthetic data improves all metrics, with RL achieving the best overall performance.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

 Outcome-levelSequence-levelSemantic-level
MethodOutcome JSD ↓\downarrow [0,1][0,1]Action Freq. JSD ↓\downarrow [0,1][0,1]Transition Matrix L1\bm{L_{1}} Distance ↓\downarrow [0,1][0,1]Trajectory Levenshtein ↓\downarrow [0,1][0,1]Product Coherence Gap ↓\downarrow [0,1][0,1]Product Diversity Ratio ↓\downarrow [0,∞)[0,\infty)Product Alignment ↑\uparrow [−1,1][-1,1]Rationale Alignment ↑\uparrow [−1,1][-1,1]
Prompting-based methods
Persona1.94×10−11.94{\times}10^{-1}4.39×10−24.39{\times}10^{-2}3.82×10−13.82{\times}10^{-1}6.08×10−16.08{\times}10^{-1}6.06×10−26.06{\times}10^{-2}1.03×10−11.03{\times}10^{-1}5.97×10−15.97{\times}10^{-1}6.47×10−1\mathbf{6.47{\times}10^{-1}}
Trajectory3.98×10−33.98{\times}10^{-3}1.65×10−21.65{\times}10^{-2}2.28×10−12.28{\times}10^{-1}2.59×10−12.59{\times}10^{-1}7.80×10−27.80{\times}10^{-2}6.45×10−26.45{\times}10^{-2}5.41×10−15.41{\times}10^{-1}6.38×10−16.38{\times}10^{-1}
Trajectory + Persona1.20×10−31.20{\times}10^{-3}2.03×10−22.03{\times}10^{-2}2.38×10−12.38{\times}10^{-1}2.80×10−12.80{\times}10^{-1}8.00×10−28.00{\times}10^{-2}7.14×10−27.14{\times}10^{-2}5.89×10−15.89{\times}10^{-1}6.34×10−16.34{\times}10^{-1}
Agentic methods
Appendix
Table 6: Fidelity comparison between different generation methods on the gpt-5.6-sol model. The best results are shown in bold . Statistically significant improvements ( p<0.05p<0.05 ) over all other baselines are marked with ∗ .

Explore similar work

CardsList

  1. TRACER: Trajectory-Aligned Learning for Multi-Turn User Simulation

    Sep 23, 2026Geng Chen, Ruotong Pan, Zhirui Yang +9User SimulationMulti-Turn Interactions

    Peking University · Kuaishou Technology · Xi’an Jiaotong University +3

  2. TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation

    Jun 10, 2026Siyu Li, Toan Tran, Lingyi Zhao +2MobilityTraccia

    Dept. of Computer Science, Emory University, Atlanta, GA, USA · Novateur Research Solutions, Ashburn, VA, USA

  3. SenseWalk: Agent-Based Semantic Trajectory Simulation Powered by Large Language Models in Zoned Environments

    Jul 1, 2026Ziyue Lin, Xinhang Xie, Kangyi Wang +1Microscopic Traffic SimulationsSocial Dynamics

    School of Data Science, Fudan University · Shanghai, China