Large language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommendation and advertising systems. These applications demand the user dimension of LLM intelligence: inferring a person's preferences, intent, and likely future actions from weeks to years of behavior data. Yet current models are trained on static text or synthetic data, neither of which captures how user behavior evolves over time. Real-world behavior records, already available at scale on content platforms and in personal-assistant agents, offer a natural supervisory signal. We propose user behavior as supervision and introduce Vela, a framework that converts real user behavior data into trainable tasks and trajectories. From the anonymized behavior records of 100,000 users, Vela generates 100,000 tasks across 91 real-world task templates; difficulty-based filtering and trajectory acceptance criteria retain approximately 75,000 tasks with selected training trajectories. Fine-tuning Qwen3.6-35B-A3B on these trajectories improves performance over the base model by up to 11.5 points on user-related tasks and by an average of 4 points on general benchmarks. Training on more user behavior data further improves user-related performance, and reinforcement learning after fine-tuning outperforms reinforcement learning from the base model by up to 7.7 points. These gains suggest that real-world user behavior is a scalable source of supervision for both user-centered capabilities and the broader intelligence of LLMs.
Figures & tables
Figure 2: Overview of Vela, a framework for user behavior as supervision. (1) Anonymized behavior records from 100,000 users are split into visible histories and hidden futures. (2) A set of 91 question templates spanning two families, Predictive and Rubric , is instantiated to construct 100,000 composite tasks. Predictive questions are scored against ground-truth answers derived from future behavior, while Rubric questions are scored using task-specific rubrics. (3) Tasks are filtered by difficulty, and training trajectories are generated and selected to form two training corpora: Vela-Instruct and Vela-RFT.
Figure 3: Distribution of the 91 question templates across six behavior categories and four question types. Objective and Open-prediction belong to the Predictive family, while Open and Attribution belong to the Rubric family. Cell values indicate template counts.
Figure 4: An illustrative example of user behavior as supervision. A behavior history and two templates form a task with Predictive and Rubric questions. Future behavior provides the predictive ground truth, while the scorer evaluates the Open response using a task-specific rubric.
Benchmark
Gemma-4- 31B-it
Qwen3.5- 397B-A17B
DeepSeek- V4-Flash
Qwen3.8- 2.4T-A95B
Kimi-K3
Qwen3.6- 35B-A3B
Vela-35B- A3B-RFT
Vela-35B- A3B
Size
31B
397B
284B
2.4T
2.8T
35B
35B
35B
In-domain
VelaBench
55.6
55.8
58.6
62.3
64.0
51.5
53.5 ( + 2.0)
63.0 ( + 11.5)
User-related OOD
PurchaseIntent
44.4
41.7
44.6
45.5
44.9
43.8
42.9 ( − 0.9)
44.0 ( + 0.2)
ProfileSummary
22.8
22.0
31.1
34.1
38.3
20.4
23.0 ( + 2.6)
30.5 ( + 10.1)
Table 1: Performance of Vela-35B-A3B, Vela-35B-A3B-RFT, and the baselines across ten benchmarks. Results are grouped into In-domain (VelaBench), User-related OOD , and General OOD . Avg. is the mean within each OOD group. Parentheses give the change over the base model (Qwen3.6-35B-A3B): red for gains, green for drops.
Figure 5: Performance of Qwen3.6-35B-A3B fine-tuned on Vela-Instruct at different data scales: (a) VelaBench, (b) average user-related OOD performance, and (c) average general OOD performance. Dashed lines indicate base model performance.
Figure 6: Performance before and after GSPO on (a) VelaBench and (b) ProfileSummary, starting from Qwen3.6-35B-A3B or Vela-35B-A3B. Vela-35B-A3B here denotes the SFT checkpoint trained on 18.8K Vela-Instruct examples.
In-dom.
User-related OOD
VelaBench
PurchaseIntent
ProfileSummary
PersonaBench
AdRecommendation
Qwen3.6-35B-A3B
51.5
43.8
20.4
50.0
51.1
w/o difficulty-based filtering
62.4
44.6
29.0
51.5
61.0
w/ difficulty-based filtering
62.4
45.4
30.5
52.7
61.9
Δ
( + 0.0)
( + 0.8)
( + 1.5)
( + 1.2)
( + 0.9)
Table 2: Ablation of difficulty-based filtering (Section 2.3 ). Both models are fine-tuned on 37.5K training examples. Δ denotes the score difference between models trained with and without difficulty-based filtering.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Behavior
Question type
Template
Views
Objective
Select the items in [candidate content] that the user will engage with during [future window]. Engagement includes extended viewing, liking, saving, commenting, or following; exposure or a click alone does not qualify.
Views
Open-prediction
Predict the topics and key entities of content the user will engage with during [future window], without a candidate list.
Views
Open
Summarize the user’s content interests in the visible history. Support each interest with observed behavior and state uncertainty when the evidence is weak.
Views
Attribution
Given [content the user engaged with], identify earlier topics, content, or interactions that help explain this engagement.
Purchases
Objective
Select the products in [candidate products] that the user will purchase during [future window]. Distinguish completed purchases from browsing, clicking, or adding products to a cart.
Purchases
Open-prediction
Predict which products the user will purchase during [future window], without a candidate list. Keep the prediction at the product level.
Appendix
Table 3: Representative task templates across six behavior categories and four question types.
Parameter
Value
Optimization
Optimizer
Adam
Learning rate
0.000001, constant; no warmup
Adam beta coefficients
(0.9,0.98)
Weight decay / gradient clipping norm
0.1 / 1.0
KL loss / coefficient
Low-variance KL / 0.001
Appendix
Table 4: Shared reinforcement learning configuration for all four experiments.
Although a growing body of research has begun to describe user--LLM interactions, the picture it paints is largely static; little is known about how individual users change their behavior over time. To address this gap, we analyze the conversational trajectories of ~12,000 randomly sampled Microsoft Bing Copilot users and compare these with data from WildChat-4.8M. While the Copilot data contains significant population-level trends, we find that trends in individual user trajectories are much weaker; user habits prove to be overwhelmingly sticky. We also find stark differences between users of different activity levels: more active users have more successful conversations and use the LLM for more complex and professionally oriented tasks. Some user trends also appear in WildChat-4.8M, but we find evidence that this dataset is significantly skewed towards highly proficient "power" users. Ultimately, our results suggest that existing user behavior is difficult to change and demonstrate the extent of user heterogeneity. Our comparison between datasets highlights that WildChat does not represent typical user--AI interactions, an important caveat for downstream uses of the data.
Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities. To address this gap, we introduce LUNAR, the first benchmark for evaluating how LLMs personalize responses from longitudinal app interaction histories across universal daily-life domains, including clothing, food, housing, and mobility. To support scalable benchmark construction while mitigating data sparsity and privacy concerns, LUNAR uses a multi-stage coarse-to-fine synthesis pipeline grounded in real-world behavioral patterns. Fidelity analyses show closer alignment with real behavioral distributions than other synthetic benchmarks. Experiments on 19 mainstream LLMs show that access to behavioral logs is necessary but not sufficient for deep personalization: neither more context nor larger models guarantees better performance; effective personalization depends on selecting and integrating relevant evidence across domains. Direct retrieval of fine-grained behavioral records consistently outperforms compressed memory, while stronger personalization can come at the cost of privacy protection. These findings identify evidence selection, cross-domain integration, and privacy control as key challenges for personalized LLMs.
Jiahao Zhang, Yongzhi Tong, Zelin Fu +4
1Shenzhen University of Advanced Technology · ♣Work done at Ant Group · 2Ant Group +1
To align a Large Language Model (LLM), most existing methods collect explicit human feedback and train a reward model to predict the human preference based on the response text. These existing methods have two key limitations. First, the users rarely provide explicit feedback for LLM responses, which makes the high-quality preference annotation expensive to collect. Second, the methods do not leverage implicit human feedback, which has proven vital to the economic moats of Internet giants. To quantify the value of implicit feedback, we build a new dataset called IFLLM, which collects 1336 multi-turn questions from the 59 Mechanical Turk workers, their mouse trajectories, and eye gazing points to the LLMs' responses from their webcams. IFLLM shows that the users have very diverse types of gazing behavior and mouse trajectories. Our reward model based on the implicit user feedback boosts the accuracy of the text-based reward model from 55% to 64% and nearly triples the relative response quality improvements after applying the DPO to eight LLMs, demonstrating the value of implicit feedback in the wild. Our data collection website, dataset, and codes can be found at https://github.com/themehulpatwari/llm-implicit-feedback/.
Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari +2
University of Massachusetts, Amherst, USA · York University, Canada