Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

Jiahao Yuan; Yike Xu; Jinyong Wen; Baokun Wang; Ziyi Gao; Xiaotong Lin; Yun Liu; Xing Fu

cs.CLFeb 16, 2026

Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

Authors: Jiahao Yuan, Yike Xu, Jinyong Wen, Baokun Wang, Ziyi Gao, Xiaotong Lin, Yun Liu, Xing Fu,

Organizations: Ant Group · East China Normal University · Zhejiang University

Abstract

Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.

Figures & tables

Figure 1. Comparison between (A) General User Embedding ( Dou et al., 2025 ) and (B) our Query-as-anchor. (A) learns transferable user representations across domains but generates fixed embeddings regardless of downstream context. (B) extends (A) with query-as-anchor modulation, enabling a single model to produce adaptive, domain-specific embeddings via natural language instructions.

Figure 2. Overview of Query-as-Anchor Framework for our Q-Anchor Embedding.

Panel of Figure 3 — Figure 3. Input Template of UserU . The Hierarchical User Tokens: $\mathbf{e}_{i}$ injects the precomputed hierarchical embedding $\mathbf{e}_{i}$ (Eq. 2 ). An optional instruction is followed by the special <USER_EMB> token, which signals the model to extract a unified user embedding (Sec. 3.2.2 ).

Figure 4. KV-Cache optimized Query-as-Anchor inference: pre-computed user prefixes enable sequence, low-latency re-anchoring across diverse tasks.

Dataset	Domain	Scenario	Number
$\mathcal{D}_{train}$	General ( 3.1 )	General	$\approx$ $1.024\times 10^{8}$
$\mathcal{D}_{test}$	❶User Engagement	Interest Community Active User Identification (Active), Concert Click Prediction (Concert), User Log-in Prediction (Login), Ant Forest Engagement (Forest)	$\approx$ 50w per task
	❷Risk	Fraud Detection (Fraud), Money Laundering Detection (Money)	$\approx$ 50w per task
	❸Marketing Se- nsitivity	Takeout Interest (Takeout), Brand Sensitivity (Brand), Big Sale Sensitivity (Promo), Cost-Performance Sensitivity (Value)	$\approx$ 50w per task

Table 1. Data information for user pretraining and test benchmarks, with number of tests per task.

	User Engagement				Risk		Marketing Sensitivity				Avg. AUC
Method	Active	Concert	Login	Forest	Fraud	Money	Takeout	Brand	Promo	Value
General Embedding Models
Qwen2.5-0.5B-Instruct	0.5269	0.5173	0.7219	0.7161	0.6969	0.6885	0.6361	0.5855	0.5483	0.7134	0.6351
Qwen3-Embedding-0.6B	0.5378	0.5226	0.7294	0.7287	0.7106	0.7123	0.6914	0.6076	0.5508	0.7172	0.6508
Llama-Embed-Nemotron-8B	0.5882	0.5627	0.7735	0.8372	0.8632	0.8708	0.8176	0.7525	0.5918	0.8303	0.7488
KaLM-Embed.-Gemma3-12B	0.5597	0.5359	0.7609	0.7729	0.8229	0.8174	0.7812	0.6564	0.589	0.7609	0.7058

Table 2. AUC performance on 10 key scenarios where our Q-Anchor (base version: pretrained) and (prompt-tuning version: post-trained) shows consistent improvement. KS results are detailed in Appendix C.1 .

Figure 5. Average KS performance of Q-Anchor and baselines across 10 Alipay scenarios. Per-scenario results are provided in Appendix C.1 .

Panel of Figure 6 — Figure 6. Scalability analysis of Q-Anchor Embedding (Base). (a) Performance increases with more pretraining data. (b) Performance vs. model parameters (0.5B to 3.0B).

Panel of Figure 7 — Figure 7. Modal attention shift after prompt tuning. Left : Takeout Interest—Bill attention increases by +26.0%, capturing dining purchasing power. Right : Ant Forest—SPM attention rises by +6.4%, consistent with navigation-heavy usage. Red arrows mark positive attention gains, highlighting Q-Anchor’s dynamic feature re-anchoring.

Panel of Figure 8 — Figure 8. Scalability analysis of Q-Anchor Embedding (Prompt Tuned). (a) Performance vs. the number of learnable prompt tokens (1-16, Ours: 6). (b) Performance vs. the training-step budget (100–500, Ours: 500).

Panel of Figure 9 — Figure 9. t-SNE Visualization of universal and prompt-tuned representation of 10 scenarios. PCA visualizations are provided in Appendix C.4 .

	User Engagement				Risk		Marketing Sensitivity				Avg. AUC
Method	Active	Concert	Login	Forest	Fraud	Money	Takeout	Brand	Promo	Value
Q-Anchor (Base)	0.6568	0.5739	0.8420	0.9700	0.9218	0.9382	0.8799	0.7979	0.6189	0.9049	0.8104
Ablation on Modality Token
w / o User Tok.	0.6525	0.5757	0.8424	0.9702	0.9219	0.9358	0.8785	0.7884	0.6195	0.9006	0.8086
w / o Modal Tok.	0.6546	0.5701	0.8423	0.9700	0.9214	0.9372	0.8778	0.7903	0.6210	0.9036	0.8088
w / o Modal & User Tok.	0.6544	0.5703	0.8403	0.9694	0.9211	0.9374	0.8783	0.7819	0.6141	0.8981	0.8065

Table 3. Ablation study (AUC) of Q-Anchor Embedding across modality, training method and prompt tuning. KS results are detailed in Appendix C.1 .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10. Illustrative toy instances from the UserU dataset. Figure (a) illustrates the behavior-to-behavior prediction task, while Figure (b) showcases the behavior-to-semantic reasoning capability enabled by query-aware alignment.

Domain	Scenario (Task)	Target	Positive ( $y{=}1$ )	Negative ( $y{=}0$ )
User Engagement	Interest Community Active User Identification (Active)	Whether a user will be active in the Alipay interest community.	Any qualified interest-community activity occurs (e.g., community visit/app open, content browse, like, comment, post).	No qualified interest-community activity occurs.
	Concert Click Prediction (Concert)	Whether a user will click concert-related content.	Any click on items categorized as concert-related occurs.	No click on concert-related items occurs.
	User Log-in Prediction (Login)	Whether a user will log in.	At least one login event occurs.	No login event occurs.
	Ant Forest Engagement (Forest)	Whether a user will engage with Ant Forest.	Any Ant Forest interaction occurs (e.g., enter/collect/participate).	No Ant Forest interaction occurs.
Risk	Fraud Detection (Fraud)	Whether a user will be associated with a fraud-related risk event.	User is confirmed/flagged by the risk-control pipeline as fraud-related.	User is not flagged as fraud-related.
Risk	Money Laundering Detection (Money)	Whether a user will be associated with a money-laundering-related risk event.	User is confirmed/flagged by compliance/risk-control as money-laundering-related.	User is not flagged as money-laundering-related.

Appendix

Table 4. Downstream test benchmarks and scenario-level binary label definitions. Labels are assigned by whether a user triggers the target event within the prediction window. Each task contains

\approx

50w test data.

	User Engagement				Risk		Marketing Sensitivity				Avg. KS
Method	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
General Embedding Models
Qwen2.5-0.5B-Instruct	0.0421	0.0280	0.3326	0.3174	0.3034	0.2958	0.2066	0.1324	0.0880	0.3262	0.2073
Qwen3-Embedding-0.6B	0.0524	0.0330	0.3459	0.3274	0.3102	0.3086	0.275	0.1581	0.0814	0.3270	0.2219
Llama-Embed-Nemotron-8B	0.1302	0.0857	0.4003	0.5201	0.5758	0.5868	0.4918	0.3685	0.1364	0.5091	0.3805
KaLM-Embed.-Gemma3-12B	0.0880	0.0533	0.3830	0.4036	0.4944	0.4879	0.4207	0.2139	0.1312	0.3937	0.3070

Appendix

Table 5. KS performance on 10 key scenarios where our Q-Anchor (base version: pretrained) and (prompt-tuning version: post-trained) shows consistent improvement.

	User Engagement				Risk		Marketing Sensitivity				Avg. KS
Method	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
Q-Anchor (Base)	0.2228	0.1023	0.5216	0.8437	0.7054	0.7450	0.6210	0.4527	0.1727	0.6567	0.5044
Ablation on Modality Token
w / o User Tok.	0.2176	0.1021	0.5204	0.8440	0.7056	0.7387	0.6200	0.4309	0.1744	0.6478	0.5002
w / o Modal Tok.	0.2186	0.0982	0.5192	0.8415	0.7039	0.7407	0.6187	0.4321	0.1775	0.6574	0.5008
w / o Modal & User Tok.	0.2181	0.0966	0.5169	0.8411	0.7030	0.7402	0.6184	0.4246	0.1654	0.6414	0.4966

Appendix

Table 6. Ablation study (KS) of Q-Anchor Embedding across modality, training method and prompt tuning.

	User Engagement				Risk		Marketing Sensitivity				Avg.
Steps / Data scale	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
AUC Performance
w/ 10k / 20.48M	0.6443	0.5630	0.8419	0.9707	0.9170	0.9340	0.8759	0.7750	0.6206	0.8869	0.8029
w/ 20k / 40.96M	0.6481	0.5713	0.8429	0.9705	0.9198	0.9362	0.8776	0.7988	0.6259	0.8999	0.8091
w/ 30k / 61.44M	0.6472	0.5735	0.8429	0.9706	0.9203	0.9365	0.8782	0.8027	0.6229	0.9041	0.8099
w/ 40k / 81.92M	0.6532	0.5742	0.8419	0.9704	0.9204	0.9369	0.8792	0.7986	0.6159	0.9045	0.8095

Appendix

Table 7. Scalability performance of Q-Anchor Embedding (Base) across pretraining steps (10k–50k) and corresponding data scales (20.5M–102.4M samples, batch size=2048) (AUC & KS).

Panel of Figure 11 — Figure 11. Scalability analysis of Q-Anchor (Base) . (a) Smaller 0.5B backbone achieves best Avg. AUC/KS; larger models show no consistent gains. (b) Gradient magnitude diminishes as model size increases, explaining the non-monotonic performance trend.

	User Engagement				Risk		Marketing Sensitivity				Avg.
Model	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
AUC Performance
Q-Anchor -0.5B (Base)	0.6571	0.5739	0.8420	0.9700	0.9218	0.9382	0.8799	0.7979	0.6190	0.9049	0.8105
Q-Anchor -1.5B (Base)	0.6436	0.5693	0.8444	0.9706	0.9220	0.9376	0.8759	0.7864	0.6156	0.9000	0.8065
Q-Anchor -3B (Base)	0.6404	0.5626	0.8449	0.9712	0.9220	0.9370	0.8777	0.7824	0.6066	0.8989	0.8044
KS Performance

Appendix

Table 8. Scalability performance of Q-Anchor Embedding (Base) across model scales (0.5B–3B).

	User Engagement				Risk		Marketing Sensitivity				Avg.
Token	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
AUC Performance
w / 1 prompt tok.	0.6513	0.5746	0.8408	0.9702	0.9215	0.9424	0.8798	0.8290	0.6250	0.9114	0.8146
w / 2 prompt tok.	0.6576	0.5755	0.8408	0.9696	0.9221	0.9407	0.8810	0.8380	0.6269	0.9170	0.8169
w / 4 prompt tok.	0.6625	0.5792	0.8450	0.9715	0.9220	0.9430	0.8804	0.8501	0.6321	0.9155	0.8201
w / 6 prompt tok.	0.6678	0.5844	0.8443	0.9716	0.9242	0.9439	0.8811	0.8535	0.6350	0.9194	0.8225

Appendix

Table 9. Scalability performance of Q-Anchor Embedding (Prompt Tuned) across learnable token scales (1–16) for prompt tuning (Ours: 6 prompt tokens).

	User Engagement				Risk		Marketing Sensitivity				Avg.
Steps	Active	Concert	Login	Forest	Fraud	Money	TakeOut	Brand	Promo	Value
AUC Performance
w / 100	0.6541	0.5775	0.8441	0.9703	0.9230	0.9382	0.8787	0.8329	0.6308	0.9091	0.8159
w / 200	0.6601	0.5774	0.8432	0.9701	0.9237	0.9403	0.8796	0.8454	0.6303	0.9139	0.8184
w / 300	0.6622	0.5806	0.8434	0.9698	0.9234	0.9428	0.8810	0.8484	0.6281	0.9167	0.8196
w / 400	0.6657	0.5821	0.8434	0.9718	0.9236	0.9433	0.8799	0.8513	0.6348	0.9167	0.8213

Appendix

Table 10. Scalability of Q-Anchor Embedding (Prompt Tuned) across training steps (100–500) for prompt tuning (Ours: 500 steps).

Panel of Figure 12 — Figure 12. PCA Visualization of universal and prompt-tuned representation of 10 scenarios.

Method	AUC	KS
Scenario: IVR Response
Business SOTA	0.6362	0.2411
Q-Anchor (Base)	0.6534	0.2682
Q-Anchor (Prompt Tuned)	0.6863	0.3016
Scenario: Delinquency
Business SOTA	-	0.1499

Appendix

Table 11. Offline performance of Q-Anchor embeddings. Base uses universal embeddings; Prompt Tuned adds lightweight, scenario-specific soft prompts. Both scenarios report AUC and KS, with Delinquency primarily optimizing KS.

User Modeling

User modeling represents people's interests, knowledge, goals, and changing behavior to personalize predictions and AI assistance.

See all 151 papers

ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling
Jun 23, 2026Tianbao Ma, Chang Xi, Yichuan Zou +7User Modeling LLM Reasoning
Kuaishou Technology, Beijing, China
Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization
Jun 3, 2026Heng Cao, Fan Zhang, Jian Yao +8LLM Personalization Prompt Tuning
*Microsoft · †The Hong Kong Polytechnic University · ⋄Shanghai International Studies University +1
TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems
Jun 23, 2026Qingyun Liu, Bo Yan, Yang Liu +16Semantic ID Recommendation Representation Learning
Google DeepMind · YouTube

Jun 23, 2026J/K move · Enter open · S save

ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

Tianbao Ma, Chang Xi, Yichuan Zou, Chengen Li+6

Kuaishou Technology, Beijing, China

User Modeling LLM Reasoning

Open paper AskPDF arXiv

Accurate user modeling often depends on rich interaction histories, which are unavailable for billions of low-activity users. Large Language Models (LLMs) can infer latent user states from static profiles, but this reasoning becomes unreliable when profiles are sparse, and applying an LLM to billions of users is prohibitively expensive. We present ScaleToT, which learns structured reasoning from a small LLM-processed subset and extends it to the broader low-activity user population. To improve reasoning reliability, ScaleToT constructs typed user-state chains with a bounded entropy-guided Tree-of-Thought (ToT) refinement procedure. To make this structured reasoning usable from sparse profiles, the teacher-curated chains are used to train a student model on static profiles through supervised fine-tuning (SFT) and Outcome-Driven Segment-Aware Implicit Reward Policy Optimization (OSIPO). ScaleToT then transfers the student's reasoning representations to a lightweight profile encoder, providing shared reasoning signals for the remaining users without LLM inference. We evaluate ScaleToT on lifetime value (LTV) prediction in a billion-scale advertising deployment. A randomized online A/B test increased LT30 by 6.738%, while offline reasoning covered only 7.32% of the potential population, greatly reducing compute cost compared with full-population reasoning.

Abstract

User Modeling

Explore similar work

ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization

TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems