Learning user preferences from long behavioral sequences is fundamental to large-scale advertising, recommender systems, and conversational agents. In industrial systems, preference modeling is typically task-wise, with different downstream tasks independently processing the same user history to capture different aspects of preference. This repeated processing creates substantial redundancy at billion-user scale, especially as preferences evolve and representations require frequent updates. We present KuaFu, a unified user preference modeling system that replaces repeated user-level sequence encoding with reusable item-level compressed representations. Each behavioral item is encoded once and shared across users and downstream tasks, while a user's history is built chronologically and extended with new item representations as they arrive, letting KuaFu capture evolving preferences without re-encoding the entire history. To reduce inference and storage overhead, KuaFu introduces a two-axis projector: token-axis compression shortens behavior sequences by up to 15x and reduces each item from 150 KB to 10 KB, while width-axis compression reduces the cached representation to 0.5 KB. We further develop a four-stage training pipeline consisting of reconstruction pre-training, preference-understanding post-training, joint compression-generation training, and hallucination-aware reinforcement learning, progressively improving representation fidelity and downstream utility. Across six MRQA and five RecBench benchmarks, KuaFu consistently outperforms existing compression methods at comparable ratios, with gains of up to 17.7 EM and 1.90 AUC points respectively. KuaFu has been deployed and evaluated on the Tencent advertising and recommendation platform for ten months. Over this period, it improves per-GPU throughput by up to 350%, saves 190 GPUs, and increases overall GMV by 1.37%.
Figures & tables
Figure 1. Overall architecture and workflow of KuaFu. Each behavioral item is independently encoded by the item compressor and further reduced by the two-axis projector into a small set of compact memory tokens. Items may correspond to heterogeneous behavioral content, such as product descriptions, content summaries, or advertisement context segments. For a given user, the resulting item representations are assembled in chronological order. The LLM decoder then consumes the assembled sequence to perform diverse user-understanding tasks, including interest modeling, user-feature inference, and open-ended question answering. Since each compressed representation depends only on the item itself, it can be cached offline and reused across users and downstream tasks, avoiding repeated encoding of long behavioral histories.
Figure 2. The architecture of the proposed KuaFu framework.
Stage
Objective
C
P
D
Pre-training
reconstruct
update
—
freeze
Post-training
compressed QA
update
update
freeze
Joint-training
fit both ends
update
update
update
Hallu.-aware RL
de-hallucinate
freeze
freeze
update
Table 1. Trainable set per stage. C/P/D denote compressor, projector and decoder respectively.
Task
Family
items
ratio
online
KuaFu
Δ
Content interest
summary
400/1200
10×
85.9
86.5
+0.6
E-com. interest
summary
150/200
10×
95.7
96.1
+0.4
Industry
profile
30/100
15×
81.0
82.1
+1.1
Occupation
profile
30/100
15×
86.7
87.6
+0.9
Life stage
profile
16/100
15×
76.0
78.0
+2.0
Table 2. Downstream user-understanding tasks in production. Δ is the improvement in percentage compared with the multi-task online model.
Method
E-com.
Occup.
Content
Life
Overall
LLMLingua-2 †
49.7
31.1
22.9
22.7
31.6
ICAE
50.5
43.1
30.9
21.7
36.6
SAC
48.3
50.7
34.3
23.7
39.2
500xCompressor
55.7
46.1
32.0
23.9
39.4
EPL
53.9
56.3
34.7
24.5
42.4
KuaFu
71.7
64.5
64.7
51.1
63.0
Table 3. In-house benchmark (percentage scale). All soft baselines are retrained under the aligned protocol described in the text. † : training-free hard compression at 5× , listed as an external reference only.
Figure 3. MRQA exact match vs. compression ratio (Llama-3.2-1B). Shaded band: 15× , the ratio used everywhere else. ROUGE-1 F1 and all baseline values: Appendix B .
Figure 4. Hallucination-aware RL, scored by GPT-5.5, a stronger LLM than the training reward model; agreement with human raters and a second judge is in Appendix E . (a) hit rate per error type (lower better), pp change above each pair; (b) fully correct answers; (c) completeness, shares summing to 100% . All values in percent.
Figure 5. Needle-in-a-haystack retrieval fidelity (F1) across the four training stages (rows) by items per test sample (columns); greener (darker) shades are higher.
Group
Details
Curriculum
Setting: mixed 1 – 1200 (980K) vs. five increasing buckets
Metric: train loss 2.27 vs. 0.2, 0.28, 0.16, 0.02, 0.034
Inheritance
Setting: without vs. with pre-training
Metric: init/train/val loss 3.2/0.7/4.1 vs. 1.2/0.6/ 0.8
Token axis
Setting: pooling / adapter / adapter + pooled residual
Metric: train loss 1.9967 / 1.8026 / 1.4148
Table 4. Ablation summary. Best value of each group is in bold; per-group details are in Appendix D .
Figure 6. Production deployment of KuaFu: a daily item-side pipeline and a weekly user-side pipeline, served by two independently scaled clusters. The compressor is an offline batch, per-item cacheable computation; the decoder is a routine, per-user sequence-understanding computation. The offline cache grows with item inventory rather than with user count.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Objective
C
P
D
Task composition
Samples
Tokens
Pre-training
restore text
upd.
—
frz.
behavior seq. + long text, bucketed by item count
2.13M
8.06B
Post-tr. (simple)
compressed QA
upd.
upd.
frz.
8 behavior-understanding + 7 generic NLP tasks
150K
0.80B
Post-tr. (complex)
compressed QA
upd.
upd.
frz.
22 complex long-sequence + 4 profiling tasks
350K
2.30B
Joint-training
fit both ends
upd.
upd.
upd.
same composition as post-training
350K
2.30B
Hallu.-aware RL
suppress 4 errors
frz.
frz.
upd.
long-sequence open-ended QA
5K
41M
Appendix
Table 5. Four-stage training recipe and data statistics. C/P/D denote the compressor, the projector and the decoder; upd. / frz. mark whether the module is updated or frozen in that stage.
In-domain
Out-of-domain
Method
5×
15×
51×
5×
15×
51×
ICAE ( Ge et al., 2023 )
33.82 / 47.53
31.28 / 44.5
27.99 / 40.39
20.51 / 31.22
20.18 / 30.81
17.85 / 27.98
500xCompressor ( Li et al., 2024 )
40.14 / 55.26
34.98 / 49.35
30.26 / 43.19
25.4 / 38.46
22.09 / 34.04
20.09 / 30.43
EPL ( Zhao et al., 2024 )
46.33 / 62.9
36.65 / 51.52
30.26 / 43.22
31.3 / 46.95
23.83 / 36.74
19.48 / 30.22
SAC ( Liu et al., 2025b )
46.95 / 63.63
39.67 / 54.95
33.08 / 46.37
32.3 / 47.72
26.02 / 39.26
21.44 / 32.24
KuaFu (ours)
52.5 / 66.1
50.8 / 61.7
36.4 / 45.9
50.0 / 60.1
43.1 / 54.2
37.5 / 48.2
Appendix
Table 6. Full MRQA results, EM / ROUGE-1 F1 on a Llama-3.2-1B backbone. The out-of-domain columns use the same checkpoint, zero-shot.
Figure 7. RecBench zero-shot pairwise CTR, mean AUC over the five datasets, against parameter count. Four notes. (i) GPT-3.5 (0.523) is served through an API and P5-Beauty (0.505) reports no parameter count, so neither is plotted; the two strongest models overall are Mistral-7B (0.6199) and GLM-4-9B (0.6231), both marginally above KuaFu-ZS. (ii) All differences in the body are computed on unrounded means – KuaFu-ZS is 0.6166 , drawn here as 0.617 – so recomputing them from three-decimal figures differs by 0.03 – 0.05 pp. (iii) The mean hides one regression: on the CDs subset the compressed representation brings no gain ( 0.489 , below the 0.510 of the Qwen3-4B backbone). (iv) KuaFu-ZS gets no task-specific training; only its user-behavior input is compressed.
Metric (DAPO − joint-training)
Human
GPT-5.5
Opus-4.8
LLM mean ± SD
κ (GPT / Opus)
Fabrication hit rate ↓
−28.4
−29.0
−27.8
−28.4±0.8
0.89 / 0.84
Missed detection hit rate ↓
+1.3
+1.5
+1.7
+1.6±0.1
0.79 / 0.72
Date misattribution hit rate ↓
−8.1
−8.5
−8.8
−8.7±0.2
0.86 / 0.80
Logic / task error hit rate ↓
−5.2
−5.5
−5.1
−5.3±0.3
0.83 / 0.76
Fully correct answers ↑
+33.0
+34.0
+32.6
+33.3±1.0
0.91 / 0.86
Complete answers ↑
+0.7
+0.5
+0.9
+0.7±0.3
0.81 / 0.74
Appendix
Table 8. Judge robustness for the hallucination-aware RL results. Each cell is the DAPO − joint-training change of the metric in percentage points, measured under human annotation, the primary test judge GPT-5.5, and a second judge Opus-4.8. “LLM mean ± SD” is taken over the two LLM judges; “ κ ” is Cohen’s agreement of each LLM judge with the human labels. The GPT-5.5 column reproduces exactly the values drawn in Figure 4 .
As LLM agents tackle longer tasks, they increasingly compress growing histories of reasoning, actions, and tool outputs. Compression can reduce token use, but it also changes the information available for later decisions. Existing agentic harnesses bundle decisions about what to compress, when to compress, and how much to remove into fixed policies. A systematic characterization is needed to disentangle these decisions and reveal how each affects task success and execution cost. We systematically vary these decisions across three open-weight models on SWE-bench Verified and Terminal-Bench 1.0. Across nearly 35,000 agent runs, we measure task success, token use, end-to-end latency, and estimated cost. We find that fewer tokens need not mean faster or cheaper execution: on Terminal-Bench with Qwen, policies using roughly one-third as many tokens can take 20-80% longer than the uncompressed agent. Policies with similar overall success can solve different tasks, while the same policy can perform quite differently across models. Our results motivate evaluating compression by its effects on agent execution and tailoring policies to the task, model, and workload.
KV caches are a major bottleneck in long-context inference and long-form generation with large language models. Existing training-free eviction policies largely rely on proxy importance signals, such as attention mass, to decide which past tokens to retain. We argue that cache compression should instead preserve the predictive behavior of the full-cache model, retaining entries whose removal would substantially change the model's output distribution. We propose Behavior-Preserving KV Cache Compression, a training-free framework that scores candidate evictions by estimating the compressed-cache logits induced by their removal and evaluating the resulting KL to the full-cache next-token distribution. Using pre-eviction forward statistics, the method avoids running separate masked forward passes for each candidate. Across diverse architectures and both prefill-time and generation-time compression, our method delivers substantial gains in downstream task quality over lightweight attention-based heuristics at matched retained-KV budgets, with the largest gains under aggressive compression. It achieves these gains with additional compression-time computation while retaining an end-to-end speedup over full-cache inference in our evaluated settings.
Doo Hwan Hwang, Junyoung Jang, Junho Na +2
Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea · KT Corporation, Seoul, Republic of Korea
Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints. While summarizing history via interest centers offers a practical alternative, existing methods struggle to (1) identify user-specific centers at appropriate granularity and (2) accurately assign behaviors, leading to quantization errors and loss of long-tail preferences. To alleviate these issues, we propose Hierarchical Sparse Activation Compression (HiSAC), an efficient framework for personalized sequence modeling. HiSAC encodes interactions into multi-level semantic IDs and constructs a global hierarchical codebook. A hierarchical voting mechanism sparsely activates personalized interest-agents as fine-grained preference centers. Guided by these agents, Soft-Routing Attention aggregates historical signals in semantic space, weighting by similarity to minimize quantization error and retain long-tail behaviors. Deployed on Taobao's "Guess What You Like" homepage, HiSAC achieves significant compression and cost reduction, with online A/B tests showing a consistent 1.65% CTR uplift -- demonstrating its scalability and real-world effectiveness.
Kun Yuan, Junyu Bi, Daixuan Cheng +5
Alibaba Group · Beijing, China · Renmin University