CRIM: Composable and Reusable Item Memory for Efficient Personalized Retrieval

Haolin Li; Haipeng Zhang; Mang Li; Zhaohui Xia; Xun Zhu; Miao Li; Yu Zhang; Ji Wu

cs.IROct 26, 2026

CRIM: Composable and Reusable Item Memory for Efficient Personalized Retrieval

Authors: Haolin Li, Haipeng Zhang, Mang Li, Zhaohui Xia, Xun Zhu, Miao Li, Yu Zhang, Ji Wu

Organizations: Tsinghua University, Beijing, China · Alibaba International Digital Commerce Group, Beijing, China

Abstract

Large language models incur substantial training and inference costs in personalized search and recommendation, as growing interaction histories lengthen request inputs and repeatedly expose the same item text to the model. We propose Composable and Reusable Item Memory (CRIM), a framework that separates item-level semantic encoding from request-level history modeling. A small set of learnable summary tokens compresses each item into compact continuous memories, while segment-level readout tokens (REC tokens) introduce predictive supervision within the interaction history. These memories are independent of users and queries and can therefore be reused across requests. At inference time, CRIM composes cached memories in interaction order and incorporates an optional query to produce a personalized retrieval representation, reducing repeated item-text encoding while retaining request-dependent contextual modeling. We further introduce a joint training mechanism that combines cached memories with sampled live re-encoding. Re-encoding only a few history items per request allows the retrieval objective to jointly optimize the item and request encoders at limited computational cost, while periodic cache refreshes maintain consistency. Extensive evaluation and cost analysis on MovieLens-1M, Qilin, and our new LazSeq dataset show that CRIM achieves state-of-the-art retrieval performance with substantially lower inference costs at the same backbone scale, while limiting item re-encoding during joint training. We will publicly release our code and dataset.

Figures & tables

Figure 1. Recall@100 versus mean request latency on LazSeq (single A100, histories up to 500 interactions). The inset reports mean serving input lengths, including memory slots, REC tokens, queries, and special tokens. A scatter plot compares eight models on LazSeq. CRIM has the highest Recall at 100 and the lowest mean request latency among the displayed models. The inset shows 967 mean serving input positions for CRIM and 5,020 for Qwen3-Embedding-4B.

Figure 2. Overview of CRIM. The IME encodes each item into reusable continuous memories. The RE composes cached memory blocks in interaction order, incorporates an optional query, and produces a representation for retrieval against separately encoded catalog candidates. The CRIM framework separates independent item memory encoding, reuse of cached memories in ordered histories, and request-conditioned retrieval.

Figure 3. Semantic pretraining: summary tokens compress item identity and text under restricted-attention reconstruction and full-context distillation with a frozen backbone. Summary tokens form a bottleneck between the source item and its reconstructed title. Reconstruction and distribution matching supervise the memory slots.

Figure 4. Joint training: sampled positions use live IME outputs; the others reuse cached memories. Retrieval, segment prediction, and reconstruction provide supervision. The training pipeline combines a cached memory path and a sampled live IME path, preserving the full ordered history for the RE and refreshing cached representations after training epochs.

		Recall					NDCG			MRR
Method	Year	@10	@50	@100	@1000	@3000	@10	@50	@100	@100	All
GRU4Rec	2016	0.0300	0.0670	0.0944	0.2598	0.3951	0.0164	0.0246	0.0290	0.0144	0.0151
NARM	2017	0.0223	0.0634	0.0894	0.2544	0.3941	0.0119	0.0207	0.0250	0.0109	0.0116
SASRec	2018	0.0313	0.0697	0.0937	0.2487	0.3765	0.0170	0.0253	0.0292	0.0147	0.0154
STAMP	2018	0.0290	0.0620	0.0794	0.2274	0.3588	0.0153	0.0225	0.0253	0.0128	0.0135
BERT4Rec	2019	0.0250	0.0600	0.0860	0.2661	0.4201	0.0127	0.0204	0.0246	0.0109	0.0117

Table 1. Full-catalog retrieval on LazSeq (histories up to 500 items). Bold/underlined values mark the best/second-best results; dashes denote metrics unavailable from beam-limited outputs.

		Recall		NDCG		MRR
Method	Year	@10	@100	@10	@100	@100	All
GRU4Rec	2016	0.0000	0.0000	0.0000	0.0000	0.0000	0.0001
NARM	2017	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
SASRec	2018	0.0000	0.0000	0.0000	0.0000	0.0000	0.0001
STAMP	2018	0.0034	0.0034	0.0028	0.0028	0.0026	0.0026
BERT4Rec	2019	0.0000	0.0011	0.0000	0.0002	0.0000	0.0001

Table 2. Full-catalog retrieval on Qilin (histories up to 100 items). Bold/underlined values mark the best/second-best results; dashes denote metrics unavailable from beam-limited outputs.

		Recall		NDCG		MRR
Method	Year	@10	@100	@10	@100	@100	All
GRU4Rec	2016	0.0510	0.1555	0.0269	0.0465	0.0227	0.0244
NARM	2017	0.0510	0.1671	0.0302	0.0523	0.0273	0.0290
SASRec	2018	0.0534	0.1740	0.0300	0.0531	0.0263	0.0279
STAMP	2018	0.0534	0.1647	0.0314	0.0531	0.0282	0.0298
BERT4Rec	2019	0.0487	0.2019	0.0279	0.0565	0.0259	0.0276

Table 3. Full-catalog retrieval on ML1M (histories up to 100 items). Bold/underlined values mark the best/second-best results; dashes denote metrics unavailable from beam-limited outputs.

Figure 5. Cumulative A100 compute on LazSeq: optimization, initial cache, and warm-cache inference. Inset: published service volumes ( DuckDuckGo, n.d. ; Fu et al., 2026 ; Pinterest, 2026 ; Algolia, 2025 ; Eksombatchai et al., 2018 ; Srinivasan, 2025 ) ; the two P logos denote Pinterest Search and Pixie recommendations. Lazada: 30M searches/day (2020) ( MARKETECH APAC, 2020 ) . Stars mark conversions from requests/s to requests/day (Appendix A.3 ). CRIM and Qwen3-Embedding-4B cumulative compute curves cross near 31,500 requests. At one million requests they reach 71.6 and 146.3 GPU-hours, respectively. A separate inset places public services on a logarithmic daily request-equivalent scale. The two Pinterest logos identify Search and Pixie recommendation workloads.

Variant	R@100	N@100	M@100
CRIM	69.69	26.14	15.76
w/o $\mathcal{L}_{\mathrm{seg}}$	65.99	24.19	14.23
w/o $\mathcal{L}_{\mathrm{sem}}$	69.42	25.69	15.40
Retrieval loss only	65.66	24.15	14.23
w/o REC and $\mathcal{L}_{\mathrm{seg}}$	66.29	24.39	14.39
w/o semantic pretraining	68.66	25.36	15.05

Table 4. LazSeq supervision and pretraining ablations (%). R, N, and M denote Recall, NDCG, and MRR. Best retrieval metrics are bold.

Figure 6. LazSeq construction. (a) Requests are filtered by query–history lexical overlap. (b) Item frequencies in history and target positions determine the retained candidate set; requests are kept when their target is retained. An example compares histories that meet or fail query overlap filtering. A second panel shows item frequency aggregation, target-based request filtering, and construction of a shared item catalog.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	LazSeq	Qilin	ML1M
Backbone	Qwen3-Embedding-4B; bf16; LoRA rank 16
Slots / live history positions	$K=4$ / at most 8	$K=4$ / at most 4	$K=4$ / at most 4
Segment length / cache refresh	4 items / every epoch
Joint training learning rate	$10^{-4}$ ; weight decay 0; gradient clip 1
Joint training maximum epochs / batch	3 / 64	3 / 64	10 / 8
Selected epoch	3	2	7

Appendix

Table 5. CRIM training configurations for the main results.

Variant	R@100	N@100	M@100
CRIM	$\mathbf{69.68}\pm 1.40$	$\mathbf{26.02}\pm 0.20$	$\mathbf{15.66}\pm 0.33$
w/o segment loss	$66.29\pm 0.73$	$24.10\pm 0.16$	$14.10\pm 0.12$
w/o semantic loss	$69.12\pm 1.11$	$25.61\pm 0.32$	$15.40\pm 0.13$
Retrieval loss only	$66.43\pm 0.69$	$24.12\pm 0.03$	$14.07\pm 0.15$

Appendix

Table 6. Objective ablations across seeds 42, 43, and 44. Values are mean

\pm

sample standard deviation, in percentages. Best means are bold.

Variant	R@100	N@100	M@100
CRIM ( $\kappa=8$ , epoch refresh)	69.69	26.14	15.76
Re-encoding $\kappa=1$	69.56	25.99	15.60
Re-encoding $\kappa=4$	69.66	25.89	15.45
Refresh every two epochs	68.49	25.49	15.21
No item-memory cache refresh	66.62	24.70	14.70

Appendix

Table 7. Training-time history re-encoding and item-memory cache refresh on LazSeq. Metrics are percentages; evaluation uses rebuilt caches. Best retrieval metrics are bold.

Variant	R@100	N@100	M (all)	Cache (GB)
$K=1$	69.02	25.38	15.09	1.024
CRIM ( $K=4$ )	69.69	26.14	15.87	2.560
$K=8$	68.22	25.20	15.07	4.607
$K=16$	69.22	25.75	15.49	8.703
$K=32$	68.20	25.31	15.13	16.894

Appendix

Table 8. Memory capacity on LazSeq. R and N use cutoff 100; M is full-catalog MRR. Metrics are percentages. Cache includes FP32 item memories and candidate vectors. Best retrieval metrics are bold.

History content
Variant	R@100	N@100	M@100
Original history	69.52	25.93	15.53
Cross-user history swap	66.12	23.67	13.62
No history	65.82	23.69	13.68
Mean item memories	64.39	23.23	13.45
History order, length, and query

Appendix

Table 9. Inference-time input interventions on LazSeq. Values are percentages. Best values within each group are bold.

Group	Requests	CRIM	Dense
History 1–50	116	81.03	70.69
History 51–100	450	78.22	65.78
History 101–250	1,540	76.04	68.70
History 251–500	893	52.97	67.53
Frequency 1–10	7	100.00	57.14
Frequency 11–100	1,718	72.47	68.86

Appendix

Table 10. Recall@100 (%) by retained history length and training frequency on the same 2,999 requests. Dense denotes Qwen3-Embedding-4B. Best values per row are bold.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 471 papers

rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation
Aug 19, 2026Minh Hoang Nguyen, Tung Le, Huy Tien NguyenRecommender Systems Large Language Model-Based Recommendation
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
Response-Aware User Memory Selection for LLM Personalization
Apr 15, 2026Jillian Fisher, Jennifer Neville, Chan Young ParkLLM Personalization Memory-Augmented Language Models
Department of Computer Science and Engineering, University of Washington, Seattle, WA, United States of America · Microsoft Research, Redmond, WA, United States of America
Latent Personal Memory: Represent personal memory as dynamic soft prompts
Jun 18, 2026Debrup Das, Avinash Amballa, Yashas Malur Saidutta +3LLM Inference Efficiency Memory-Efficient Fine-Tuning
University of Massachusetts Amherst · Samsung Research America

Aug 19, 2026J/K move · Enter open · S save

rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation

Minh Hoang Nguyen, Tung Le, Huy Tien Nguyen

Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Recommender Systems Large Language Model-Based Recommendation Persistent Memory for Language Models Language Model Distillation

Open paper AskPDF arXiv

Large language models can improve recommendation quality by reasoning explicitly over user history and candidate items - for example, extracting a user's preferences or explaining why one item fits better than another - rather than mapping history directly to a ranked list. This reasoning, however, is expensive to repeat on every ranking request and, once produced, is typically consumed once and discarded, leaving it neither reusable across future requests nor easy to inspect or correct as user tastes drift. Our insight is that reasoning does not need to be regenerated at every call if it can instead be compressed once into a compact, structured memory that a lightweight model retrieves from. We propose rEDMRec, which distills a teacher LLM's reasoning into four typed, editable experience channels - long-term preference, short-term context, item-perception, and counterfactual hard-negative comparisons - maintained by an LLM memory controller that performs Add/Delete/Modify/Keep operations and refines entries via K-agent debate. A lightweight student LLM then ranks candidates purely by retrieving from this memory, without invoking the teacher again, decoupling online inference cost from reasoning depth. Across ML-1M, Amazon Beauty, and Steam and ten student backbones, rEDMRec improves HR@1 over zero-shot, few-shot, and RAG on every backbone, and over GraphRAG on most backbones, with Impv up to 13.3% vs. the second-best baseline on ML-1M. Channel ablations show that short-term context is the only channel that helps consistently across capacity tiers, whereas long-term, item-perception, and counterfactual contributions are capacity-dependent (and can reverse on the strongest students); debate-based memory optimization lowers bank duplication by 7.4 percentage points while raising downstream HR@1 by up to +0.029 over six optimization epochs.

Abstract

Information Retrieval

Explore similar work

rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation

Response-Aware User Memory Selection for LLM Personalization

Latent Personal Memory: Represent personal memory as dynamic soft prompts