How Can Recommendation Feedback Evolve Agent Memory?

Shanwen Mao; Mingming Li; Hao Zhang; Zhiheng Li; Yige Wang; Penghua Yu; Junxiong Zhu

cs.AISep 29, 2026

How Can Recommendation Feedback Evolve Agent Memory?

Authors: Shanwen Mao, Mingming Li, Hao Zhang, Zhiheng Li, Yige Wang, Penghua Yu, Junxiong Zhu

Organizations: Harbin Institute of Technology, Harbin, China · Alibaba Group, Hangzhou, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China

Abstract

Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution. However, these signals are delayed and noisy, confounded by audience composition, placement, and recommendation policies, and may result from the combined influence of multiple memories, making accurate attribution difficult. Existing methods rely primarily on immediate feedback or semantic retrieval and therefore struggle to reliably translate recommendation outcomes into memory fitness. To address this challenge, we propose TIDE (Trajectory-Informed Directed Memory Evolution), an external memory evolution framework driven by delayed recommendation feedback. We further introduce Memory Evolution Gain (MEG), which measures the utility improvement of evolved memory over a no memory baseline on strictly future tasks. TIDE treats memory as a capacity-constrained population of experiences: temporal and semantic credit assignment estimates contextual fitness, while responsibility credit distributes outcome signals according to the memories referenced during generation. These signals are then used to reinforce, crossover, mutate, or evict memories. On an e-commerce membership marketing content-generation agent, TIDE achieves a +7.75-percentage-point MEG in offline temporal replay and significantly improves both unique click-through rate (UCTR) and activation rate in an online A/B test. On a delayed-label benchmark, TIDE achieves the lowest mean absolute error (MAE) and root mean squared error (RMSE) and the highest MEG among the compared methods, demonstrating its effectiveness.

Figures & tables

Figure 1: Overview of the closed-loop content-generation system. The agent retrieves reusable experience from memory to generate advertising materials, deploys them through online channels, and updates the memory from delayed engagement and conversion feedback.

Figure 2: TIDE framework. Produce : a frozen generator retrieves task-relevant memories to produce candidate materials. Attribute : temporal, semantic, and responsibility credit convert delayed recommendation outcomes into memory-level fitness. Evolve : the memory curator revises, merges, specializes, or down-weights memories, followed by capacity-constrained population selection.

Method	Generation quality					Memory evolution		Efficiency
	Comp. $\uparrow$ (%)	Fmt. $\uparrow$ (%)	Eff. $\uparrow$ (%)	Div. $\uparrow$ (0–100)	Joint $\uparrow$ (%)	MEG $\uparrow$	Neg. Trans. $\downarrow$ (%)	Adapt. $\downarrow$ (PFLOPs)	Inference $\downarrow$ (tokens/req.)
Backbone: qwen3.5-27b
No Memory	56.0	79.2	56.0	65.5	12.0	0.00	0.0	–	–
Initial Memory (Frozen)	62.0	83.6	80.0	61.5	20.3	-1.88	22.0	0.803	1,271
Full History	57.0	77.0	46.0	68.0	12.0	-1.13	30.0	0.973	1,968
ExpRAG	58.0	77.0	32.0	59.0	12.0	-2.50	32.0	0.876	1,967

Table 1: Offline generation quality and adaptation efficiency.

Metric	TIDE Rel. Uplift
UCTR	+5.10%
Activation Rate	+4.79%

Table 2: Relative uplift in the one-week online comparison.

Figure 3: Pre-launch CTR/CVR ranking and downstream feedback-memory utility.

Figure 4: Responsibility attribution and directed memory editing, with all metrics direction-adjusted so that higher values indicate better performance.

Method	MAE $\downarrow$	RMSE $\downarrow$	MEG@ $\delta{=}5$ $\uparrow$	$\bm{\Delta_{\mathrm{delay}}}$ $(\|\cdot\|\downarrow)$	User Neg. Trans. $\downarrow$	Memory Tok.	LLM Tok. / Feedback-eq. $\downarrow$
StaticMemory ( $\mu_{0}$ )	0.676	1.204	0.0000	+0.0000	0.0%	0	838.3
FullHistory $\dagger$	0.575	1.073	+0.0253	+0.0163	33.3%	54,875	15,273.2
RAG ( Lewis et al., 2021 )	0.587	1.131	+0.0222	+0.0205	25.0%	54,875	2,438.2
ReAct ( Yao et al., 2023 )	0.610	1.152	+0.0167	+0.0035	25.0%	51,022	2,438.1
Reflexion ( Shinn et al., 2023 )	0.606	1.156	+0.0177	+0.0153	25.0%	54,875	2,438.1
Self-RAG ( Asai et al., 2023 )	0.653	1.180	+0.0059	-0.0066	33.3%	51,254	2,427.6

Table 3: Results on the MemoryCD delayed-label benchmark.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Ten-round evolution trajectories on 12 fixed tasks with seed 13. Left: mean normalized utility with task-bootstrap 95% confidence intervals. Right: active-memory tokens. TIDE maintains a bounded 80-item population, whereas Full History and Append-only grow throughout replay.

Figure 6: Diagnostic quality–cost trajectories from Rounds 3–10. The horizontal axis reports exact cumulative generation input tokens, and the vertical axis reports mean normalized utility. Numbers indicate replay rounds. Rounds 1–2 are omitted because per-job generation-token telemetry was not recorded during the original run.

Figure 7: Normalized Compliance, Format, Efficiency, and batch-level Diversity over ten replay rounds. TIDE’s advantage is concentrated in the first three dimensions; its final Diversity is comparable to that of the other methods rather than uniformly higher.

Figure 8: Selectivity of eviction operations over the ten-round E1 replay. Bars report the proportion of all evictions that target injected noisy memories; error bars denote Wilson 95% confidence intervals. TIDE targets injected noise in 9 of 10 evictions, compared with 29 of 120 for Random Replacement.

Method	Mean Utility $\uparrow$	95% CI	Comp. $\uparrow$	Fmt. $\uparrow$	Eff. $\uparrow$	Div. $\uparrow$	Joint Pass $\uparrow$	Memory Tok. $\downarrow$
Full History	0.655	[0.520, 0.774]	0.565	0.779	0.621	0.510	0.442	2,653
Append-only	0.579	[0.466, 0.693]	0.481	0.694	0.562	0.490	0.367	2,666
Random Replacement	0.663	[0.503, 0.800]	0.594	0.771	0.623	0.438	0.500	2,423
TIDE	0.809	[0.698, 0.909]	0.713	0.915	0.800	0.500	0.617	2,121

Appendix

Table 4: Final-round generation quality and memory cost after ten feedback rounds. Mean Utility averages normalized compliance, format, and efficiency scores; 95% CIs are computed by task-level bootstrap over 12 fixed tasks.

Figure 9: Credit and memory-operator ablations on 12 fixed tasks with seed 13. Points show the paired task-level change in normalized Mean Utility relative to Full TIDE; horizontal bars denote task-bootstrap 95% confidence intervals. Negative values indicate degradation. Responsibility credit is the only ablation whose interval excludes zero.

Variant	Mean Utility $\uparrow$	$\Delta$ vs. Full $\uparrow$	95% CI of $\Delta$	Joint Pass $\uparrow$	Diversity $\uparrow$
Full TIDE	0.852	0.000	[0.000, 0.000]	0.658	0.562
w/o Temporal Credit	0.846	-0.006	[-0.069, 0.058]	0.725	0.708
w/o Semantic Credit	0.838	-0.015	[-0.106, 0.063]	0.650	0.615
w/o Responsibility Credit	0.703	-0.149	[-0.262, -0.044]	0.292	0.469
w/o Revise	0.840	-0.012	[-0.082, 0.056]	0.667	0.677
w/o Merge	0.813	-0.040	[-0.142, 0.053]	0.617	0.667

Appendix

Table 5: Credit and memory-operator ablations. Differences and confidence intervals are computed relative to Full TIDE using a paired bootstrap over the same 12 tasks.

Figure 10: Quality–diversity trajectories over 11 memory snapshots on 12 fixed tasks with seed 13. Color indicates the replay round, and marker size indicates active-memory tokens. TIDE attains the highest final utility with fewer memory tokens. Its lower embedding-space dispersion is interpreted jointly with Effective Population Size and batch-level Diversity because the feasible language space is tightly constrained.

Figure 11: Population-health diagnostics across replay rounds. We report Near-duplicate Rate, Effective Population Size, active population size, and Memory Churn. TIDE keeps its active population bounded at 80 items and retains a substantially larger effective population than the baselines at Round 10 without unbounded token growth.

Method	Mean Utility $\uparrow$	Semantic Diversity $\uparrow$	Near-duplicate Rate $\downarrow$	Effective Population $\uparrow$	Active Items	Memory Tokens $\downarrow$
Full History	0.655	0.196	0.113	29.19	90	2,653
Append-only	0.579	0.199	0.112	32.77	90	2,666
Random Replacement	0.663	0.202	0.111	33.86	80	2,423
TIDE	0.809	0.174	0.131	60.80	80	2,121

Appendix

Table 6: Round-10 population quality and health. Utility and population statistics are averaged over the same 12 fixed tasks. Semantic Diversity and Near-duplicate Rate measure surface-level embedding dispersion within a tightly constrained material space; they are diagnostics rather than complete measures of functional diversity.

Figure 12: Recovery after a controlled distribution shift. Round 0 denotes the change point; curves show normalized future utility with task-level bootstrap 95% confidence intervals. Results use 12 fixed tasks and seed 13.

Figure 13: Task-level relationship between post-shift stale-memory survival and recovery time. Small deterministic jitter is used only to reveal overlapping observations; all reported statistics use the exact values. TIDE achieves rapid recovery despite high stale-memory survival, indicating that its adaptation advantage does not depend on aggressive memory deletion.

Method	Immediate Drop $\downarrow$	Recovery Time $\downarrow$	Post-shift AUC $\uparrow$	Post-shift MEG $\uparrow$	Stale-memory Survival $\downarrow$
Frozen Memory	0.088	1.25	0.759	+0.000	1.000
Full History	0.045	1.58	0.794	+0.035	1.000
Append-only	0.038	0.75	0.824	+0.065	1.000
Random Replacement	0.043	0.67	0.796	+0.037	0.906
TIDE	0.034	0.33	0.833	+0.074	0.996

Appendix

Table 7: Adaptation and recovery after the controlled distribution shift. Recovery Time measures the number of rounds required to return to 95% of pre-shift utility, and Post-shift MEG is measured relative to Frozen Memory.

Figure 14: Task-level post-shift utility-AUC difference between TIDE and each baseline. Positive cells favor TIDE. Task aliases T1–T12 preserve the fixed paired evaluation units without exposing scene identifiers.

Figure 15: Paired effect of delaying TIDE feedback by one or two rounds. Each point represents a fixed future task; horizontal segments denote means across tasks. Values below zero indicate lower Post-shift AUC than under the no-delay TIDE condition.

Explore similar work

Self-Evolving Memory for Generative Recommendation
Sep 14, 2026Xinyu Lin, Zhuosong Jiang, Zixiao Suo +12Real-World Content Recommendation Problem Recommendation
National University of Singapore · Meta AI
Capability-Driven Self-Evolution of Agent Memory
Oct 5, 2026Yaoqi Chen, Yuru Feng, Qianxi Zhang +8Self-Evolving Agents
University of Science and Technology of China · Microsoft · University of California, San Diego
GenMem: Generative Symbolic Memory for Self-Evolving Harness
Sep 28, 2026Xinke Jiang, Tao Feng, Weixuan Xu +7Agentic Memory Nativemem
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +2

Sep 14, 2026, cs.IRJ/K move · Enter open · S save

Self-Evolving Memory for Generative Recommendation

Xinyu Lin, Zhuosong Jiang, Zixiao Suo, Siqin Wang+11

National University of Singapore · Meta AI

Real-World Content Recommendation Problem Recommendation Convergence

Open paper AskPDF arXiv

Generative recommendation has emerged as a promising end-to-end paradigm for personalized recommendation. However, user preferences continuously evolve over time, making self-evolving an essential capability for generative recommender systems. Existing evolving strategies, such as continual retraining and distillation-based adaptation, directly update the shared model parameters using streaming interactions. Nevertheless, we find that directly applying such strategies to generative recommendation introduces a critical issue, termed evolution conflict. Specifically, heterogeneous preference shifts from different users are optimized within a fully shared autoregressive parameter space, causing dominant behavioral patterns to progressively dominate the model evolution process while underrepresented patterns become increasingly overlooked. To address this issue, we propose a self-evolving memory paradigm for generative recommendation, aiming to enable effective evolution across heterogeneous behavioral patterns. We further identify three key principles for effective self-evolving recommendation systems, including isolated memorization, reinforced evolution, and scalable application. Guided by these principles, we develop LION, a simple yet effective framework centered on a sparse Key-Value memory layer. Specifically, LION introduces sparse memory activation to isolate the evolution of different behavioral patterns, while a consolidation loss is designed to reinforce the learning of underrepresented preference dynamics during continual adaptation. Extensive experiments on diverse real-world datasets demonstrate the effectiveness of LION under various continual evolution settings (e.g., per-period evaluation, user/item group evaluation, and evolution convergence analysis). The codes are released at https://github.com/JazyJiang/Self-Evolving-Memory-for-Generative-Recommendation.

Abstract

Explore similar work

Self-Evolving Memory for Generative Recommendation

Capability-Driven Self-Evolution of Agent Memory

GenMem: Generative Symbolic Memory for Self-Evolving Harness