Prototype-Aligned Federated Soft-Prompts for Continual Web Personalization

Canran Xiao; Liwei Hou

cs.LGMar 29, 2026

Prototype-Aligned Federated Soft-Prompts for Continual Web Personalization

Authors: Canran Xiao, Liwei Hou

Organizations: Shenzhen Campus of Sun Yat-sen University Shenzhen, Guangdong, China · Hunan University Changsha, Hunan, China

Abstract

Continual web personalization is essential for engagement, yet real-world non-stationarity and privacy constraints make it hard to adapt quickly without forgetting long-term preferences. We target this gap by seeking a privacy-conscious, parameter-efficient interface that controls stability-plasticity at the user/session level while tying user memory to a shared semantic prior. We propose ProtoFed-SP, a prompt-based framework that injects dual-timescale soft prompts into a frozen backbone: a fast, sparse short-term prompt tracks session intent, while a slow long-term prompt is anchored to a small server-side prototype library that is continually refreshed via differentially private federated aggregation. Queries are routed to Top-M prototypes to compose a personalized prompt. Across eight benchmarks, ProtoFed-SP improves NDCG@10 by +2.9% and HR@10 by +2.0% over the strongest baselines, with notable gains on Amazon-Books (+5.0% NDCG vs. INFER), H&M (+2.5% vs. Dual-LoRA), and Taobao (+2.2% vs. FedRAP). It also lowers forgetting (AF) and Steps-to-95% and preserves accuracy under practical DP budgets. Our contribution is a unifying, privacy-aware prompting interface with prototype anchoring that delivers robust continual personalization and offers a transparent, controllable mechanism to balance stability and plasticity in deployment.

Figures & tables

Figure 1. ProtoFed-SP: Prototype-Aligned Federated Soft-Prompt Personalization. Left (Client side). Given a user query or interaction history, the client encodes the context into a pooled embedding and routes it to a small set of Top- $M$ population prototypes via similarity-based retrieval. A dual-timescale prompt state is maintained locally: a long-term prompt capturing stable preferences and a short-term prompt modeling session-level intent. The routed prompt is composed by summing the long-term prompt, a drift-adaptive short-term prompt, and a weighted mixture of retrieved prototypes, and is injected into a frozen backbone for ranking. Gradients from the recommendation loss (BCE/BPR) update only the prompt parameters, with sparse proximal updates for the short-term prompt and prototype-aligned updates for the long-term prompt. Right (Server side). Clients periodically upload compressed long-term prompt embeddings protected by clipping and Gaussian noise. The server performs differentially private aggregation (e.g., DP-FedKMeans or robust alternatives) to update a compact prototype library, with momentum updates, prototype separation, and pruning to prevent collapse. The refreshed prototypes are broadcast back to clients, enabling privacy-conscious, parameter-efficient continual personalization with explicit stability–plasticity control.

	Books		Electronics		ML-20M		Yelp		RetailRocket		Gowalla		Taobao		H&M
Method	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10
FullRetrain	0.114	0.231	0.133	0.257	0.319	0.644	0.109	0.220	0.244	0.466	0.205	0.514	0.217	0.438	0.262	0.487
FineTune-Last	0.102	0.208	0.122	0.240	0.302	0.616	0.101	0.206	0.232	0.446	0.195	0.498	0.208	0.421	0.251	0.472
EWC	0.109	0.223	0.128	0.248	0.307	0.625	0.106	0.214	0.236	0.454	0.198	0.505	0.211	0.428	0.255	0.476
SI	0.108	0.221	0.127	0.246	0.306	0.623	0.105	0.213	0.235	0.452	0.197	0.503	0.207	0.420	0.251	0.472
MAS	0.109	0.222	0.127	0.247	0.305	0.622	0.105	0.213	0.235	0.452	0.197	0.504	0.210	0.427	0.254	0.476

Table 1. Overall Top-

K

accuracy. NDCG@10 / HR@10 on eight benchmarks: Amazon-Books , Amazon-Electronics , MovieLens-20M , Yelp , RetailRocket , Gowalla , Taobao , and H&M .

Figure 2. Core hyperparameter sensitivity (NDCG@10, mean

\pm

std). One parameter varied per block; others fixed at defaults.

	Amazon-Books		ML-20M		Taobao		H&M
Method	N@10	HR@10	N@10	HR@10	N@10	HR@10	N@10	HR@10
ProtoFed-SP (Full)	0.126	0.248	0.329	0.658	0.231	0.461	0.284	0.512
w/o Prototype Alignment $\mathcal{A}$	0.120 (-0.006)	0.238 (-0.010)	0.324 (-0.005)	0.649 (-0.009)	0.225 (-0.006)	0.451 (-0.010)	0.277 (-0.007)	0.501 (-0.011)
w/o Short-term prompt ( $\alpha\!=\!0$ )	0.122 (-0.004)	0.241 (-0.007)	0.325 (-0.004)	0.652 (-0.006)	0.227 (-0.004)	0.454 (-0.007)	0.279 (-0.005)	0.504 (-0.008)
w/o Long-term prompt (short only)	0.116 (-0.010)	0.232 (-0.016)	0.320 (-0.009)	0.644 (-0.014)	0.223 (-0.008)	0.448 (-0.013)	0.274 (-0.010)	0.496 (-0.016)
Static Prototypes (no federated update)	0.121 (-0.005)	0.240 (-0.008)	0.326 (-0.003)	0.653 (-0.005)	0.226 (-0.005)	0.453 (-0.008)	0.278 (-0.006)	0.502 (-0.010)

Table 2. Single-factor ablations. NDCG@10 / HR@10 on four representative datasets. (

\Delta

) is the decrease w.r.t. ProtoFed-SP (Full) .

Panel of Figure 3 — Figure 3. Stability–plasticity verification . Prototype alignment consistently lowers forgetting, while short-term prompts markedly reduce the steps needed to adapt; combining both yields the best AF–NDCG frontier.

Panel of Figure 4 — Figure 4. Privacy–utility trade-off. Across datasets, moderate DP (e.g., $\sigma\!=\!0.4$ , mid $\varepsilon$ ) preserves personalization: NDCG stays within the Non-DP band while AF increases only slightly; very strong DP ( $\sigma\!=\!0.8$ ) degrades gracefully rather than catastrophically.

Panel of Figure 5 — Figure 5. High-drift users see the largest, most significant gains; cold-start users also benefit. Heavy users gain, but improvements are not exclusive to them.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

Hypotheses-Guided Self Distillation for Continual Personalization
Aug 31, 2026EunJeong Hwang, Kushan Mitra, Dan Zhang +2Continual Learning for LLMs LLM Personalization
University of British Columbia · Work done as an intern at Megagon Labs. · Megagon Labs, USA
SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generation
Jan 15, 2026Seoyeon Kim, Jaehyung KimContinual Learning for LLMs LLM Personalization
Yonsei University
ProTTT: Learning to Learn Semantic User Memory with Test-Time Training
Sep 26, 2026Sejun Park, Hyoungjo Bhang, Hyein Jeong +1Personalized Language Models Persistent Memory for Language Models
Graduate School of Data Science, Seoul National University

Aug 31, 2026J/K move · Enter open · S save

Hypotheses-Guided Self Distillation for Continual Personalization

EunJeong Hwang, Kushan Mitra, Dan Zhang, Hannah Kim+1

University of British Columbia · Work done as an intern at Megagon Labs. · Megagon Labs, USA

Continual Learning for LLMs LLM Personalization User Preference Modeling

Open paper AskPDF arXiv

As people increasingly interact with LLM assistants in daily life, continually adapting to individual preferences has become essential for effective long-term interactions. However, user preferences are rarely stated in full, and instead emerge through heterogeneous, latent, and noisy signals, with existing methods relying on raw interaction histories or costly reward-based optimization to manage personalization. We introduce HypReflect, a reliable, scalable framework for continual personalization that infers explicit, uncertainty-aware preference hypotheses from diverse user signals, reflectively refines them as new evidence accumulates, and incorporates the resulting user model through hypotheses-guided self-distillation. Experiments across three personalization settings: online personalization, multi-session interactions, and implicit behavioral signals, show that HypReflect outperforms a range of baselines, including raw-history and incremental-update methods. We further demonstrate strong generalization to unseen users and cross-domain settings, along with stability across context budgets, reusable hypotheses, and more focused personalization. These results suggest a step towards reliable and scalable continual personalization through explicit, revisable user preference hypotheses.

Abstract

Recommender Systems

Explore similar work

Hypotheses-Guided Self Distillation for Continual Personalization

SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generation

ProTTT: Learning to Learn Semantic User Memory with Test-Time Training