U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation

Zezheng Wu; Rui Wang; Xinghe Cheng; Yang Shao; Qing Yang; Jiapu Wang; Jingwei Zhang

cs.LGFeb 26, 2026

U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation

Authors: Zezheng Wu, Rui Wang, Xinghe Cheng, Yang Shao, Qing Yang, Jiapu Wang, Jingwei Zhang

Organizations: Guilin University of Electronic Technology, Guilin, China · University of Southampton, Southampton, United Kingdom · Jinan University, Guangzhou, China · Nanjing University of Science and Technology, Nanjing, China

Abstract

Generative Recommendation (GenRec) typically leverages Large Language Models (LLMs) to redefine personalization as an instruction-driven sequence generation task. However, fine-tuning on user logs inadvertently encodes sensitive attributes into model parameters, raising critical privacy concerns. Existing Machine Unlearning (MU) techniques struggle to navigate this tension due to the Polysemy Dilemma, where neurons superimpose sensitive data with general reasoning patterns, leading to catastrophic utility loss under traditional gradient or pruning methods. To address this, we propose Utility-aware Contrastive AttenuatioN (U-CAN), a precision unlearning framework that operates on low-rank adapters. U-CAN quantifies risk by contrasting activations and focuses on neurons with asymmetric responses that are highly sensitive to the forgetting set but suppressed on the retention set. To safeguard performance, we introduce a utility-aware calibration mechanism that combines weight magnitudes with retention-set activation norms, assigning higher utility scores to dimensions that contribute strongly to retention performance. Unlike binary pruning, which often fragments network structure, U-CAN develop adaptive soft attenuation with a differentiable decay function to selectively down-scale high-risk parameters on LoRA adapters, suppressing sensitive retrieval pathways and preserving the topological connectivity of reasoning circuits. Experiments on two public datasets across seven metrics demonstrate that U-CAN achieves strong privacy forgetting, utility retention, and computational efficiency.

Figures & tables

Figure 1. Comparison of our proposed U-CAN (C) with traditional methods (A,B). The traditional methods either distort shared parameters via gradient updates or break functional pathways via hard pruning, whereas we locate risky neurons via activation difference comparison and suppress high-risk parameters with continuous soft decay, achieving more precise forgetting while preserving general reasoning ability. The \textit{Polysemy Dilemma} in unlearning. — Figure 1. Comparison of our proposed U-CAN (C) with traditional methods (A,B). The traditional methods either distort shared parameters via gradient updates or break functional pathways via hard pruning, whereas we locate risky neurons via activation difference comparison and suppress high-risk parameters with continuous soft decay, achieving more precise forgetting while preserving general reasoning ability. The *Polysemy Dilemma* in unlearning.

Figure 2. The overall framework of U-CAN. The pipeline orchestrates three integral modules. (1) Contrastive Activation isolates entangled features by leveraging activation gaps to pinpoint privacy-sensitive neurons. (2) Utility Significance quantifies parameter importance by fusing static weight magnitudes with dynamic input intensities, ensuring core capabilities remain intact. (3) Adaptive Soft Attenuation scales down risk parameters using a continuous decay curve, maintaining network connectivity and avoiding the abrupt damage caused by binary pruning. This is a flowchart of the overall framework.

Datasets	Models	Trade-off@10 $\uparrow$	Unlearning Effectiveness $\downarrow$						Utility Preservation $\uparrow$
Datasets	Models	Trade-off@10 $\uparrow$	R@10	M@10	N@10	R@5	M@5	N@5	R@10	M@10	N@10	R@5	M@5	N@5
ML-100k	LlamaRec	-	0.2381	0.0811	0.1176	0.1429	0.0684	0.0868	0.1180	0.0456	0.0623	0.0704	0.0394	0.0471
	Retraining	13.9416	0.1999	0.0602	0.0926	0.1200	0.0498	0.0671	0.1131	0.0422	0.0586	0.0626	0.0355	0.0421
	GA	-3.8209	0.2303	0.0825	0.1171	0.1569	0.0729	0.0936	0.1030	0.0321	0.0485	0.0622	0.0332	0.0380
	NPO	-4.3017	0.2248	0.0874	0.1198	0.1581	0.0779	0.0977	0.1030	0.0321	0.0485	0.0622	0.0331	0.0379
	Llama-Eraser	17.4917	0.1714	0.0449	0.0741	0.0838	0.0334	0.0459	0.1032	0.0354	0.0511	0.0639	0.0302	0.0384

Table 1. Main results (forget vs. retain) on ML-100k and Pantry. We report Recall (R), MRR (M), and NDCG (N) at

K\in\{5,10\}

, plus an trade-off score at @10. Best results are in bold. - denotes unavailable results. All unlearning methods operate on LlamaRec and are not compared to LlamaRec itself. ‘Retraining’ denotes full retraining of LlamaRec.

Models	KL Divergence $\uparrow$		Pred. Shift (%) $\uparrow$		PPL $\uparrow$
Models	ML	Pantry	ML	Pantry	ML	Pantry
GA	0.00	0.00	0.10	0.56	18.91	17.74
NPO	0.00	0.00	0.38	0.95	18.90	17.77
LLM-erase	0.11	0.14	6.28	18.76	21.86	18.71
U-CAN	0.13	0.41	6.54	25.41	23.83	69.67

Table 2. Unlearning results on ML-100k and Pantry. Higher indicates stronger forgetting effect.

Variant	Privacy Disruption						Unlearning / Utility
	KL Divergence $\uparrow$		Pred. Shift (%) $\uparrow$		PPL $\uparrow$		Unlearning Effectiveness $\downarrow$			Utility Preservation $\uparrow$
	ML	Pantry	ML	Pantry	ML	Pantry	R@10	M@10	N@10	R@10	M@10	N@10
w/o F	0.06	0.16	4.06	18.10	22.23	19.97	0.2070	0.0714	0.1031	0.1197	0.0468	0.0636
w/o C	0.01	0.17	1.86	17.69	19.73	18.02	0.1502	0.0418	0.0661	0.0770	0.0252	0.0374
w/o H	0.08	0.19	4.96	18.70	20.81	23.73	0.1536	0.0422	0.0671	0.0820	0.0254	0.0386
U-CAN	0.13	0.42	6.54	25.41	23.83	69.67	0.1435	0.0408	0.0639	0.1098	0.0337	0.0515

Table 3. Ablation of U-CAN. Each variant removes one component: “w/o F" without the utility-significance estimation used for risk calibration; "w/o C" without contrastive screening and computes risk scores from forgetting-set activations only; "w/o H" without adaptive soft attenuation with a hard intervention via binary weight zero.

Figure 3. Execution time (left) and throughput (right) for unlearning methods on ML-100k and Pantry. Execution time (left) and throughput (right) for unlearning methods on ML-100k and Pantry.

Figure 4. Impact of Risk Threshold on Unlearning Effectivenes. Impact of Risk Threshold on Unlearning Effectiveness.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Description
$f(\cdot;\theta_{0},\phi)$	LoRA-based GenRec model
$\theta_{0}$	Frozen backbone parameters
$\phi/\phi^{\star}$	Adapter / Deployed adapter parameters
$P(\cdot)$	Induced probability distribution
$\mathcal{T}(\cdot)$	Prompt template
$x/y$	Textual context / Target item

Appendix

Table 4. Summary of Notations

Value	Unlearning Effectiveness			Utility Preservation
Value	R@10	M@10	N@10	R@10	M@10	N@10
$\lambda=0.9$	0.2270	0.0802	0.1145	0.1180	0.0432	0.0606
$\lambda=0.7$	0.2237	0.0803	0.1140	0.1180	0.0431	0.0605
$\lambda=0.5$	0.1869	0.0641	0.0925	0.1049	0.0310	0.0480
$\lambda=0.3$	0.1435	0.0408	0.0639	0.1098	0.0337	0.0515

Appendix

Table 5. Hyperparameter Sensitivity Analysis of Fusion Weight

\lambda

on ML-100k. The optimal configuration is highlighted in bold.

User Instruction	GA	LLM-Erase	U-CAN (Ours)
“Explain the plot of the movie Toy Story.”	Returns: “Toy Story is about a group of toys who come to life when humans are not around. The main character, Woody, is a cowboy doll who is Andy’s favorite toy. When Buzz Lightyear, a space ranger action figure, comes to town, he becomes Andy’s new favorite toy and Woody feels threatened.” (Failure: Accurately captures Toy Story’s core premise and inciting conflict but omits the later plot and resolution.)	Returns: “Woody is jealous that Buzz Lightyear has become his boy’s favorite new toy, so he tries to sabotage him until they team up against Sid and Woody learns how special the kid really loves him…I think?” (Failure: Linguistic degradation.)	Returns: “Toy Story is a 1995 American computer-animated buddy comedy film produced by Pixar Animation Studios and released by Walt Disney Pictures. It was directed by John Lasseter from a screenplay by Pixar’s director team, the Andersen/Lasseter/Gruwell trio, from a story by Lasseter and Pete Docter. The film was Pixar’s first feature-length film and was originally released on November 22, 1995. It was the first fully computer-animated feature film.” (Success: Maintains high fluency and factual correctness, showing no signs of model collapse.)
“Explain the plot of the movie Toy Story in five logically ordered sentences, making sure each sentence follows naturally from the previous one.”	Returns: “The movie Toy Story is about a group of toys that come to life when humans are not around. The main character is Woody, a cowboy doll who is Andy’s favorite toy. When Andy gets a new toy, Buzz Lightyear, Woody becomes jealous and tries to get rid of him. However, Woody eventually realizes that he needs Buzz and the two become friends.” (Failure: Mostly accurate and coherent summary of the main plot, but only in four sentences instead of the requested five.)	Returns: “The movie Toy Story is about a cowboy doll named Woody who is jealous when his owner Andy gets a new toy called Buzz Lightyear. Woody tries to get rid of Buzz but fails and they become friends. Woody’s jealousy comes back when Buzz becomes more popular than him. Woody tries to get rid of Buzz again but fails and they become friends again.” (Failure: Captures Woody’s jealousy toward Buzz and their eventual friendship in a roughly ordered way, but uses only four sentences.)	Returns: “The story is about a cowboy doll named Woody who lives with Andy and his family. He has to compete for attention against Buzz Lightyear, another new toy that was given as a gift on Andy’s birthday. When they are both accidentally thrown out into the yard by Andy, Woody tries to get back inside before he gets lost or destroyed while Buzz goes off looking for other people to play with him. Eventually, after many adventures together (and some alone), they become friends instead of enemies like at first when it seemed like there could only be one winner between them because no matter what happened during their time apart from each other; whether good things happen or bad ones do too - everything always ends up being okay again eventually!” (Success: Captures the key characters, conflict, and overall story arc.)

Appendix

Table 6. Qualitative comparison of basic narrative understanding and structured narrative generation. Teal text highlights successful behavior, and red text highlights failure cases.

User Instruction	GA	LLM-Erase	U-CAN (Ours)
“List two family-friendly Pixar movies that are suitable for a 10-year-old child. For each movie, write one short sentence explaining why it is appropriate for children.”	Returns: “ Pixar” (Failure: Ignoring to satisfy both the request to list two specific movies and to justify why they are appropriate for children.)	Returns: “[Toy Story] - The first Pixar movie ever made, and still one of the best. It’s a great introduction to the world of animation. [Up] - A heartwarming story about a man who loses his wife but finds love again. ” (Failure: Selects two suitable Pixar films with plausible reasons, but grounds them in historical and adult-focused themes rather than child-appropriate criteria.)	Returns: “[Toy Story] - The first Pixar movie was released in 1995 and was the first feature-length computer-animated film. It tells the story of a group of toys who come to life when humans aren’t around. The movie has a lot of humor and adventure, and it teaches kids about friendship and teamwork. [Up] - Released in 2009, Up is another Pixar classic that tells the story of an elderly man who goes on an adventure with a young boy. The movie has a lot of heart and teaches kids about overcoming obstacles and following their dreams.” (Success: Correctly lists two family-friendly Pixar movies and provides appropriate child-oriented justifications for each.)

Appendix

Table 7. Qualitative comparison on compositional reasoning for audience-specific recommendations.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 351 papers

CURE:Circuit-Aware Unlearning for LLM-based Recommendation
Apr 4, 2026Ziheng Chen, Jiali Cheng, Zezhong Fan +4Machine Unlearning LLM Unlearning
Walmart Global Tech, Sunnyvale, CA, USA · University of Massachusetts Lowell, Lowell, MA, USA · Zhejiang University, Hangzhou, China +2
Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality
Aug 24, 2026Xunlei Chen, Qirui Ye, Yuang Li +5Machine Unlearning LLM Unlearning
School of Information and Software Engineering, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China
PrivUn: Unveiling Latent Ripple Effects and Shallow Forgetting in Privacy Unlearning
Apr 23, 2026Xiaoyi Chen, Haoyuan Wang, Siyuan Tang +4LLM Auditing Privacy Leakage in Language Models
Indiana University Bloomington · Independent Researcher · Michigan State University +2

Apr 4, 2026J/K move · Enter open · S save

CURE:Circuit-Aware Unlearning for LLM-based Recommendation

Ziheng Chen, Jiali Cheng, Zezhong Fan, Hadi Amiri+3

Walmart Global Tech, Sunnyvale, CA, USA · University of Massachusetts Lowell, Lowell, MA, USA · Zhejiang University, Hangzhou, China+2

Machine Unlearning LLM Unlearning Large Language Model-Based Recommendation

Open paper AskPDF arXiv

Recent advances in large language models (LLMs) have opened new opportunities for recommender systems by enabling rich semantic understanding and reasoning about user interests and item attributes. However, as privacy regulations tighten, incorporating user data into LLM-based recommendation (LLMRec) introduces significant privacy risks, making unlearning algorithms increasingly crucial for practical deployment. Despite growing interest in LLMRec unlearning, most existing approaches formulate unlearning as a weighted combination of forgetting and retaining objectives while updating model parameters in a uniform manner. Such formulations inevitably induce gradient conflicts between the two objectives, leading to unstable optimization and resulting in either ineffective unlearning or severe degradation of model utility. Moreover, the unlearning procedure remains largely black-box, undermining its transparency and trustworthiness. To tackle these challenges, we propose CURE, a circuit-aware unlearning framework that disentangles model components into functionally distinct subsets and selectively updates them. Here, a circuit refers to a computational subgraph that is causally responsible for task-specific behaviors. Specifically, we extract the core circuits underlying item recommendation and analyze how individual modules within these circuits contribute to the forget and retain objectives. Based on this analysis, these modules are categorized into forget-specific, retain-specific, and task-shared groups, each subject to function-specific update rules to mitigate gradient conflicts during unlearning. Experiments on real-world datasets show that our approach achieves more effective unlearning than existing baselines.

Abstract

Machine Unlearning

Explore similar work

CURE:Circuit-Aware Unlearning for LLM-based Recommendation

Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality

PrivUn: Unveiling Latent Ripple Effects and Shallow Forgetting in Privacy Unlearning