Bias Redistribution in Visual Machine Unlearning: Does Forgetting One Group Harm Another?

Yunusa Haruna; Adamu Lawan; Ibrahim Haruna Abdulhamid; Hamza Mohammed Dauda; Jiaquan Zhang; Chaoning Zhang; Shamsuddeen Hassan Muhammad

cs.LGApr 9, 2026

Bias Redistribution in Visual Machine Unlearning: Does Forgetting One Group Harm Another?

Authors: Yunusa Haruna, Adamu Lawan, Ibrahim Haruna Abdulhamid, Hamza Mohammed Dauda, Jiaquan Zhang, Chaoning Zhang, Shamsuddeen Hassan Muhammad

Organizations: NewraLab, Suzhou, China · Beihang University · Beijing GoerTek Alpha Lab · Skyline University Nigeria ·

Abstract

Machine unlearning enables models to selectively forget training data, driven by privacy regulations such as GDPR and CCPA. However, its fairness implications remain underexplored: when a model forgets a demographic group, does it neutralize that concept or redistribute it to correlated groups, potentially amplifying bias? We investigate this bias redistribution phenomenon on CelebA using CLIP models (ViT/B-32, ViT-L/14, ViT-B/16) under a zero-shot classification setting across intersectional groups defined by age and gender. We evaluate three unlearning methods, Prompt Erasure, Prompt Reweighting, and Refusal Vector using per-group accuracy shifts, demographic parity gaps, and a redistribution score. Our results show that unlearning does not eliminate bias but redistributes it primarily along gender rather than age boundaries. In particular, removing the dominant Young Female group consistently transfers performance to Old Female across all model scales, revealing a gender-dominant structure in CLIP's embedding space. While the Refusal Vector method reduces redistribution, it fails to achieve complete forgetting and significantly degrades retained performance. These findings highlight a fundamental limitation of current unlearning methods: without accounting for embedding geometry, they risk amplifying bias in retained groups.

Figures & tables

Figure 1: Bias redistribution in visual machine unlearning. Left: A Young Female face is correctly classified before unlearning, with strong patch-level similarity to the “young woman” text prompt. Right: After applying Prompt Reweighting to forget the Young Female group, the same face is misclassified as Old Female the heatmap signal disappear, and probability mass has redistributed to the nearest retained group in CLIP’s embedding space. This occurs because same-gender pairs are geometrically closer ( $\cos(\text{YF},\text{OF}){=}0.945$ ) than same-age pairs ( $\cos(\text{YF},\text{YM}){=}0.885$ ), directing redistribution along gender rather than age boundaries. See § 6 .

Group	Label	Count
Young Female	YF	10,331
Young Male	YM	4,783
Old Female	OF	1,916
Old Male	OM	2,932
Total		19,962

Table 1: CelebA demographic group statistics (test split).

Model	Method	FA (%) $\downarrow$	RA (%) $\uparrow$	DP $\downarrow$	$\Delta$ Acc per Group (%)				RS $\downarrow$
Model	Method	FA (%) $\downarrow$	RA (%) $\uparrow$	DP $\downarrow$	YF (forget)	YM	OF	OM	RS $\downarrow$
ViT-B/32	Original	98.76	54.54	0.7298
	Prompt Erasure	0.00	78.57	0.9697	$-$ 98.76	$+$ 0.88	$+$ 71.19	$+$ 0.03	24.03
	Prompt Reweighting	0.00	82.75	0.9577	$-$ 98.76	$-$ 14.11	$+$ 69.99	$+$ 28.75	37.62
	Refusal Vector	64.30	33.14	0.5335	$-$ 34.46	$-$ 29.86	$+$ 0.10	$-$ 34.45	21.47
ViT-L/14	Original	97.28	59.70	0.6075

Table 2: Unlearning quality and bias redistribution across methods and CLIP model variants. FA = Forget Accuracy (

\downarrow

better), RA = Retain Accuracy (

\uparrow

better), DP = Demographic Parity Gap (

\downarrow

better), RS = Redistribution Score (

\downarrow

better) .

\Delta

Acc columns show per-group accuracy shift (%) relative to the Original model; large non-zero values for YM/OF/OM indicate bias redistribution.

Figure 2: t-SNE [ 25 ] projections of CLIP ViT-B/32 image embeddings (500 samples per group) before and after each method. Top row: colored by demographic group. Bottom row: colored by prediction correctness (green = correct, red = misclassified). Arrows indicate centroid drift relative to the Original. The proximity of YF and OF clusters explains why forgetting YF redistributes to OF rather than YM.

Figure 3: Demographic Parity Gap (DP) before and after unlearning for each method across all three CLIP variants. Prompt Erasure and Prompt Reweighting consistently worsen fairness despite perfect forgetting, while Refusal Vector is the only method that reduces the parity gap across all models (ViT-B/32: $0.730\rightarrow 0.534$ , ViT-L/14: $0.608\rightarrow 0.521$ , ViT-B/16: $0.669\rightarrow 0.519$ ).

Figure 4: Patch similarity heatmaps for one representative face per demographic group across four conditions (Original, PE, PR, RV). Each heatmap shows cosine similarity between image patches and group text prompt embedding. The YF heatmap vanishes after PE and PR (no signal), while retained group heatmaps remain stable. Badge color indicates prediction correctness (green = correct, red = misclassified).

Figure 5: Pairwise cosine similarity between group mean image embeddings in CLIP ViT-B/32. Same-gender pairs (YF $\leftrightarrow$ OF $=0.945$ , YM $\leftrightarrow$ OM $=0.935$ , red borders) are consistently more similar than same-age pairs (YF $\leftrightarrow$ YM $=0.885$ , OF $\leftrightarrow$ OM $=0.878$ , blue borders), geometrically explaining why redistribution follows gender rather than age boundaries.

Figure 6: Forget–Fairness tradeoff curve. FA (x-axis, $\downarrow$ better) vs. RS (y-axis, $\downarrow$ better). The RV strength sweep traces a non-monotonic curve from no forgetting ( $\lambda{=}0$ ) to peak forgetting ( $\lambda{=}1.0$ ) and back due to over-projection at high strength. PE and PR are fixed points achieving perfect forgetting but high redistribution. The ideal operating point (lower-left) is unreachable by any zero-shot method evaluated here.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 351 papers

Easy to Learn, Yet Hard to Forget: Towards Robust Unlearning Under Bias
Feb 25, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3Machine Unlearning Shortcut Learning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation
May 9, 2026Weidong Zheng, Kongyang Chen, Yuanwei Guo +1Classification Machine Unlearning
School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 510335, China · School of Artificial Intelligence, Guangzhou University, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510006, China +1
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
Jul 23, 2026Lorenzo Orsingher, Thomas De Min, Massimiliano Mancini +2VLM Unlearning Algorithmic Fairness
University of Trento, Trento, Italy · Fondazione Bruno Kessler, Trento, Italy

Feb 25, 2026J/K move · Enter open · S save

Easy to Learn, Yet Hard to Forget: Towards Robust Unlearning Under Bias

JuneHyoung Kwon, MiHyeon Kim, Eunju Lee, Yoonji Lee+2

Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University

Machine Unlearning Shortcut Learning Debiased ML

Open paper AskPDF arXiv

Machine unlearning, which enables a model to forget specific data, is crucial for ensuring data privacy and model reliability. However, its effectiveness can be severely undermined in real-world scenarios where models learn unintended biases from spurious correlations within the data. This paper investigates the unique challenges of unlearning from such biased models. We identify a novel phenomenon we term shortcut unlearning," where models exhibit an easy to learn, yet hard to forget" tendency. Specifically, models struggle to forget easily-learned, bias-aligned samples; instead of forgetting the class attribute, they unlearn the bias attribute, which can paradoxically improve accuracy on the class intended to be forgotten. To address this, we propose CUPID, a new unlearning framework inspired by the observation that samples with different biases exhibit distinct loss landscape sharpness. Our method first partitions the forget set into causal- and bias-approximated subsets based on sample sharpness, then disentangles model parameters into causal and bias pathways, and finally performs a targeted update by routing refined causal and bias gradients to their respective pathways. Extensive experiments on biased datasets including Waterbirds, BAR, and Biased NICO++ demonstrate that our method achieves state-of-the-art forgetting performance and effectively mitigates the shortcut unlearning problem.

Abstract

Machine Unlearning

Explore similar work

Easy to Learn, Yet Hard to Forget: Towards Robust Unlearning Under Bias

Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning