Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models

Ivan Luiz De Moura Matos; Abdel Djalil Sad Saoud; Ekaterina Iakovleva; Vito Paolo Pastore; Enzo Tartaglione

cs.LGMar 5, 2026

Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models

Authors: Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva, Vito Paolo Pastore, Enzo Tartaglione

Organizations: LTCI, Télécom Paris, Institut Polytechnique de Paris, France · MaLGa-DIBRIS, University of Genoa, Italy · AI for Good (AIGO), Istituto Italiano di Tecnologia, Italy

Abstract

The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or dataset manipulation. However, an intriguing question arises: is it possible to extract fair and bias-agnostic subnetworks from standard vanilla-trained models without relying on additional data, such as unbiased training set? In this work, we introduce Bias-Invariant Subnetwork Extraction (BISE), a learning strategy that identifies and isolates "bias-free" subnetworks that already exist within conventionally trained models, without retraining or finetuning the original parameters. Our approach demonstrates that such subnetworks can be extracted via pruning and can operate without modification, effectively relying less on biased features and maintaining robust performance. Our findings contribute towards efficient bias mitigation through structural adaptation of pre-trained neural networks via parameter removal, as opposed to costly strategies that are either data-centric or involve (re)training all model parameters. Extensive experiments on common benchmarks show the advantages of our approach in terms of the performance and computational efficiency of the resulting debiased model.

Figures & tables

Figure 1: Overview of the proposed method. BISE aims to extract an unbiased subnetwork from the biased vanilla-trained network.

Figure 2: Illustration of BISE. Solid black arrows indicate forward propagation; dashed blue arrows indicate backward gradient propagation. During the training of the mask $\mathcal{M}$ and of $\mathcal{C}_{\text{aux}}$ , the original model, $f=\mathcal{C}\circ\mathcal{E}$ , is kept frozen.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
Rubi [ 14 ]	93.6	43.0	90.4
EnD [ 92 ]	96.0	93.9	83.7
BCon+BBal [ 31 ]	98.1	97.7	97.3
ReBias [ 5 ]	88.4	75.4	65.8

Table 1: Results on BiasedMNIST for different

\rho

values.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
BISE	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$	$29.9_{\pm 7.5}$	$290.8_{\pm 31.0}$	$35.0_{\pm 1.5}$	$269.6_{\pm 6.2}$
BISE (last)	$18.9_{\pm 4.3}$	$336.4_{\pm 17.7}$	$30.0_{\pm 6.8}$	$290.7_{\pm 28.3}$	$33.8_{\pm 4.0}$	$274.6_{\pm 16.7}$

Table 2: Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset.

Method	Accuracy (%)
Method	$\rho=0.995$	$\rho=0.99$	$\rho=0.98$	$\rho=0.95$
Vanilla	$22.03_{\pm 0.39}$	$26.89_{\pm 0.25}$	$33.42_{\pm 0.65}$	$47.18_{\pm 0.34}$
EnD [ 92 ]	19.38	23.12	34.07	36.57
HEX [ 98 ]	13.87	14.81	15.20	16.04
ReBias [ 5 ]	22.27	25.72	31.66	43.43
LfF [ 70 ]	28.57	33.07	39.91	50.27

Table 3: Results on Corrupted-CIFAR10, for different values of

\rho

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.995$		$\rho=0.99$		$\rho=0.98$		$\rho=0.95$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	37.1	0	37.1	0	37.1	0	37.1
BISE (last)	$77.8_{\pm 2.3}$	$15.5_{\pm 0.9}$	$92.2_{\pm 1.0}$	$15.7_{\pm 1.1}$	$89.6_{\pm 0.6}$	$19.2_{\pm 0.8}$	$82.3_{\pm 0.6}$	$22.5_{\pm 0.6}$

Table 4: Sparsity (

S

) and computational cost (MFLOPs) of models, for Corrupted-CIFAR10.

Table 7

Method	Accuracy (%) $\uparrow$
Method	(Alig., Alig.)	(Alig., Conf.)	(Conf., Alig.)	(Conf., Conf.)	Unbiased
Vanilla	$\bf 100.0_{\pm 0.0}$	$\underline{97.2_{\pm 0.8}}$	$30.3_{\pm 1.4}$	$5.2_{\pm 0.3}$	$58.2_{\pm 0.6}$
LfF [ 70 ]	$\underline{99.6}$	4.7	$\bf 98.6$	5.1	52.0
EIIL [ 19 ]	100.0	$\underline{97.2}$	70.8	10.9	69.7
PGI [ 3 ]	98.6	82.6	26.6	9.5	54.3
DebiAN [ 53 ]	100.0	95.6	76.5	16.0	72.0

Table 7: Results on the Multi-Color MNIST dataset. (*) indicates that debiasing is performed by leveraging an unbiased dataset.

Reweighting of $\mathcal{L}_{\text{CE}}(\hat{y},y)$	$\mathcal{I}(\hat{b},b)$	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
-	-	$91.9_{\pm 0.7}$	$5.8_{\pm 2.6}$	$390.9_{\pm 10.6}$
$\checkmark$	-	$96.0_{\pm 0.4}$	$18.5_{\pm 6.5}$	$338.2_{\pm 27.1}$
-	$\checkmark$	$91.1_{\pm 0.5}$	$18.7_{\pm 0.5}$	$337.3_{\pm 2.0}$
$\checkmark$	$\checkmark$	$96.1_{\pm 0.5}$	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$

Table 8: Ablation study for the components of the objective

J

Panel of Figure 3 — Figure 3: Analysis of (a) coefficient $\gamma$ and (b) pruning strategies. Shaded areas indicate the interval of one standard deviation.

Method	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	$76.5_{\pm 2.1}$	0	1818.6
Unsup. BISE	$87.5_{\pm 2.5}$	$67.3_{\pm 4.8}$	$953.1_{\pm 218.4}$
Unsup. BISE (last)	$85.1_{\pm 1.8}$	$67.7_{\pm 4.6}$	$936.5_{\pm 190.1}$

Table 9: Results for unsupervised ( Unsup. ) debiasing on CelebA.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure C.1 : Samples from the BiasedMNIST dataset [ 5 ] . In the first row, bias-aligned images; in the second row, bias-conflicting images.

Figure C.2 : Samples from the CelebA dataset [ 58 ] .

Figure C.3 : Samples from the Corrupted-CIFAR10 dataset [ 30 ] . We show two images belonging to each class.

Figure C.4 : Samples from the Multi-Color MNIST dataset [ 53 ] . In the first row, images that are bias-aligned with respect to both background colors; in the second row, images that are bias-conflicting with respect to at least one of the colors.

Figure C.5 : Samples from the CivilComments dataset [ 11 , 44 ] . Label

y

indicates whether a comment is toxic, while

b

indicates if it mentions any sensitive attribute.

	Biased MNIST	CelebA	Corrupted- CIFAR10	Multi-Color MNIST	Civil Comments
# weights	$531\,210$	$11\,177\,538$	$11\,181\,642$	$256\,510$	$109\,361\,664$
# $m_{i}$	$240\;(0.05\%)$	$3\,840\;(0.03\%)$	$3\,840\;(0.03\%)$	$300\;(0.12\%)$	$46\,848\;(0.04\%)$

Appendix

Table E.1 : Number of weights vs. of updated parameters

m_{i}

Method	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	$10.0_{\pm 0.1}$	0	$415.4$
BISE	$15.7_{\pm 1.3}$	$85.6_{\pm 4.0}$	$59.8_{\pm 16.7}$
BISE + finetuning	$60.5_{\pm 9.4}$	$85.6_{\pm 4.0}$	$59.8_{\pm 16.7}$
BISE (last)	$14.7_{\pm 2.5}$	$78.2_{\pm 2.9}$	$90.5_{\pm 11.9}$
FFW * [ 68 ]	$19.9$	–	–

Appendix

Table G.1 : Results on BiasedMNIST for

\rho=0.999

. (*) indicates that debiasing is performed by leveraging an unbiased dataset.

Metric	$\boldsymbol{E}$
Metric	1	5	10	20	50
Acc. (%) $\uparrow$	$95.9_{\pm 0.4}$	$96.0_{\pm 0.5}$	$96.0_{\pm 0.4}$	$95.7_{\pm 0.5}$	$96.1_{\pm 0.5}$
$S$ (%) $\uparrow$	$18.9_{\pm 7.3}$	$17.6_{\pm 7.1}$	$20.0_{\pm 6.1}$	$17.2_{\pm 7.6}$	$20.9_{\pm 4.4}$
MFLOPs $\downarrow$	$336.7_{\pm 30.4}$	$342.1_{\pm 29.5}$	$331.9_{\pm 25.2}$	$343.7_{\pm 31.6}$	$328.3_{\pm 18.2}$

Appendix

Table H.1 : Effect of

E

on the extracted subnetwork.

Metric	$\boldsymbol{\kappa}$
Metric	0.1	0.5	0.8
Acc. (%) $\uparrow$	$96.0_{\pm 0.4}$	$96.1_{\pm 0.5}$	$96.2_{\pm 0.6}$
$S$ (%) $\uparrow$	$20.6_{\pm 3.9}$	$20.9_{\pm 4.4}$	$18.9_{\pm 7.6}$
MFLOPs $\downarrow$	$329.3_{\pm 16.3}$	$328.3_{\pm 18.2}$	$336.5_{\pm 31.4}$

Appendix

Table H.2 : Effect of

\kappa

on the extracted subnetwork.

Metric	$\boldsymbol{\upsilon}$
Metric	1	5	10	20	50
Acc. (%) $\uparrow$	$95.9_{\pm 0.5}$	$95.8_{\pm 0.5}$	$96.1_{\pm 0.5}$	$96.0_{\pm 0.5}$	$96.5_{\pm 0.3}$
$S$ (%) $\uparrow$	$22.4_{\pm 6.3}$	$18.3_{\pm 6.2}$	$20.9_{\pm 4.4}$	$19.2_{\pm 6.3}$	$19.8_{\pm 8.0}$
MFLOPs $\downarrow$	$322.2_{\pm 26.2}$	$338.9_{\pm 25.7}$	$328.3_{\pm 18.2}$	$335.2_{\pm 26.2}$	$333.0_{\pm 33.3}$

Appendix

Table H.3 : Effect of

\upsilon

on the extracted subnetwork.

Metric	$\boldsymbol{\tau_{\textrm{min}}}$
Metric	$10^{-1}$	$10^{-2}$	$10^{-3}$	$10^{-4}$
Acc. (%) $\uparrow$	$95.9_{\pm 0.6}$	$96.1_{\pm 0.5}$	$96.1_{\pm 0.5}$	$96.1_{\pm 0.5}$
$S$ (%) $\uparrow$	$20.4_{\pm 5.0}$	$20.9_{\pm 4.4}$	$20.9_{\pm 4.4}$	$20.9_{\pm 4.4}$
MFLOPs $\downarrow$	$330.3_{\pm 20.7}$	$328.3_{\pm 18.2}$	$328.3_{\pm 18.2}$	$328.3_{\pm 18.2}$

Appendix

Table H.4 : Effect of

\tau_{\textrm{min}}

on the extracted subnetwork.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
BISE ( Tab. 1 )	$96.1_{\pm 0.5}$	$92.2_{\pm 1.9}$	$90.8_{\pm 0.6}$
BISE + trainable BN (avg., std.)	$97.3_{\pm 0.1}$	$94.9_{\pm 1.1}$	$94.0_{\pm 1.2}$
BISE + trainable BN (avg., std., $\beta$ , $\gamma$ )	$96.9_{\pm 0.4}$	$92.2_{\pm 1.9}$	$90.1_{\pm 3.7}$

Appendix

Table I.1 : Experiments on BiasedMNIST with trainable vs. non-trainable batch normalization layers.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
BISE ( Tab. 2 )	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$	$29.9_{\pm 7.5}$	$290.8_{\pm 31.0}$	$35.0_{\pm 1.5}$	$269.6_{\pm 6.2}$
BISE + trainable BN (avg., std.)	$18.3_{\pm 6.8}$	$339.2_{\pm 28.1}$	$26.5_{\pm 4.0}$	$305.0_{\pm 16.7}$	$33.3_{\pm 7.9}$	$276.9_{\pm 32.9}$
BISE + trainable BN (avg., std., $\beta$ , $\gamma$ )	$32.7_{\pm 7.7}$	$279.5_{\pm 32.0}$	$45.7_{\pm 3.9}$	$225.6_{\pm 16.2}$	$48.2_{\pm 12.0}$	$214.9_{\pm 49.8}$

Appendix

Table I.2 : Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset, when updating the batch normalization layers during BISE.

Figure J.1 : Analysis of the effect of varying the threshold

\zeta

on the pruned network. Threshold

\zeta=0

corresponds to the vanilla dense model;

\zeta=0.5

is the original threshold used in BISE.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
Unsup. BISE	$95.8_{\pm 0.5}$	$92.0_{\pm 2.0}$	$90.4_{\pm 0.6}$
Unsup. BISE (last)	$95.4_{\pm 0.7}$	$90.6_{\pm 1.9}$	$90.0_{\pm 0.2}$

Appendix

Table K.1 : Unsupervised ( Unsup. ) debiasing on BiasedMNIST.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
Unsup. BISE	$19.9_{\pm 4.4}$	$332.4_{\pm 18.2}$	$26.2_{\pm 4.7}$	$306.3_{\pm 19.5}$	$30.9_{\pm 2.3}$	$286.6_{\pm 9.7}$
Unsup. BISE (last)	$19.7_{\pm 3.6}$	$333.4_{\pm 14.9}$	$28.8_{\pm 4.9}$	$295.6_{\pm 20.3}$	$28.8_{\pm 4.5}$	$295.5_{\pm 18.7}$

Appendix

Table K.2 : Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset, in the unsupervised debiasing scenario.

Figure L.1 : Results on BiasedMNIST (training set with

\rho=0.99

) under noisy bias labels. Noise level

p=0\%

corresponds to the standard setting ( i.e. , use of true bias labels).

Model	Accuracy of the aux. classifier on color prediction (%)
Vanilla	$97.6_{\pm 1.0}$
Debiased with BISE	$82.1_{\pm 7.3}$

Appendix

Table M.1 : Comparison of accuracy on color prediction for two auxiliary classifiers: one trained on top of a vanilla encoder, and the other trained on top of the BISE-pruned encoder.

Metric	Model	Dataset
Metric	Model	BiasedMNIST ( $\rho=0.99$ )	Corrupted-CIFAR10 ( $\rho=0.95$ )	CelebA	Multi-Color MNIST ( $\rho_{\text{L}}=0.99$ , $\rho_{\text{R}}=0.95$ )	CivilComments
Acc. (%)	Vanilla	$88.9_{\pm 0.4}$	$47.18_{\pm 0.34}$	$76.5_{\pm 2.1}$	$58.2_{\pm 0.6}$	$59.6_{\pm 2.7}$
	Best competitor	$98.1$ (BCon+BBal [ 31 ] )	$51.13$ (DFA [ 51 ] )	$91.4$ (BCon+BBal [ 31 ] )	$73.1$ (VCBA [ 69 ] )	$80.4$ (Group DRO [ 82 ] )
	BISE ∗	$96.1_{\pm 0.5}$	$55.38_{\pm 1.96}$	$89.7_{\pm 0.8}$	$60.3_{\pm 1.0}$	$80.4_{\pm 0.2}$
	BISE + finetuning ∗	$98.1_{\pm 0.1}$	$57.22_{\pm 1.81}$	$91.8_{\pm 1.3}$	$70.6_{\pm 1.6}$	$81.0_{\pm 0.1}$
$S$ (%)	BISE ∗	$20.9_{\pm 4.4}$	$82.3_{\pm 0.6}$	$67.6_{\pm 0.8}$	$17.1_{\pm 5.3}$	$26.0_{\pm 5.4}$

Appendix

Table N.1 : Summary of results across datasets. ( ∗ ) indicates that the result for BISE “best” is reported whenever a validation set is available; otherwise, we report the result for BISE “last” . The metric Acc. corresponds to the accuracy on an unbiased test set, in the case of BiasedMNIST, CelebA and Corrupted-CIFAR10; for Multi-Color MNIST, it denotes the average test accuracy across the four existing groups in the dataset (as described in Sec. 4.2 ); for CivilComments, it denotes the worst-group test accuracy. The sparsity (

S

) for the vanilla and competing methods is zero, since no parameter is removed from the network.

Explore similar work

Mar 31, 2026

Unbiased Model Prediction Without Using Protected Attribute Information

The problem of bias persists in the deep learning community as models continue to provide disparate performance across different demographic subgroups. Therefore, several algorithms have been proposed to improve the fairness of deep models. However, a majority of these algorithms utilize the protected attribute information for bias mitigation, which severely limits their application in real-world scenarios. To address this concern, we have proposed a novel algorithm, termed as Non-Protected Attribute-based Debiasing (NPAD) algorithm for bias mitigation, that does not require the protected attribute information. The proposed NPAD algorithm utilizes the auxiliary information provided by the non-protected attributes to optimize the model for bias mitigation. Further, two different loss functions, Debiasing via Attribute Cluster Loss (DACL) and Filter Redundancy Loss (FRL) have been proposed to optimize the model for fairness goals. Multiple experiments are performed on the LFWA and CelebA datasets for facial attribute prediction, and a significant reduction in bias across different gender and age subgroups is observed.

Puspita Majumdar, Surbhi Mittal, Saheb Chhabra +2

IIIT-Delhi, New Delhi, India, 110020

Algorithmic Fairness Debiased ML

Ask PDF

Jun 23, 2026

Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes, resulting in poor generalization on minority groups. Bias mitigation remains challenging due to two fundamental issues. First, when group labels are unavailable, existing group-unsupervised methods typically infer spurious attributes implicitly from model behavior, making it difficult to identify spurious factors that are semantically aligned with real-world biases. Second, even with pseudo spurious supervision, most existing debiasing methods follow a single-branch design that operates within a single shared feature space, where target and spurious attributes are intrinsically entangled. To address the first challenge, we introduce Confidence-guided Bias Concept Mining (CBCM), which leverages diffusion-disentangled, semantically grounded concept representations to identify reliable spurious attributes without attribute annotations. To address the second challenge, we propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches and explicitly removes spurious information through cross null-space projection while preserving target-relevant semantics. Extensive experiments on four benchmark datasets show that our method achieves state-of-the-art worst group accuracy among group-unsupervised approaches, while tuning at most 0.22% of the model parameters. The source code is available in the supplementary materials.

Xiangqian Zhao, Xinyang Jiang, Zhipeng Xu +5

Xidian University · Shaanxi, China · Microsoft Research Asia +1

Disentangled Representation Learning Debiased ML Algorithmic Fairness Spurious Correlation Robustness

Ask PDF

Aug 1, 2026

A Heuristic Perspective on Debiasing Language Models

Language models (LMs) often acquire various biases during pre-training and may express them in interactions, potentially causing social harm. Existing methods often rely on counterfactual augmentation or representation projection. These strategies remain limited in practice due to their high computational costs and difficulty in scaling to larger models. Additionally, many of these strategies require manual data annotation, narrowing their scope to specific cultures and bias categories. To overcome these limitations, we propose HEIMAT, a HEurIstic-style autoMATic debiasing framework for LMs. HEIMAT consists of two main steps: bias disclosure and debiasing fine-tuning. In the first step, it uses simple templates to construct heuristic prompts, which are applied to reveal model biases and generate corresponding context prompts. In the second step, it fine-tunes the model by minimizing the Jensen-Shannon divergence of predictions on these context prompts to reduce bias. Extensive experiments show that HEIMAT effectively mitigates bias in different cultures while maintaining the model's natural language understanding (NLU) performance.

Tian Lan, Yemin Wang, Chuancheng Shi +6

Inner Mongolia University · Xiamen University · University of Sydney +3

Social Bias in Language Models Cultural Bias in Language Models Linguistic Bias in Language Models

Ask PDF