Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models

Ivan Luiz De Moura Matos; Abdel Djalil Sad Saoud; Ekaterina Iakovleva; Vito Paolo Pastore; Enzo Tartaglione

cs.LGMar 5, 2026

Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models

Authors: Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva, Vito Paolo Pastore, Enzo Tartaglione

Organizations: LTCI, Télécom Paris, Institut Polytechnique de Paris, France · MaLGa-DIBRIS, University of Genoa, Italy · AI for Good (AIGO), Istituto Italiano di Tecnologia, Italy

Abstract

The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or dataset manipulation. However, an intriguing question arises: is it possible to extract fair and bias-agnostic subnetworks from standard vanilla-trained models without relying on additional data, such as unbiased training set? In this work, we introduce Bias-Invariant Subnetwork Extraction (BISE), a learning strategy that identifies and isolates "bias-free" subnetworks that already exist within conventionally trained models, without retraining or finetuning the original parameters. Our approach demonstrates that such subnetworks can be extracted via pruning and can operate without modification, effectively relying less on biased features and maintaining robust performance. Our findings contribute towards efficient bias mitigation through structural adaptation of pre-trained neural networks via parameter removal, as opposed to costly strategies that are either data-centric or involve (re)training all model parameters. Extensive experiments on common benchmarks show the advantages of our approach in terms of the performance and computational efficiency of the resulting debiased model.

Figures & tables

Figure 1: Overview of the proposed method. BISE aims to extract an unbiased subnetwork from the biased vanilla-trained network.

Figure 2: Illustration of BISE. Solid black arrows indicate forward propagation; dashed blue arrows indicate backward gradient propagation. During the training of the mask $\mathcal{M}$ and of $\mathcal{C}_{\text{aux}}$ , the original model, $f=\mathcal{C}\circ\mathcal{E}$ , is kept frozen.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
Rubi [ 14 ]	93.6	43.0	90.4
EnD [ 92 ]	96.0	93.9	83.7
BCon+BBal [ 31 ]	98.1	97.7	97.3
ReBias [ 5 ]	88.4	75.4	65.8

Table 1: Results on BiasedMNIST for different

\rho

values.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
BISE	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$	$29.9_{\pm 7.5}$	$290.8_{\pm 31.0}$	$35.0_{\pm 1.5}$	$269.6_{\pm 6.2}$
BISE (last)	$18.9_{\pm 4.3}$	$336.4_{\pm 17.7}$	$30.0_{\pm 6.8}$	$290.7_{\pm 28.3}$	$33.8_{\pm 4.0}$	$274.6_{\pm 16.7}$

Table 2: Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset.

Method	Accuracy (%)
Method	$\rho=0.995$	$\rho=0.99$	$\rho=0.98$	$\rho=0.95$
Vanilla	$22.03_{\pm 0.39}$	$26.89_{\pm 0.25}$	$33.42_{\pm 0.65}$	$47.18_{\pm 0.34}$
EnD [ 92 ]	19.38	23.12	34.07	36.57
HEX [ 98 ]	13.87	14.81	15.20	16.04
ReBias [ 5 ]	22.27	25.72	31.66	43.43
LfF [ 70 ]	28.57	33.07	39.91	50.27

Table 3: Results on Corrupted-CIFAR10, for different values of

\rho

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.995$		$\rho=0.99$		$\rho=0.98$		$\rho=0.95$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	37.1	0	37.1	0	37.1	0	37.1
BISE (last)	$77.8_{\pm 2.3}$	$15.5_{\pm 0.9}$	$92.2_{\pm 1.0}$	$15.7_{\pm 1.1}$	$89.6_{\pm 0.6}$	$19.2_{\pm 0.8}$	$82.3_{\pm 0.6}$	$22.5_{\pm 0.6}$

Table 4: Sparsity (

S

) and computational cost (MFLOPs) of models, for Corrupted-CIFAR10.

Table 7

Method	Accuracy (%) $\uparrow$
Method	(Alig., Alig.)	(Alig., Conf.)	(Conf., Alig.)	(Conf., Conf.)	Unbiased
Vanilla	$\bf 100.0_{\pm 0.0}$	$\underline{97.2_{\pm 0.8}}$	$30.3_{\pm 1.4}$	$5.2_{\pm 0.3}$	$58.2_{\pm 0.6}$
LfF [ 70 ]	$\underline{99.6}$	4.7	$\bf 98.6$	5.1	52.0
EIIL [ 19 ]	100.0	$\underline{97.2}$	70.8	10.9	69.7
PGI [ 3 ]	98.6	82.6	26.6	9.5	54.3
DebiAN [ 53 ]	100.0	95.6	76.5	16.0	72.0

Table 7: Results on the Multi-Color MNIST dataset. (*) indicates that debiasing is performed by leveraging an unbiased dataset.

Reweighting of $\mathcal{L}_{\text{CE}}(\hat{y},y)$	$\mathcal{I}(\hat{b},b)$	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
-	-	$91.9_{\pm 0.7}$	$5.8_{\pm 2.6}$	$390.9_{\pm 10.6}$
$\checkmark$	-	$96.0_{\pm 0.4}$	$18.5_{\pm 6.5}$	$338.2_{\pm 27.1}$
-	$\checkmark$	$91.1_{\pm 0.5}$	$18.7_{\pm 0.5}$	$337.3_{\pm 2.0}$
$\checkmark$	$\checkmark$	$96.1_{\pm 0.5}$	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$

Table 8: Ablation study for the components of the objective

J

Panel of Figure 3 — Figure 3: Analysis of (a) coefficient $\gamma$ and (b) pruning strategies. Shaded areas indicate the interval of one standard deviation.

Method	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	$76.5_{\pm 2.1}$	0	1818.6
Unsup. BISE	$87.5_{\pm 2.5}$	$67.3_{\pm 4.8}$	$953.1_{\pm 218.4}$
Unsup. BISE (last)	$85.1_{\pm 1.8}$	$67.7_{\pm 4.6}$	$936.5_{\pm 190.1}$

Table 9: Results for unsupervised ( Unsup. ) debiasing on CelebA.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure C.1 : Samples from the BiasedMNIST dataset [ 5 ] . In the first row, bias-aligned images; in the second row, bias-conflicting images.

Figure C.2 : Samples from the CelebA dataset [ 58 ] .

Figure C.3 : Samples from the Corrupted-CIFAR10 dataset [ 30 ] . We show two images belonging to each class.

Figure C.4 : Samples from the Multi-Color MNIST dataset [ 53 ] . In the first row, images that are bias-aligned with respect to both background colors; in the second row, images that are bias-conflicting with respect to at least one of the colors.

Figure C.5 : Samples from the CivilComments dataset [ 11 , 44 ] . Label

y

indicates whether a comment is toxic, while

b

indicates if it mentions any sensitive attribute.

	Biased MNIST	CelebA	Corrupted- CIFAR10	Multi-Color MNIST	Civil Comments
# weights	$531\,210$	$11\,177\,538$	$11\,181\,642$	$256\,510$	$109\,361\,664$
# $m_{i}$	$240\;(0.05\%)$	$3\,840\;(0.03\%)$	$3\,840\;(0.03\%)$	$300\;(0.12\%)$	$46\,848\;(0.04\%)$

Appendix

Table E.1 : Number of weights vs. of updated parameters

m_{i}

Method	Acc. (%) $\uparrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	$10.0_{\pm 0.1}$	0	$415.4$
BISE	$15.7_{\pm 1.3}$	$85.6_{\pm 4.0}$	$59.8_{\pm 16.7}$
BISE + finetuning	$60.5_{\pm 9.4}$	$85.6_{\pm 4.0}$	$59.8_{\pm 16.7}$
BISE (last)	$14.7_{\pm 2.5}$	$78.2_{\pm 2.9}$	$90.5_{\pm 11.9}$
FFW * [ 68 ]	$19.9$	–	–

Appendix

Table G.1 : Results on BiasedMNIST for

\rho=0.999

. (*) indicates that debiasing is performed by leveraging an unbiased dataset.

Metric	$\boldsymbol{E}$
Metric	1	5	10	20	50
Acc. (%) $\uparrow$	$95.9_{\pm 0.4}$	$96.0_{\pm 0.5}$	$96.0_{\pm 0.4}$	$95.7_{\pm 0.5}$	$96.1_{\pm 0.5}$
$S$ (%) $\uparrow$	$18.9_{\pm 7.3}$	$17.6_{\pm 7.1}$	$20.0_{\pm 6.1}$	$17.2_{\pm 7.6}$	$20.9_{\pm 4.4}$
MFLOPs $\downarrow$	$336.7_{\pm 30.4}$	$342.1_{\pm 29.5}$	$331.9_{\pm 25.2}$	$343.7_{\pm 31.6}$	$328.3_{\pm 18.2}$

Appendix

Table H.1 : Effect of

E

on the extracted subnetwork.

Metric	$\boldsymbol{\kappa}$
Metric	0.1	0.5	0.8
Acc. (%) $\uparrow$	$96.0_{\pm 0.4}$	$96.1_{\pm 0.5}$	$96.2_{\pm 0.6}$
$S$ (%) $\uparrow$	$20.6_{\pm 3.9}$	$20.9_{\pm 4.4}$	$18.9_{\pm 7.6}$
MFLOPs $\downarrow$	$329.3_{\pm 16.3}$	$328.3_{\pm 18.2}$	$336.5_{\pm 31.4}$

Appendix

Table H.2 : Effect of

\kappa

on the extracted subnetwork.

Metric	$\boldsymbol{\upsilon}$
Metric	1	5	10	20	50
Acc. (%) $\uparrow$	$95.9_{\pm 0.5}$	$95.8_{\pm 0.5}$	$96.1_{\pm 0.5}$	$96.0_{\pm 0.5}$	$96.5_{\pm 0.3}$
$S$ (%) $\uparrow$	$22.4_{\pm 6.3}$	$18.3_{\pm 6.2}$	$20.9_{\pm 4.4}$	$19.2_{\pm 6.3}$	$19.8_{\pm 8.0}$
MFLOPs $\downarrow$	$322.2_{\pm 26.2}$	$338.9_{\pm 25.7}$	$328.3_{\pm 18.2}$	$335.2_{\pm 26.2}$	$333.0_{\pm 33.3}$

Appendix

Table H.3 : Effect of

\upsilon

on the extracted subnetwork.

Metric	$\boldsymbol{\tau_{\textrm{min}}}$
Metric	$10^{-1}$	$10^{-2}$	$10^{-3}$	$10^{-4}$
Acc. (%) $\uparrow$	$95.9_{\pm 0.6}$	$96.1_{\pm 0.5}$	$96.1_{\pm 0.5}$	$96.1_{\pm 0.5}$
$S$ (%) $\uparrow$	$20.4_{\pm 5.0}$	$20.9_{\pm 4.4}$	$20.9_{\pm 4.4}$	$20.9_{\pm 4.4}$
MFLOPs $\downarrow$	$330.3_{\pm 20.7}$	$328.3_{\pm 18.2}$	$328.3_{\pm 18.2}$	$328.3_{\pm 18.2}$

Appendix

Table H.4 : Effect of

\tau_{\textrm{min}}

on the extracted subnetwork.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
BISE ( Tab. 1 )	$96.1_{\pm 0.5}$	$92.2_{\pm 1.9}$	$90.8_{\pm 0.6}$
BISE + trainable BN (avg., std.)	$97.3_{\pm 0.1}$	$94.9_{\pm 1.1}$	$94.0_{\pm 1.2}$
BISE + trainable BN (avg., std., $\beta$ , $\gamma$ )	$96.9_{\pm 0.4}$	$92.2_{\pm 1.9}$	$90.1_{\pm 3.7}$

Appendix

Table I.1 : Experiments on BiasedMNIST with trainable vs. non-trainable batch normalization layers.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
BISE ( Tab. 2 )	$20.9_{\pm 4.4}$	$328.3_{\pm 18.2}$	$29.9_{\pm 7.5}$	$290.8_{\pm 31.0}$	$35.0_{\pm 1.5}$	$269.6_{\pm 6.2}$
BISE + trainable BN (avg., std.)	$18.3_{\pm 6.8}$	$339.2_{\pm 28.1}$	$26.5_{\pm 4.0}$	$305.0_{\pm 16.7}$	$33.3_{\pm 7.9}$	$276.9_{\pm 32.9}$
BISE + trainable BN (avg., std., $\beta$ , $\gamma$ )	$32.7_{\pm 7.7}$	$279.5_{\pm 32.0}$	$45.7_{\pm 3.9}$	$225.6_{\pm 16.2}$	$48.2_{\pm 12.0}$	$214.9_{\pm 49.8}$

Appendix

Table I.2 : Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset, when updating the batch normalization layers during BISE.

Figure J.1 : Analysis of the effect of varying the threshold

\zeta

on the pruned network. Threshold

\zeta=0

corresponds to the vanilla dense model;

\zeta=0.5

is the original threshold used in BISE.

Method	Accuracy (%)
Method	$\rho=0.99$	$\rho=0.995$	$\rho=0.997$
Vanilla	$88.9_{\pm 0.4}$	$75.1_{\pm 4.2}$	$66.1_{\pm 1.7}$
Unsup. BISE	$95.8_{\pm 0.5}$	$92.0_{\pm 2.0}$	$90.4_{\pm 0.6}$
Unsup. BISE (last)	$95.4_{\pm 0.7}$	$90.6_{\pm 1.9}$	$90.0_{\pm 0.2}$

Appendix

Table K.1 : Unsupervised ( Unsup. ) debiasing on BiasedMNIST.

Method	Proportion of bias-aligned samples in the training set ( $\rho$ )
	$\rho=0.99$		$\rho=0.995$		$\rho=0.997$
	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$	$S$ (%) $\uparrow$	MFLOPs $\downarrow$
Vanilla	0	415.4	0	415.4	0	415.4
Unsup. BISE	$19.9_{\pm 4.4}$	$332.4_{\pm 18.2}$	$26.2_{\pm 4.7}$	$306.3_{\pm 19.5}$	$30.9_{\pm 2.3}$	$286.6_{\pm 9.7}$
Unsup. BISE (last)	$19.7_{\pm 3.6}$	$333.4_{\pm 14.9}$	$28.8_{\pm 4.9}$	$295.6_{\pm 20.3}$	$28.8_{\pm 4.5}$	$295.5_{\pm 18.7}$

Appendix

Table K.2 : Sparsity (

S

) and computational cost (MFLOPs) of models, for the BiasedMNIST dataset, in the unsupervised debiasing scenario.

Figure L.1 : Results on BiasedMNIST (training set with

\rho=0.99

) under noisy bias labels. Noise level

p=0\%

corresponds to the standard setting ( i.e. , use of true bias labels).

Model	Accuracy of the aux. classifier on color prediction (%)
Vanilla	$97.6_{\pm 1.0}$
Debiased with BISE	$82.1_{\pm 7.3}$

Appendix

Table M.1 : Comparison of accuracy on color prediction for two auxiliary classifiers: one trained on top of a vanilla encoder, and the other trained on top of the BISE-pruned encoder.

Metric	Model	Dataset
Metric	Model	BiasedMNIST ( $\rho=0.99$ )	Corrupted-CIFAR10 ( $\rho=0.95$ )	CelebA	Multi-Color MNIST ( $\rho_{\text{L}}=0.99$ , $\rho_{\text{R}}=0.95$ )	CivilComments
Acc. (%)	Vanilla	$88.9_{\pm 0.4}$	$47.18_{\pm 0.34}$	$76.5_{\pm 2.1}$	$58.2_{\pm 0.6}$	$59.6_{\pm 2.7}$
	Best competitor	$98.1$ (BCon+BBal [ 31 ] )	$51.13$ (DFA [ 51 ] )	$91.4$ (BCon+BBal [ 31 ] )	$73.1$ (VCBA [ 69 ] )	$80.4$ (Group DRO [ 82 ] )
	BISE ∗	$96.1_{\pm 0.5}$	$55.38_{\pm 1.96}$	$89.7_{\pm 0.8}$	$60.3_{\pm 1.0}$	$80.4_{\pm 0.2}$
	BISE + finetuning ∗	$98.1_{\pm 0.1}$	$57.22_{\pm 1.81}$	$91.8_{\pm 1.3}$	$70.6_{\pm 1.6}$	$81.0_{\pm 0.1}$
$S$ (%)	BISE ∗	$20.9_{\pm 4.4}$	$82.3_{\pm 0.6}$	$67.6_{\pm 0.8}$	$17.1_{\pm 5.3}$	$26.0_{\pm 5.4}$

Appendix

Table N.1 : Summary of results across datasets. ( ∗ ) indicates that the result for BISE “best” is reported whenever a validation set is available; otherwise, we report the result for BISE “last” . The metric Acc. corresponds to the accuracy on an unbiased test set, in the case of BiasedMNIST, CelebA and Corrupted-CIFAR10; for Multi-Color MNIST, it denotes the average test accuracy across the four existing groups in the dataset (as described in Sec. 4.2 ); for CivilComments, it denotes the worst-group test accuracy. The sparsity (

S

) for the vanilla and competing methods is zero, since no parameter is removed from the network.

Explore similar work

Unbiased Model Prediction Without Using Protected Attribute Information
Mar 31, 2026Puspita Majumdar, Surbhi Mittal, Saheb Chhabra +2Algorithmic Fairness Debiased ML
IIIT-Delhi, New Delhi, India, 110020
Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement
Jun 23, 2026Xiangqian Zhao, Xinyang Jiang, Zhipeng Xu +5Disentangled Representation Learning Debiased ML
Xidian University · Shaanxi, China · Microsoft Research Asia +1
A Heuristic Perspective on Debiasing Language Models
Aug 1, 2026Tian Lan, Yemin Wang, Chuancheng Shi +6Social Bias in Language Models Cultural Bias in Language Models
Inner Mongolia University · Xiamen University · University of Sydney +3

Mar 31, 2026J/K move · Enter open · S save

Unbiased Model Prediction Without Using Protected Attribute Information

Puspita Majumdar, Surbhi Mittal, Saheb Chhabra, Mayank Vatsa+1

IIIT-Delhi, New Delhi, India, 110020

Algorithmic Fairness Debiased ML

Open paper AskPDF arXiv

The problem of bias persists in the deep learning community as models continue to provide disparate performance across different demographic subgroups. Therefore, several algorithms have been proposed to improve the fairness of deep models. However, a majority of these algorithms utilize the protected attribute information for bias mitigation, which severely limits their application in real-world scenarios. To address this concern, we have proposed a novel algorithm, termed as Non-Protected Attribute-based Debiasing (NPAD) algorithm for bias mitigation, that does not require the protected attribute information. The proposed NPAD algorithm utilizes the auxiliary information provided by the non-protected attributes to optimize the model for bias mitigation. Further, two different loss functions, Debiasing via Attribute Cluster Loss (DACL) and Filter Redundancy Loss (FRL) have been proposed to optimize the model for fairness goals. Multiple experiments are performed on the LFWA and CelebA datasets for facial attribute prediction, and a significant reduction in bias across different gender and age subgroups is observed.

Abstract

Explore similar work

Unbiased Model Prediction Without Using Protected Attribute Information

Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

A Heuristic Perspective on Debiasing Language Models