cs.LGDec 15, 2023

Integrating Fairness and Model Pruning Through Bi-level Optimization

Authors: Yucong Dai, Gen Li, Feng Luo, Xiaolong Ma, Yongkai Wu

Organizations: Clemson University

Abstract

Deep neural networks achieve high predictive performance but often require substantial computational and storage resources, which motivates model compression techniques such as neural network pruning. Conventional pruning methods, however, can introduce or amplify algorithmic bias and produce disparate outcomes across demographic groups. We study fair model pruning, which seeks sparse models that satisfy fairness criteria while maintaining predictive performance. We characterize fairness using both group accuracy disparity and differences in compression-induced accuracy degradation across demographic groups. We propose a framework that jointly optimizes pruning masks and model weights under fairness constraints, thereby integrating accuracy and fairness into the compression process. We formulate fair pruning as a constrained bi-level optimization problem and derive an efficient optimization procedure. Experiments across multiple datasets, predictive tasks, model architectures, and sparsity levels show that our framework improves the trade-off among fairness, accuracy, and training efficiency relative to the baseline pruning strategies. Compared with the strongest evaluated baselines, BiFP maintains competitive accuracy and fairness while requiring more than 94% fewer training iterations in the real-world experiments.

Figures & tables

Explore similar work

CardsList
  1. FairCompressAgent: An Agentic Framework for Fairness-Aware Model Compression for FPGA Deployment

    Sep 15, 2026Yuanbo Guo, Yiyu ShiModel CompressionAlgorithmic Fairness

  2. Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning

    May 16, 2026David Troxell, Noah Roemer, Guido MontúfarDeep Learning OptimizationAlgorithmic Fairness

  3. Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

    Jul 30, 2026Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña LópezLLM PruningLLM Interpretability