Standard fairness audits of foundation models quantify that a model is biased, but not where inside the network the bias resides. We propose a mechanistic fairness audit that combines projected residual-stream decomposition, zero-shot Concept Activation Vectors, and bias-augmented TextSpan analysis to locate demographic bias at the level of individual attention heads in vision transformers. As a feasibility case study, we apply this pipeline to the CLIP ViT-L-14 encoder on 42 profession classes of the FACET benchmark, auditing both gender and age bias. For gender, the pipeline identifies four terminal-layer heads whose ablation reduces global bias (Cramer's V: 0.381 -> 0.362) while marginally improving accuracy (+0.42%); a layer-matched random control confirms that this effect is specific to the identified heads. A single head in the final layer contributes to the majority of the reduction in the most stereotyped classes, and class-level analysis shows that corrected predictions shift toward the correct occupation. For age, the same pipeline identifies candidate heads, but ablation produces weaker and less consistent effects, suggesting that age bias is encoded more diffusely than gender bias in this model. These results provide preliminary evidence that head-level bias localisation is feasible for discriminative vision encoders and that the degree of localisability may vary across protected attributes. keywords: Bias . CLIP . Mechanistic Interpretability . Vision Transformer . Fairness
Figures & tables
Concept Prototypes for Demographic and Occupation Attributes
Concept
Prototype Texts (5 per concept)
Demographic Attributes (Bias Prototypes)
Male
Male person, Man, Masculine face, Male individual, He
Female
Female person, Woman, Feminine face, Female individual, She
Young person, Youth, Young adult, Youthful face, Teenager
Table 1: Text prototypes used for zero-shot CAV analysis. For occupations, 5 synonyms per class are defined (representative examples shown). For demographics, the full set of prototypes is listed.
GENDER BIAS
AGE BIAS
Class
V
→ Pred.
Male
Fem.
Class
V
→ Pred.
Yng.
Old.
Nurse
0.450
Nurse
44.9
83.1
Guard
0.238
Guard
89.3
44.8
Doctor
20.1
3.1
Laborer
0.6
10.3
Repairman
7.7
0.4
Seller
1.3
6.9
Doctor
0.399
Nurse
39.4
78.2
Ballplayer
0.186
Ballplayer
68.2
83.3
Doctor
37.2
13.4
Basket. pl.
10.3
0.0
Table 2: Top-5 gender- and age-biased occupation classes. The left half shows Gender Bias and the right half shows Age Bias ranked by Cramér’s V , including top-3 predicted classes with per-group rates (%). The correct class is highlighted.
Global Bias Reduction ( ΔV ): Suspected vs. Random Control
Condition
Accuracy
Cramér’s V
ΔV
Gender-Head Experiment ( V measured on gender)
Baseline
64.30%
0.381
—
Suspected (4)
64.72%
0.362
−0.019
Random avg (4)
64.16% ± 0.27
0.381 ± 0.002
−0.000
Age-Head Experiment ( V measured on age)
Table 3: Global ablation results for gender-head and age-head experiments, each vs. a layer-matched random control ( ± std over 10 runs). V refers to the target attribute of each experiment.
Figure 1: Per-class accuracy change (in percentage points) after combined gender-head ablation across all 19 classes with statistically significant gender bias (BH-corrected p<0.05 ). Green bars indicate improved accuracy; red bars indicate declines. Sample sizes ( n ) are shown below each class label.
Doctor Class — Per-Head Ablation and Prediction Redistribution (Gender)
Male ( N=180 )
Female ( N=179 )
Condition
V
ΔV
→ Doctor
→ Nurse
→ Doctor
→ Nurse
Baseline
0.399
—
37.2
39.4
13.4
78.2
L23H4
0.237
−0.162
38.3
37.8
26.3
60.9
L21H2
0.355
−0.044
36.7
39.4
14.0
74.3
L21H10
0.356
−0.043
37.8
38.9
16.2
73.7
Table 4: Doctor class: per-head and combined ablation results (gender). Left columns show bias metrics; right columns show prediction redistribution by gender. Heads sorted by ΔV .
Craftsman Class — Per-Head Ablation and Prediction Redistribution (Gender)
Male ( N=598 )
Female ( N=158 )
Condition
V
ΔV
→ Craftsman
→ Seller
→ Craftsman
→ Seller
Baseline
0.279
—
41.5
14.2
31.6
38.6
L23H4
0.226
−0.053
38.6
15.4
39.2
32.3
L21H2
0.273
−0.007
41.3
14.2
32.3
38.0
L21H10
0.273
−0.007
41.6
14.4
31.0
38.6
Table 5: Craftsman class: per-head and combined ablation results (gender). Left columns show bias metrics; right columns show prediction redistribution by gender. Heads sorted by ΔV .
Figure 2: Qualitative examples of correction after gender-bias head ablation. All images were misclassified at baseline (e.g., nurse for doctor, machinist or seller for craftsman) and correctly reclassified after ablating the top-10 identified gender heads. The images are chosen from cases where the demographic-parity gap was most pronounced.
Guard Class — Per-Head Ablation and Prediction Redistribution (Age)
Young ( N=159 )
Middle ( N=460 )
Older ( N=29 )
Condition
V
ΔV
→ Guard
→ Garden.
→ Guard
→ Report.
→ Guard
→ Seller
Baseline
0.238
—
89.3
1.3
69.1
3.5
44.8
6.9
L21H5
0.234
−0.004
88.7
1.3
68.7
3.9
44.8
10.3
L22H4
0.246
+0.008
90.6
1.3
70.0
3.0
44.8
10.3
L23H4
0.252
+0.015
89.3
1.3
67.6
3.9
41.4
6.9
Table 6: Guard class: per-head and combined ablation results (age). Left columns show bias metrics; right columns show top-2 predictions by age group (top confusion per group: Gardener for Young, Reporter for Middle, Seller for Older). Heads sorted by ΔV .
We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models, offering an internal audit of model behaviour through representational analysis. Using BERT and Llama2 as representative architectures, we assess the shifts in associations between gender and occupation terms by comparing baseline and bias-mitigated variants of the models. Our findings show that bias mitigation reduces gender-occupation disparities in the embedding space, leading to more neutral and balanced internal representations. These representational shifts are consistent across both model types, suggesting that fairness improvements can manifest as interpretable and geometric transformations. These results position embedding analysis as a valuable tool for understanding and validating the effectiveness of debiasing methods in foundation models. To further promote the assessment of decoder-only models, we introduce WinoDec, a dataset consisting of 4,000 sequences with gender and occupation terms, and release it to the general public. (https://github.com/winodec/wino-dec)
Svetoslav Nizhnichenkov, Rahul Nair, Elizabeth Daly +1
IBM Research, Dublin, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland
This paper presents FairNVT, a lightweight debiasing framework for pretrained transformer-based encoders that improves both representation and prediction level fairness while preserving task accuracy. Unlike many existing debiasing approaches that address these notions separately, we argue they are inherently connected: suppressing sensitive information at the representation level can facilitate fairer predictions. Our approach learns task-relevant and sensitive embeddings via lightweight adapters, applies calibrated Gaussian noise to the sensitive embedding, and fuses it with the task representation. Together with orthogonality constraints and fairness regularization, these components jointly reduce sensitive-attribute leakage in the learned embeddings and encourage fairer downstream predictions. The framework is compatible with a wide range of pretrained transformer encoders. Across three datasets spanning vision and language, FairNVT reduces sensitive-attribute attacker accuracy, improves demographic-parity and equalized-odds metrics, and maintains high task performance.
Vision language models (VLMs) demonstrate strong zero-shot performance, but often perpetuate social stereotypes in person-centric queries, yielding skewed demographic distributions. Current debiasing methods apply uniform bias corrections across all input queries regardless of their bias sensitivity, creating a fundamental fairness--utility trade-off. Strong debiasing distorts semantically meaningful information in bias-insensitive queries, while weak debiasing fails to mitigate stereotypes in bias-sensitive ones. This one-size-fits-all approach hampers simultaneously achieving high utility on bias-insensitive queries and fairness on bias-sensitive queries. We introduce Reward-Gated Test-Time Adaptation (RG-TTA), a reinforcement learning-based test-time adaptation framework that selectively applies debiasing based on input sensitivity. RG-TTA adaptively triggers fairness regularization based on the bias sensitivity of each input during test-time policy adaptation, while focusing exclusively on optimizing cross-modal alignment for bias-insensitive inputs. Experiments on fairness benchmarks (e.g., FairFace, UTKFace) demonstrate substantial bias reduction while simultaneously improving zero-shot utility, resolving the trade-off of uniform debiasing.
Jaeho Han, Jisoo Yang, Hyeondong Woo +3
Department of Artificial Intelligence, Chung-Ang University