cs.CLNov 17, 2026

What Does a Bias Head Measure? Non-Identifiability in Head-Level Bias Attribution

Authors: S. M Seefat Alam

Organizations: Bangladesh University of Engineering and Technology, Dhaka

Abstract

Methods that localise social bias in large language models to individual attention heads and remove it by ablation are validated by their effect on a fairness metric, never by evidence that the statistic selecting the heads measures bias. We audit this on Qwen-2.5-7B-Instruct and Llama-3.1-8B-Instruct. The head map correlates at ρ=−0.950ρ= -0.950 and −0.768-0.768 with a label-blind response-length map, apparently identifying a surface confound. An intervention that breaks the length-label coupling rejects this reading: with the prompt held fixed, response length explains 1% of the statistic's variance. What does explain it is prompt identity, at 64% of variance, in an ordering that is not monotonic in length. Within any fixed prompt the fairness label, response length and the presence of a protected-group token are collinear at φ=0.95φ= 0.95, partly by definition of the labelling rule, so no observational analysis can attribute the map to any of them, and no method in this family performs an experiment that could. The substantive claims fail independently: bias selectivity is not late-layer-concentrated on either model under either reference reading, and no head set we tested reduces the unfairness rate when ablated. Its complement, scored on the opposite label, reliably destroys refusal behaviour (p=2.7×10−5p = 2.7\times10^{-5}), demonstrating that the pipeline detects effects at this scale.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering

    Oct 21, 2026Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar ThakkerSocial Bias in Language ModelsLanguage Model Bias Evaluation

  2. BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence

    May 9, 2026Jialing Gan, Junhao Dong, Songze LiPrompt SensitivityLLM Auditing

  3. Deep and shallow biases in language models

    Sep 9, 2026An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen +4Social Bias in Language ModelsLanguage Model Bias Evaluation