cs.LGNov 17, 2026

Beyond Attribute-Centric Fairness: Modelling and Mitigating Unified Bias Representation in Omni-modal Large Language Models

Authors: Runqi Lin, Suqin Yuan, Zihao Fu, Chris Russell

Organizations: Univeristy of Oxford · The Chinese University of Hong Kong · The University of Sydney

Abstract

Omni-modal large language models (OLLMs) provide joint multimodal understanding and generation capabilities, making them well-suited to complex environments and promising backbones for next-generation AI. Despite these advances, the harmful stereotypes encoded in such models manifest in increasingly diverse forms, leading to broader and more severe societal consequences. Existing debiasing methods typically impose separate fairness objectives for each sensitive attribute and modality to minimise demographic-dependent variations. However, in omni-modal settings, this attribute-centric paradigm struggles to jointly address diverse social biases with distinct fairness objectives and to generalise beyond predefined attributes. To this end, we propose OmniDebias, one of the first methods for promoting fairness in OLLMs, which employs a bias-centric formulation to extract and mitigate a shared bias representation across attributes and modalities while preserving attribute-related knowledge. Specifically, we exploit activation inconsistencies of a given attribute (e.g., gender) across semantic contexts (e.g., occupations) to disentangle bias from legitimate knowledge. We further consolidate multiple extracted biases into a shared representation, formulating a unified objective that captures the underlying structure across social biases and generalises to unseen attributes. Extensive experiments demonstrate that eliminating unified bias with both post-hoc editing and fine-tuning can consistently improve fairness across diverse attributes and modalities in OLLMs while maintaining favourable utility.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Collective Bias Mitigation via Model Routing and Collaboration

    Oct 2, 2026Mingzhe Du, Luu Anh Tuan, Xiaobao Wu +7Social Bias in Language ModelsDemographic Bias in Language Models

  2. Knowing Bias, Doing Better: Mitigating Social Bias in LLMs via Know-Bias Neuron Enhancement

    Jan 29, 2026Jinhao Pan, Chahat Raj, Anjishnu Mukherjee +4Debiased MLSocial Bias in Language Models

  3. Interpretable Debiasing of Vision-Language Models for Social Fairness

    Feb 27, 2026Na Min An, Yoonna Jang, Yusuke Hirota +3Cultural Bias in VLMsVLM Interpretability