cs.CLOct 10, 2026

Social Bias in Chinese Large Language Models under Social Media Expressions

Authors: Mo Zhou

Organizations: School of Chinese Language and Literature, Nanjing Normal University, Nanjing 210000, China

Abstract

Existing Chinese social-bias benchmarks for large language models largely rely on normalized or template-based expressions, leaving the influence of internet slang, emoji, and sarcasm insufficiently examined. Safety alignment may further compress conventional plausibility ratings toward the lower end of the scale. This study constructs 73 paired probes across gender, region, and occupation. Each probe contains a neutral expression, a social-media-style expression, and a reversed-identity control. The task is reframed from direct plausibility judgment to social consensus estimation. Social-media expressions are divided into stylistic packaging, which preserves semantic polarity, and sarcastic packaging, which reverses surface polarity. Six Chinese-capable large language models are evaluated through paired differences measuring identity bias, stylistic packaging, and sarcasm shift. All models show significant identity-bias effects; five show significantly negative stylistic-packaging effects; and sarcasm shifts display a clear cross-model gradient. These findings demonstrate that expression form is not a transparent carrier in bias evaluation and that lower scores under sarcasm should not be interpreted directly as bias mitigation. The proposed paired design offers a measurement approach for Chinese social-media contexts while also revealing the construct boundary and prompt sensitivity of social consensus estimation.

Explore similar work

CardsList
  1. Investigating the Impact of Learner Errors on Social Bias in Large Language Models

    Oct 21, 2026Federico Marcuzzi, Martin Christoph, Iryna Gurevych +1Social Bias in Language ModelsLanguage Model Bias Evaluation

  2. APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

    Oct 14, 2026Tian Lan, Han Zhang, Shanshan Wang +3LLM EvaluationSocial Reasoning