Language Model Bias Evaluation

Ask about this topic

Momentum

40 papers in the last four weeks, up 122% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 321

All topics
CardsList

  1. AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology II: Project Planning and Proposal Evaluation

    Jul 28, 2026Jia Liu, Veena Krishnaraj, Kateryna Vovk +18LLM EvaluationAI-Assisted Scientific Research

  2. Tag Questions and the Generational Reversal of Sycophancy Across 45 Language Models

    Jul 27, 2026Tapan ParikhLLM EvaluationLLM Sycophancy

    Cornell Tech

  3. QuantiBias: Benchmarking Quantization-Induced Bias in LLMs

    Jul 23, 2026Emilio FerraraOpen-Ended GenerationLLM Quantization

    Thomas Lord Department of Computer Science University of Southern California

  4. Position Bias is Hidden Behind Ceiling Effects: A Permutation Diagnostic for LLM Benchmarks

    Jul 23, 2026Hiroki TambaLLM EvaluationLanguage Model Bias Evaluation

    Independent researcher.

  5. REGARD: Regional Affective Differences in Large Language Models

    Jul 22, 2026Andrei Chetvergov, Alexander Evseev, Mikhail Solovev +5Political Bias in Language ModelsValence-Arousal Modeling

    Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow, Russia · Russian Presidential Academy of National Economy and Public Administration, Moscow, Russia

  6. Evaluating and Mitigating Gender Bias in Pre-trained Embeddings for ML-based Recruitment

    Jul 22, 2026Farnaz Faramarzi Lighvan, Lynn HouthuysGender Bias in Language ModelsAlgorithmic Fairness

    AI Lab, Vrije Universiteit Brussel, Belgium

  7. AutoJourn: Multi-Perspective Summarisation, Bias Detection and Bias Neutralisation for LLM-Generated News in Automated Journalism

    Jul 21, 2026Himel Ghosh, Ahmed Mosharafa, Georg GrohSocial Bias in Language ModelsText Summarization

    Technical University of Munich, Germany · Sapienza University of Rome, Italy

  8. When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs

    Jul 20, 2026Md. Asaduzzaman ShuvoCultural Bias in Language ModelsLanguage Model Bias Evaluation

    United International University, Bangladesh

  9. Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies

    Jul 16, 2026Filippos Vlahos, Guillaume Bied, Tijl De BieLLM AuditingPolitical Bias in Language Models

    Ghent University, BE

  10. Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs

    Jul 16, 2026Robert Graham, Edward Stevinson, Yariv BarsheshatLLM Fine-TuningPolitical Bias in Language Models

    1Independent · 2Imperial College London.

  11. The Misclassification of Autistic Writing as AI-Generated

    Jul 16, 2026Summer Chambers, Matthew C. KelleyAI-Generated Text DetectionSocial Bias in Language Models

    George Mason University, Fairfax VA 22030, USA

  12. Investigating first-language bias in LLM-based automated essay scoring: A cross-prompt evaluation of an open-weight AI-model on TOEFL essays

    Jul 16, 2026John Maurice GayedEducational AssessmentAutomated Essay Scoring

    Global Education Center, Waseda University

  13. LLM Evaluators are Biased across Languages

    Jul 16, 2026Ej Zhou, Lucas Resck, Zheng Hui +1Multilingual Language Model EvaluationLanguage Model Safety Evaluation

    Language Technology Lab, University of Cambridge

  14. HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization

    Jul 15, 2026Abdullah Shaikh, Zain Naqi, Taha Zahid +2Multilingual Language Model EvaluationLanguage Model Bias Evaluation

    Dhanani School of Science & Engineering Habib University, Pakistan

  15. Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values

    Jul 15, 2026Jan Betley, Johannes Treutlein, Jan Dubiński +5LLM AlignmentLanguage Model Bias Evaluation

    1Truthful AI · 2Warsaw University of Technology · 3NASK National Research Institute +2

  16. The Test Oracle Problem in Synthetic LLM-as-Judge Corpora: Disappearance, Distortion and a Validation Protocol

    Jul 15, 2026Serkan BallıMultilingual Language Model EvaluationLLM-as-a-Judge

    Department of Software Engineering, Mehmet Akif Ersoy University, Burdur, Türkiye · Department of Software Engineering, Mehmet Akif Ersoy University, Türkiye

  17. Toward Localizing and Repairing Bias in Transformer Attention Heads

    Jul 14, 2026Sigma JahanAttention Head AnalysisSocial Bias in Language Models

    Dalhousie University, Canada

  18. Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

    Jul 13, 2026Zixiang Xu, Sixian Li, Huaxing Liu +4LLM-as-a-JudgeLLM Interpretability

    AMAP, Alibaba Group · Mohamed bin Zayed University of Artificial Intelligence · University of Southern California +1

  19. How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation?

    Jul 13, 2026Elmira Salari, Hazem Amamou, José Victor de Souza +3Retrieval-Augmented GenerationPolitical Bias in Language Models

    Elmira Salari, Wichita State University, Wichita, KS, USA · Hazem Amamou, Institut national de la recherche scientifique, Québec, Montréal, Canada · José Victor de Souza, Institut national de la recherche scientifique, Québec, Montréal, Canada +3

  20. Are LLMs ready for HardChoices?

    Jul 13, 2026Dmitry NikolaevPolitical Bias in Language ModelsLanguage Model Bias Evaluation

    University of Manchester

  21. DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

    Jul 13, 2026Anqi Li, Jie Zhang, Zhongqi Wang +4VLM EvaluationLanguage Model Bias Evaluation

  22. Eval-Pair Matrix: Answer-Paired Meta-Evaluation of LLM Judges for Grounded RAG

    Jul 12, 2026Sriram Selvam, Anneswa GhoshLLM-as-a-JudgeLanguage Model Generation Evaluation

  23. Articulate Intuition or Genuine Analysis? Benchmarking Epistemic Reliability in LLM-as-a-Judge Peer Reviews

    Jul 12, 2026Nuo Chen, Qian Wang, Qingyun Zou +1LLM-as-a-JudgeScholarly Peer Review

    National University of Singapore

  24. Geopolitical alignment: Endorsement effects in large language models

    Jul 10, 2026Maxim ChupilkinPolitical Bias in Language ModelsLanguage Model Bias Evaluation

    Department of Politics and International Relations, University of Oxford

  25. Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution

    Jul 9, 2026Ethan Leung, Elias Lumer, Corey Feld +3LLM-as-a-JudgeCitation Verification

    Commercial Technology and Innovation Office, PricewaterhouseCoopers, U.S.

  26. When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

    Jul 9, 2026Zongyou Yang, Yinghan Hou, Xiaokun YangLLM-as-a-JudgeLLM Auditing

    Dyson School of Design Engineering, Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering, Imperial College London London, United Kingdom

  27. When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation

    Jul 8, 2026Yahan Zheng, John Guerrerio, Soroush Vosoughi +1Debiased MLSocial Bias in Language Models

    Dartmouth College · Oakland University

  28. Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration

    Jul 8, 2026Weicheng Ma, John Guerrerio, Soroush VosoughiMultilingual Language Model EvaluationSocial Bias in Language Models

    College of Computing, Georgia Institute of Technology · Computer Science Department, Dartmouth College

  29. LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering

    Jul 7, 2026Huan Wu, Ali Emami, Muhammad Furquan Hassan +5Language Model SteeringSocial Bias in Language Models

    1York University · 2Vector Institute · Connected Minds +6

  30. RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs

    Jul 6, 2026Damian Hodel, Jevin West, Aylin CaliskanLLM EvaluationSocial Bias in Language Models

    University of Washington, Seattle, USA