Language Model Bias Evaluation

Ask about this topic

Momentum

40 papers in the last four weeks, up 122% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 321

All topics
CardsList

  1. New Conditions for Philosophers to Catch the Wave of Citizen Deliberation in the Age of Artificial Intelligence in advance

    Sep 14, 2026Bernard ReberPolitical Bias in Language ModelsLanguage Model Bias Evaluation

    CEVIPOF · National Centre for Scientific Research and Centre de recherches politiques de Sciences Po

  2. Issue Bias in Generative AI Writing Assistance: Political Issues and LLMs in the Swedish 2026 Election

    Sep 14, 2026Bastiaan Bruinsma, Annika Fredén, Paul Röttger +2Political Discourse AnalysisPolitical Bias in Language Models

    Chalmers University of Technology · University of Gothenburg · Lund University +1

  3. Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration

    Sep 14, 2026Gemma Zhang, Prachi Badarayani, Asmi Kumar +2LLM EvaluationLLM-as-a-Judge

    Microsoft · Massachusetts Institute of Technology

  4. Debiasing as a Measurement Intervention: Calibrated Ties and Resolution Loss in LLM-as-a-Judge Evaluation

    Sep 14, 2026Liang Zhao, Yong Wang, Jiangzhe ChenLLM-as-a-JudgeLLM Agent Evaluation

    Shanghai University of International Business and Economics

  5. EduFair-Bench: Evaluating Pedagogical Fairness of LLM Tutors Across Student Demographics

    Sep 14, 2026Jiaxu Zhao, Bahar Radmehr, Fares Fawzi +2Generative AI in EducationLanguage Model Bias Evaluation

    Swiss Federal Institute of Technology in Lausanne (EPFL), Switzerland

  6. One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs

    Sep 14, 2026Naihao Deng, Samee Arif, Shuaichen Chang +2LLM EvaluationLLM Alignment

    University of Michigan · The Ohio State University · University of Aberdeen +1

  7. Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model

    Sep 12, 2026Hyojung HanLLM AlignmentLLM Auditing

    ThakiCloud

  8. When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text

    Sep 10, 2026DongHyun Ryu, Jaehyeok Lee, YeongJun Hwang +1LLM-as-a-JudgeSocial Bias in Language Models

    Sungkyunkwan University, Suwon, South Korea

  9. DiSCo: A Distribution-First Steering and Cultural Prior Evaluation Framework for Measuring Cultural Preference Bias in LLMs

    Sep 9, 2026Bhuvan Arora, Devesh Saraogi, Sravya Varada +1LLM PromptingCultural Bias in Language Models

    BITS Pilani, Pilani, India.

  10. Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

    Sep 9, 2026Marek Jeliński, Jan Dubiński, Maciej Chrabaszcz +1LLM AuditingSocial Bias in Language Models

    NASK - National Research Institute, Poland · Warsaw University of Technology, Poland · Gda´nsk University of Technology, Poland

  11. Deep and shallow biases in language models

    Sep 9, 2026An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen +4Social Bias in Language ModelsLanguage Model Bias Evaluation

    MBZUAI · University of Michigan · KAIST +2

  12. The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

    Sep 8, 2026Siddharth Vohra, Manikandan RavikiranLanguage Model Bias EvaluationDemographic Bias in Language Models

    Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA · Indian Institute of Technology Mandi, India

  13. Navigating the digital spectrum: Assessing political bias, stability, and downstream fairness in Large Language Models

    Sep 8, 2026Luka Debevc, Nishan Chatterjee, Antoine Doucet +2Multilingual Language Model EvaluationPolitical Bias in Language Models

    1Jožef Stefan Institute, Ljubljana, Slovenia · University of Ljubljana, Ljubljana, Slovenia · 2Jožef Stefan International Postgraduate School, Ljubljana, Slovenia +1

  14. FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect

    Sep 7, 2026Hazel H. Kim, Andrew M. Bean, Guilherme Affonso Ferreira de Camargo +10LLM EvaluationLLM Reliability

    Department of Computer Science, University of Oxford · Oxford Internet Institute, University of Oxford · Thomson Reuters +7

  15. WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities

    Sep 2, 2026Jiska Beuk, Gerasimos SpanakisSocial Bias in Language ModelsLanguage Model Bias Evaluation

    Department of Advanced Computing Sciences Maastricht University

  16. Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close

    Sep 1, 2026Rania Elbadry, Ahmed Heakl, Saeed Almheiri +12Financial ServicesCultural Bias in Language Models

    ♠MBZUAI · ♢The University of Tokyo · §The Fin AI +4

  17. Authority Bias in Conversational Search Engines for Academic Paper Recommendation

    Aug 31, 2026Uthman Jinadu, Parsa Ghazvinian, Anjila Budathoki +3Language Model Bias EvaluationLarge Language Model-Based Recommendation

    Georgia State University · University of Tennessee, Knoxville

  18. LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

    Aug 31, 2026Daniela Occhipinti, Andrea Piergentili, Marco GueriniLLM-as-a-JudgeLLM Prompting

    Fondazione Bruno Kessler, Via Sommarive 18, Povo, Trento, Italy · Almawave Labs, Via Di Casal Boccone, 188/190, Rome

  19. LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark

    Aug 31, 2026Irem Yoldas, Martim Brandão, Jie Zhang +1Autonomous Driving BenchmarksAutonomous Driving Safety Evaluation

    King’s College London, UK

  20. Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions

    Aug 31, 2026Ahmed El Kady, Aravind Narayanan, Rehana Noorani +2LLM EvaluationEnergy-Efficient ML

    Vector Institute · Independent researcher · University of Calgary

  21. Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models

    Aug 31, 2026Melina Morch, Daniel BraunMultilingual Language Model EvaluationSocial Bias in Language Models

    Marburg University · Computer Science · Cultural Data Studies +2

  22. WildSEEK: Evaluating Language Models for Information-Seeking

    Aug 31, 2026Tanise Ceron, Joachim Baumann, Elisa Bassignana +3LLM Safety BenchmarksLLM Sycophancy

    1Bocconi University · 2Stanford University · 3IT University of Copenhagen +1

  23. Beyond Consensus: Downward Bias and Role Asymmetry in Multi-Agent LLM Judges for Subjective Evaluation

    Aug 31, 2026Minsoo Song, Chanwoo Kim, Sugyeong Eo +1Human Preference EvaluationLLM-as-a-Judge

    Soongsil University · Yonsei University

  24. Benevolent Bias in Multi-Turn Human-Agent Dialogue

    Aug 29, 2026Qianqi Liu, Jin Huang, Fethiye Irmak Dogan +1Social Bias in Language ModelsAlgorithmic Bias

    University of Cambridge

  25. It's How You Ask: Gender-Associated Linguistic Bias in LLMs

    Aug 13, 2026Katherine Van Koevering, Anjalie FieldGender Bias in Language ModelsSocial Bias in Language Models

    Data Science and AI Institute Johns Hopkins University · Computer Science Department Johns Hopkins University

  26. Accuracy and Order Sensitivity Diverge Under Label-Free Strategies

    Aug 12, 2026Karl Hanna, Chen FengLLM EvaluationMultiple-Choice Question Answering

    Queen’s University Belfast

  27. Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

    Aug 12, 2026Simone MungariLLM AuditingPolitical Bias in Language Models

    Revelis s.r.l.

  28. Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment

    Aug 12, 2026Haokai Zhao, Yunze Xiao, Weihao Xuan +3LLM SycophancyLLM Alignment

    University of New South Wales · Carnegie Mellon University · University of Tokyo

  29. Templated or fully synthetic? Prompt construction as a confound in measuring LLM political stance beyond writing assistance

    Aug 11, 2026Ilias ChalkidisLLM PromptingPolitical Bias in Language Models

    The National Center for AI in Society (CAISA) University of Copenhagen Denmark

  30. Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biases

    Aug 11, 2026Davood Wadi, Mohsen Ghodrat, Matthew PhilpLLM EvaluationCultural Bias in Language Models

    McGill University · University Canada West · Marketing Management +1