Language Model Bias Evaluation

Ask about this topic

Momentum

40 papers in the last four weeks, up 122% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 321

All topics
CardsList

  1. Durable Evaluation Framework: Adversarial Arbitration for Sycophancy Reduction in Large Language Models

    Apr 21, 2026Sam RyanLLM SycophancyMulti-Agent Debate

    Novel Systems Engineering LLC

  2. Large Language Models Exhibit Normative Conformity

    Apr 21, 2026Mikako Bito, Keita Nishimoto, Kimitaka Asatani +1Normative Conformity in Language ModelsSocial Bias in Language Models

    The University of Tokyo

  3. Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs

    Apr 21, 2026Guy Mor-Lan, Omer Goldman, Matan Eyal +6Multilingual Language ModelsMultilingual Language Model Evaluation

    GGoogle Research · BBar-Ilan University · CUniversity of Cambridge +1

  4. Beyond Semantic Similarity: A Component-Wise Evaluation Framework for Medical Question Answering Systems with Health Equity Implications

    Apr 21, 2026Abu Noman Md Sakib, Md. Main Oddin Chisty, Zijie ZhangLLM Answer VerificationHealthcare

    University of Texas at San Antonio, USA · Khulna University of Engineering and Technology, Bangladesh

  5. More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models

    Apr 21, 2026Xiao WangMultiple-Choice Question AnsweringCoT Reasoning

  6. AlignCultura: Towards Culturally Aligned Large Language Models?

    Apr 21, 2026Gautam Siddharth Kashyap, Mark Dras, Usman NaseemCross-Cultural Language Model EvaluationCultural Alignment

    School of Computing, Macquarie University, Australia

  7. LegalBench-BR: A Benchmark for Evaluating Large Language Models on Brazilian Legal Decision Classification

    Apr 20, 2026Pedro Barbosa de Carvalho NetoLegal NLPLLM Evaluation

    Teresina, PI, Brazil

  8. Investigating Counterfactual Unfairness in LLMs towards Identities through Humor

    Apr 20, 2026Shubin Kim, Yejin Son, Junyeong Park +6Social Bias in Language ModelsLanguage Model Bias Evaluation

    ♡Yonsei University · ♠KAIST · ♣Seoul National University +1

  9. Six Llamas: Comparative Religious Ethics Through LoRA-Adapted Language Models

    Apr 20, 2026Chad Coleman, W. Russell Neuman, Manan Shah +5Moral Reasoning in Language ModelsCultural Bias in Language Models

  10. First, Do No Harm (With LLMs): Mitigating Racial Bias via Agentic Workflows

    Apr 20, 2026Sihao Xing, Zaur GoulievHealthcareSocial Bias in Language Models

    School of Enterprise Computing and Digital Transformation Technological University Dublin · School of Information and Communication Studies University College Dublin

  11. Contrastive Analysis of Linguistic Representations in Large Language Model Outputs through Structured Synthetic Data Generation and Abstracted N-gram Associations

    Apr 19, 2026S. A. Desimone, L. Alonso AlemanySocial Bias in Language ModelsLanguage Model Bias Evaluation

    Universidad Nacional de C´ordoba · CONICET

  12. VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

    Apr 19, 2026Yi-Cheng Lin, Yusuke Hirota, Sung-Feng Huang +1Audio-Language Model EvaluationGender Bias in Language Models

    Graduate Institute of Communication Engineering National Taiwan University Taipei, Taiwan · NVIDIA Research NVIDIA Taipei, Taiwan · AI Center of Research Excellence National Taiwan University Taipei, Taiwan

  13. Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering

    Apr 18, 2026Zixiao Zhao, Amirreza Esmaeili, Fatemeh FardSoftware EngineeringLLM-as-a-Judge

    University of British Columbia, Canada

  14. Expressing Social Emotions: Misalignment Between LLMs and Human Cultural Emotion Norms

    Apr 18, 2026Sree Bhattacharyya, Manas Mehta, Leona Chen +4LLM EvaluationCultural Bias in Language Models

    The Pennsylvania State University · Duke University · Northeastern University +1

  15. Mitigating Prompt-Induced Cognitive Biases in General-Purpose AI for Software Engineering

    Apr 18, 2026Francesco Sovrano, Gabriele Dominici, Alberto BacchelliSoftware EngineeringLLM Prompting

    Collegium Helveticum at ETH Zurich, Switzerland · University of Zurich, Switzerland · University of Italian-speaking Switzerland, Switzerland

  16. Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation

    Apr 17, 2026Nicolò Pagan, Christopher Barrie, Chris Andrew Bail +1LLM PromptingSocial Bias in Language Models

    University of Zurich, Department of Informatics, Zurich, CH · New York University, Department of Sociology, New York, USA · University of Oxford, Department of Sociology, Oxford, UK +2

  17. Context Over Content: Exposing Evaluation Faking in Automated Judges

    Apr 16, 2026Manan Gupta, Inderjeet Nair, Lu Wang +1LLM-as-a-JudgeLanguage Model Safety Evaluation

    1BITS Pilani, India · University of Michigan

  18. SPAGBias: Uncovering and Tracing Structured Spatial Gender Bias in Large Language Models

    Apr 16, 2026Binxian Su, Haoye Lou, Shucheng Zhu +4Gender Bias in Language ModelsLanguage Model Bias Evaluation

    School of Information Science, Beijing Language and Culture University, Beijing, China · 2Libraries, Renmin University of China, Beijing, China · School of Humanities, Tsinghua University, Beijing, China +2

  19. VoxSafeBench: Not Just What Is Said, but Who, How, and Where

    Apr 16, 2026Yuxiang Wang, Hongyu Liu, Yijiang Xu +9Language Model Safety EvaluationPrivacy Leakage in Language Models

    1The Chinese University of Hong Kong, Shenzhen · 2Shenzhen Loop Area Institute · 3Amphion Technology Co., Ltd.

  20. How Does "English (US)" Become the Default? Triangulating Structural Bias Towards American English Across the LLM Pipeline

    Apr 5, 2026Mir Tafseer Nayeem, Davood RafieiLanguage Model PretrainingLLM Auditing

    Department of Computing Science University of Alberta

  21. Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF

    Mar 22, 2026K. M. Jubair Sami, Dipto Sumit, Ariyan Hossain +1Social Bias in Language ModelsLanguage Model Bias Evaluation

    Department of Computer Science and Engineering BRAC University, Dhaka, Bangladesh

  22. MultiwayPAM: Multiway Partitioning Around Medoids for LLM-as-a-Judge Score Analysis

    Mar 11, 2026Chihiro Watanabe, Jingyu SunLLM-as-a-JudgeClustering

    NTT Computer and Data Science Laboratories, 3-9-11, Midori-cho, Musashino-shi, Tokyo, Japan

  23. Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization

    Mar 9, 2026Hongli Zhou, Hui Huang, Rui Zhang +5LLM-as-a-JudgeSocial Bias in Language Models

    Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, 518055, China

  24. QEDBENCH: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs

    Feb 24, 2026Santiago Gonzalez, Alireza Amiri Bavandpour, Peter Ye +48Mathematical Reasoning BenchmarksLLM Evaluation

  25. FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

    Feb 6, 2026Bo Yang, Lanfei Feng, Yunkui Chen +3LLM EvaluationLLM-as-a-Judge

    College of Computer Science, Zhejiang University, Hangzhou, China · School of Software Technology, Zhejiang University, Hangzhou, China

  26. Investigating Social Bias Changes in Quantized Language Models

    Feb 5, 2026Stanley Z. Hua, Sanae Lotfi, Irene Y. ChenLLM QuantizationSocial Bias in Language Models

    UC Berkeley & UCSF · Meta Superintelligence Labs

  27. Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge

    Feb 2, 2026Yuzheng Xu, Tosho Hirasawa, Tadashi Kozuno +1LLM-as-a-JudgeLanguage Model Bias Evaluation

    OMRON SINIC X · NexaScience

  28. Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations

    Jan 30, 2026Dani Roytburg, Matthew Bozoukov, Matthew Nguyen +3LLM-as-a-JudgeLanguage Model Generation Evaluation

    Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA, USA · Apart Research, San Francisco, California, USA · Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA, USA +2

  29. SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

    Jan 29, 2026Ahmed Y. Radwan, Christos Emmanouilidis, Hina Tabassum +2Audio-Language Model EvaluationMultimodal Grounding

    Vector Institute for Artificial Intelligence, MaRS Centre, Toronto, ON M5G 1L7, Canada · University of Groningen, Nijenborgh 4, 9747 AG Groningen, Netherlands · York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada

  30. White-Box Sensitivity Auditing with Steering Vectors

    Jan 23, 2026Hannah Cyberey, Yangfeng Ji, David EvansAlgorithmic AuditingLLM Auditing

    University of Virginia