Language Model Bias Evaluation

Ask about this topic

Momentum

40 papers in the last four weeks, up 122% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 321

All topics
CardsList

  1. Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation

    Jul 6, 2026Sadia Kamal, Arefa Patwary, Anthony Marchiafava +2LLM EvaluationLanguage Model Generation Evaluation

    Oklahoma State University · University of North Texas

  2. The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

    Jul 6, 2026Haonan HuangMoral Reasoning in Language ModelsFraming Effects in Language Models

    Princeton University, Princeton, NJ 08540, USA

  3. Ask the Right Comparison:Bias-Aware Bayesian Active Top-kk Ranking with LLM Judges

    Jul 2, 2026Jian Xu, Delu Zeng, John Paisley +1Pairwise ComparisonLLM-as-a-Judge

    RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1

  4. Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

    Jul 1, 2026Shayan Talaei, Abhinav Chinta, Devvrit Khatri +3LLM AuditingLanguage Model Distillation

    1Stanford University · 3Foundation AI–Cisco Systems Inc. · University of Texas at Austin

  5. Evaluating Large Language Model Raters for German Open-Response Clinical Questions: A Physician-Annotated Benchmark Study of Agreement, Evaluator Bias, and Abstention

    Jul 1, 2026William Philipp, Finn Fassbender, Daniel Fister +12Multilingual Language Model EvaluationLLM-as-a-Judge

    Section for Clinical Research-IT, Institute of Medical Biometry and Statistics, University of Luebeck, Luebeck, Germany · Medical Faculty, University of Tübingen, Tübingen, Germany · German Medical Students’ Association (bvmd), Berlin, Germany +9

  6. Mapping the Evaluation Frontier: An Empirical Survey of the Bias-Reliability Tradeoff Across Eleven Evaluator-Agent Conditions

    Jul 1, 2026Zewen LiuLLM EvaluationLLM-as-a-Judge

  7. Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues

    Jun 30, 2026Mohammadamin Shafiei, Shuyue Stella Li, Yulia TsvetkovLanguage Model Safety EvaluationLanguage Model Bias Evaluation

    University of Milan · University of Washington

  8. Estimating Grammatical Gender Directions in Contextual Embeddings under Controlled and Natural Contexts

    Jun 29, 2026Huanping Xiao, Yingji LiGender Bias in Language ModelsRepresentation Disentanglement

    College of Foreign Languages and Cultures, Jilin University, Changchun, Jilin Province, China · College of Computer Science and Technology, Jilin University, Changchun, Jilin Province, China

  9. Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning

    Jun 28, 2026Craig AtkinsonAI in EducationLLM Decision-Making

    Verificate Pty Ltd, Sydney, Australia

  10. Can LLMs Hire Fairly? Racial Bias in Resume Screening

    Jun 27, 2026Zhenyu Gao, Wenxi Jiang, Yutong YanGender Bias in Language ModelsSocial Bias in Language Models

    Department of Finance, CUHK Business School, The Chinese University of Hong Kong

  11. Position Bias Correction is Insufficient for One-Pass Attention Sorting

    Jun 26, 2026Qiong Tang, Xiangkun Hu, Xiangyang Liu +2Long-Context Language ModelingLanguage Model Bias Evaluation

    Analemma

  12. To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias

    Jun 23, 2026Federico Marcuzzi, Xuefei Ning, Roy Schwartz +1Pairwise ComparisonLLM Auditing

    INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · Tsinghua University, China · The Hebrew University of Jerusalem, Israel +1

  13. UnBias-Plus: Detect, Explain, and Rewrite Bias

    Jun 22, 2026Ahmed Y. Radwan, Ahmed ElKady, Sindhuja Chaduvula +3Explainable Artificial IntelligenceSocial Bias in Language Models

    Vector Institute for Artificial Intelligence, Toronto, Canada · Independent Researcher

  14. Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory

    Jun 22, 2026Zewen LiuLLM Agent MemoryLanguage Model Bias Evaluation

    Independent Researcher

  15. Same question, different history: language, national identity, and credit in large language models

    Jun 22, 2026William Guey, Pierrick Bougault, Wei Zhang +2Social Bias in Language ModelsCultural Bias in Language Models

    Department of Industrial Engineering, Tsinghua University, Beijing, China · School of Social Sciences, Tsinghua University, Beijing, China · Department of Industrial Engineering, Federal University of Rio de Janeiro, Brazil

  16. Language-Specific Sentiment Polarity Biases in Encoder and Large Language Model Classification of Product Reviews

    Jun 22, 2026Advita Rajiv, Kavitha Kothur, Gautham ReddyMultilingual Language Model EvaluationLLM Evaluation

    John P. Stevens High School, Edison, NJ 08820 · Ford Foundation, New York, NY 10017 · NC State University, Raleigh, NC 27606

  17. Per-Entity Bias Mapping for AI Visibility: Why Brand Mentions Require Entity-Specific Calibration

    Jun 19, 2026Zoltan VargaHallucination in Language ModelsLanguage Model Bias Evaluation

    Neural Awareness Budapest, Hungary

  18. Self-Preference Is Weak or Absent in Verifiable Instruction-Following Revision: A Four-Model Test Under Genuine Authorship

    Jun 18, 2026William Guey, Pierrick BougaultLLM Self-CorrectionLanguage Model Bias Evaluation

    Department of Industrial Engineering, Tsinghua University, Beijing, China

  19. Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias

    Jun 17, 2026Justin D. Norman, Michael U. Rivera, D. Alex HughesInter-Rater ReliabilityLLM Evaluation

    UC Berkeley School of Information 102 South Hall Drive, Berkeley CA, 94708

  20. MortarBench: Evaluating Mortgage Loan Origination Agents

    Jun 17, 2026Matthew Toles, Yunan Lu, Manav Munjal +6Financial ServicesLanguage Model Calibration

    Columbia University · Tidalwave

  21. Quantifying and Auditing LLM Evaluation via Positive--Unlabeled Learning

    Jun 17, 2026Zilong Zhang, Yi-Ting Hung, Lei Ding +1Inverse Optimal TransportLLM-as-a-Judge

    Department of Mathematics and Statistics Georgia State University · Department of Statistics University of Manitoba

  22. The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs

    Jun 17, 2026Naihao Deng, Yiming Feng, Chimaobi Okite +4LLM GroundingLLM Alignment

    University of Michigan · University of Cambridge · University of Aberdeen

  23. Gender Bias in LLM Hiring Decisions: Evidence from a Japanese Context and Evaluation of Mitigation Strategies

    Jun 17, 2026Serena A. Hoffstedde, Machiko Hirota, Akshara Nadayanur Sathis Kanna +4Gender Bias in Language ModelsAI-Assisted Decision Making

    Shibaura Institute of Technology, Tokyo, Japan · Amsterdam University of Applied Sciences, Amsterdam, Netherlands · University of Pennsylvania, Philadelphia, USA +2

  24. Unintended Effects of Geographic Conditioning in Large Language Models

    Jun 16, 2026Naz Col, David M. ChanPrivacy Leakage in Language ModelsLanguage Model Bias Evaluation

    University of California, Berkeley

  25. LLMs Infer Cultural Context but Fail to Apply It When Responding

    Jun 16, 2026Yisong Miao, Jian Zhu, Vered ShwartzPragmatic Reasoning in Language ModelsCultural Bias in Language Models

    National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute

  26. Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement

    Jun 16, 2026Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed +3LLM-as-a-JudgeSocial Bias in Language Models

    University of Toronto · Vector Institute · EuroSafeAI +1

  27. Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems

    Jun 16, 2026Xi Chu, Yupeng HouGenerative Engine OptimizationLanguage Model Bias Evaluation

    Trine University · Texas A&M University

  28. Thinking Like a Scientist? A Structural Study of LLM-Generated Research Methods

    Jun 15, 2026Francesca Carlon, Brecht Verbeken, Vincent Ginis +1LLM EvaluationAI-Assisted Scientific Research

    1Data Analytics Lab, Vrije Universiteit Brussel, Pleinlaan 5, 1050 Brussels, Belgium · 2imec-SMIT, Vrije Universiteit Brussel, Pleinlaan 9, 1050 Brussels, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA

  29. P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMs

    Jun 15, 2026Rafael Ferreira, Inês Vieira, Inês Calvo +6Multilingual Language Model EvaluationSocial Bias in Language Models

    NOVA University of Lisbon, Portugal · NOVA LINCS

  30. MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions

    Jun 15, 2026Noor Islam S. Mohammad, Tamim SheikhLLM Safety BenchmarksLLM Auditing

    Department of Computer Science, Informatics Institute, Istanbul Technical University, ˙Istanbul, Türkiye · Department of Computer Science and Engineering, Jashore University of Science and Technology, Bangladesh.