LLM Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,643

All topics

Analyzing LLM Reasoning to Uncover Mental Health Stigma
Apr 27, 2026Sreehari Sankar, Aliakbar Nafar, Mona Barman +8LLM Evaluation Social Bias in Language Models
BetterHelp
The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models
Apr 27, 2026Yunze Xiao, Vivienne J. Zhang, Chenghao Yang +3LLM Evaluation Personality Modeling in Language Models
CMU · UChicago · MIT +3
Can LLMs Act as Historians? Evaluating Historical Research Capabilities of LLMs via the Chinese Imperial Examination
Apr 27, 2026Lirong Gao, Zeqing Wang, Yuyan Cai +6LLM Evaluation Question Answering
State Key Laboratory of Blockchain and Data Security, Zhejiang University · Ant Group · Zhejiang University
Benchmarking Source-Sensitive Reasoning in Turkish: Humans and LLMs under Evidential Trust Manipulation
Apr 27, 2026Sercan Karakaş, Yusuf ŞimşekLLM Evaluation
University of Chicago · Fırat University
K-MetBench: A Multi-Dimensional Benchmark for Fine-Grained Evaluation of Expert Reasoning, Locality, and Multimodality in Meteorology
Apr 27, 2026Soyeon Kim, Cheongwoong Kang, Myeongjin Lee +3LLM Evaluation Cultural Bias in Language Models
KAIST, INEEJI · Seongnam, Korea · KAIST +2
Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity
Apr 27, 2026Bojie LiParameter Estimation LLM Evaluation
Pine AI
STELLAR-E: a Synthetic, Tailored, End-to-end LLM Application Rigorous Evaluator
Apr 27, 2026Alessio Sordo, Lingxiao Du, Meeka-Hanna Lenisa +2LLM Evaluation Synthetic Data Generation
1Deutsche Bank, Berlin Technology Center, Berlin, Germany
Programming with Data: Test-Driven Data Engineering for Self-Improving LLMs from Raw Corpora
Apr 27, 2026Chenkai Pan, Xinglong Xu, Yuhang Xu +6LLM Evaluation Knowledge Augmentation for Language Models
Zhejiang University · University of Chinese Academy of Sciences · Shanghai Artificial Intelligence Laboratory
Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style
Apr 27, 2026Connor Baumler, Calvin Bao, Huy Nghiem +3LLM Evaluation Controllable Text Generation
University of Maryland
SWE-QA: A Dataset and Benchmark for Complex Code Understanding
Apr 27, 2026Laïla Elkoussy, Julien PerezLLM Evaluation Software Engineering Benchmarks
LRE, EPITA · LRE, EPITA* · EPITA, LRE +1
Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation
Apr 27, 2026Zekun Yuan, Yangfan Ye, Xiaocheng Feng +5LLM Evaluation Cultural Knowledge in Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Huawei Technologies Co., Ltd
Multi-Dimensional Evaluation of Sustainable City Trips with LLM-as-a-Judge and Human-in-the-Loop
Apr 27, 2026Ashmi Banerjee, Adithi Satish, Wolfgang Wörndl +1LLM Evaluation LLM-as-a-Judge
Technical University of Munich Munich, Germany · Polytechnic University of Bari Bari, Italy
An Information-Geometric Framework for Stability Analysis of Large Language Models under Entropic Stress
Apr 27, 2026Hikmat Karimov, Rahid Zahid AlekberliLLM Evaluation LLM Reliability
Institute of Defense Technologies and Cybersecurity„ Azerbaijan Technical University, Azerbaijan
How LLMs Fail and Generalize in RTL Coding for Hardware Design?
Apr 26, 2026Guan-Ting Liu, Chao-Han Huck Yang, Chenhui Deng +3LLM Evaluation Code Generation
NVIDIA Research
Expert Evaluation of LLM's Open-Ended Legal Reasoning on the Japanese Bar Exam Writing Task
Apr 26, 2026Jungmin Choi, Keisuke Sakaguchi, Hiroaki YamadaLLM Evaluation Hallucination in Language Models
RIKEN · Tohoku University · Tokyo Metropolitan University
Benchmarking Testing in Automated Theorem Proving
Apr 26, 2026Jongyoon Kim, Hojae Han, Seung-won HwangLLM Evaluation Automated Theorem Proving
Interdisciplinary Program in Artificial Intelligence, Seoul National University1 · Electronics and Telecommunications Research Institute2
The Collapse of Heterogeneity in Silicon Philosophers
Apr 26, 2026Yuanming Shi, Andreas HauptLLM Evaluation
Adobe Inc., San Jose, California, USA · Stanford University, Departments of Economics and Computer Science, Stanford, California, USA
JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems
Apr 26, 2026Rohith Reddy Bellibatlu, Edward Raff, Wenbin ZhangLLM Evaluation LLM-as-a-Judge
Florida International University, Miami, FL · CrowdStrike, New York, USA · University of Maryland Baltimore County, Baltimore, USA
Automating Categorization of Scientific Texts with In-Context Learning and Prompt-Chaining in Large Language Models
Apr 25, 2026Gautam Kishore Shahi, Oliver HummelLLM Evaluation LLM Prompting
Technische Hochschule Mannheim, Germany
An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code
Apr 25, 2026Jelena Ilić VulićevićLLM Evaluation On-Device Language Model Inference
Independent Researcher
Evaluating Large Language Models on Computer Science University Exams in Data Structures
Apr 25, 2026Edan Gabay, Yael Maoz, Jonathan Stahl +7LLM Evaluation Computer Science Education
Blavatnik School of Computer Science and AI
Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective
Apr 25, 2026Bishwamittra Ghosh, Soumi Das, Till Speicher +5LLM Evaluation LLM Fine-Tuning
Max Planck Institute for Software Systems, Germany · Boston University, USA
Evaluating Temporal Consistency in Multi-Turn Language Models
Apr 24, 2026Yash Kumar Atri, Steven L. Johnson, Tom HartvigsenLLM Evaluation Temporal Reasoning in Language Models
University of Virginia
A Systematic Approach for Large Language Models Debugging
Apr 24, 2026Basel Shbita, Anna Lisa Gentile, Bing Zhang +10LLM Evaluation
1IBM Research, San Jose, CA, USA · 2IBM Research, Bengaluru, India
AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
Apr 24, 2026Pouya Pezeshkpour, Estevam HruschkaLLM Evaluation LLM Answer Verification
Megagon Labs
Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity
Apr 24, 2026Erez Yosef, Oron Anschel, Shunit Haviv Hakimi +4Mathematical Reasoning Benchmarks LLM Evaluation
Tel-Aviv University · Amazon Prime Video · Ben-Gurion University
BLAST: Benchmarking LLMs with ASP-based Structured Testing
Apr 24, 2026Manuel Alejandro Borroto Santana, Erica Coppolillo, Francesco Calimeri +3Logic Programming LLM Evaluation
University of Calabria, Rende, Italy · ICAR-CNR, Rende, Italy · Artificial Intelligence LAB, DeMaCS, University of Calabria, Rende, Italy +1
Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen
Apr 24, 2026Jon-Paul CacioliLLM Evaluation Confidence Estimation in Language Models
Independent Researcher, Melbourne, Australia
An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models
Apr 24, 2026Juan Manuel ContrerasLLM Evaluation Language Model Self-Assessment
Independent Researcher
When AI Speaks, Whose Values Does It Express? A Cross-Cultural Audit of Individualism-Collectivism Bias in Large Language Models
Apr 24, 2026Pruthvinath Jeripity VenkataLLM Evaluation Cross-Cultural Language Model Evaluation
Independent Researcher

Apr 27, 2026, cs.CLJ/K move · Enter open · S save

Analyzing LLM Reasoning to Uncover Mental Health Stigma

Sreehari Sankar, Aliakbar Nafar, Mona Barman, Hannah K. Heitz+7

BetterHelp

LLM Evaluation Social Bias in Language Models Clinical Language Model Evaluation

Open paper AskPDF arXiv

While large language models (LLMs) are increasingly being explored for mental health applications, recent studies reveal that they can exhibit stigma toward individuals with psychological conditions. Existing evaluations of this stigma primarily rely on multiple-choice questions (MCQs), which fail to capture the biases embedded within the models' underlying logic. In this paper, we analyze the intermediate reasoning steps of LLMs to uncover hidden stigmatizing language and the internal rationales driving it. We leverage clinical expertise to categorize common patterns of stigmatizing language directed at individuals with psychological conditions and use this framework to identify and tag problematic statements in LLM reasoning. Furthermore, we rate the severity of these statements, distinguishing between overt prejudice and more subtle, less immediately harmful biases. To broaden the reasoning domain and capture a wider array of patterns, we also extend an existing mental health stigma benchmark by incorporating additional psychological conditions. Our findings demonstrate that evaluating model reasoning not only exposes substantially more stigma than traditional MCQ-based methods but also helps identify the flaws in the LLMs' logic and their understanding of mental health conditions.

Momentum

Related topics

Latest papers 1,643