LLM Evaluation

LLM: Large Language Model

Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,643

All topics
CardsList

  1. A dataset of rated conceptual arguments

    Jul 29, 2026Emery Cooper, Caspar Oesterheld, Linh Chi Nguyen +2LLM Evaluation

  2. Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models

    Jul 29, 2026Parishruthi Ganesh, Gerry Dozier, Cheryl SealsLLM EvaluationZero-Shot Text Classification

    Department of Computer Science and Software Engineering Auburn University Auburn, Alabama, USA

  3. Dimensionality and Measurement Precision in HLE's Multiple-Choice Subset

    Jul 29, 2026Mayank Sharma, Savira Nadela, Tyler MattesonLLM EvaluationPsychometric Validation

    Stanford University

  4. Benchmarking LLM Competence on Logical Inference over Probability Operators

    Jul 29, 2026Nayera Hasan, Jack Greff, Alvin GrissomProbabilistic LogicLLM Evaluation

    Haverford College

  5. APEX-Accounting

    Jul 29, 2026Julien Benchek, Austin Bennett, Jasmin Kern +8LLM EvaluationBusiness Process Automation

    1Mercor · 2Ramp

  6. BayesAME: Bayesian Active Model Evaluation

    Jul 29, 2026Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet +2LLM EvaluationAdaptive Sampling

    Imperial College London (work conducted while at Google DeepMind) · Google DeepMind

  7. ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

    Jul 29, 2026Ruxi Gu, Zhenliang Zhang, Wei WangKnowledge EditingContinual Learning for LLMs

    Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.

  8. Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification

    Jul 29, 2026Linyu Li, Zhi Jin, Yichi Zhang +6Retrieval-Augmented GenerationLLM Evaluation

    School of Computer Science, Peking University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · School of Computer Science, Tibet University, Lhasa, China

  9. When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses

    Jul 28, 2026Zihan Chen, Di Zhu, Lei Nico ZhengLLM EvaluationLarge Language Model-Based Social Simulation

    Stevens Institute of Technology Hoboken, NJ, USA · University of Massachusetts Boston Boston, MA, USA

  10. Position: Evaluation Scores Are Perishable Knowledge Claims

    Jul 28, 2026Sankalp Gilda, Shlok GildaLLM EvaluationLLM Reliability

    DeepThought Solutions · Meta · University of Florida

  11. AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology II: Project Planning and Proposal Evaluation

    Jul 28, 2026Jia Liu, Veena Krishnaraj, Kateryna Vovk +18LLM EvaluationAI-Assisted Scientific Research

  12. Inspect India Evals: An Open Benchmarking Framework for Evaluating Large Language Models in the Indian Linguistic and Cultural Context

    Jul 28, 2026Abhishek Kumar Singh, Shrey Nag, Sachita +2LLM Safety BenchmarksMultilingual Language Model Evaluation

    GTBIT¹ · TIET² · IGDTUW³ +2

  13. Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

    Jul 28, 2026Juan Francisco, Mandujano ReyesLLM EvaluationLanguage Model Calibration

    Independent Research

  14. Evaluating Communicative Belief Updates in Large Language Models via Implicature Recognition and Cancellation

    Jul 27, 2026Cesare Spinoso-Di Piano, Verna Dankers, Marius Mosbach +1LLM EvaluationPragmatic Reasoning in Language Models

    Mila - Quebec AI Institute & McGill University · Canada CIFAR AI Chair

  15. CogArena: A Multimethod Evaluation of Cognitive Ability Structure in Large Language Models

    Jul 27, 2026Dengzhe Hou, Lingyu Jiang, Fangzhou Lin +1LLM EvaluationCognitive Modeling

    1Graduate School of Information Sciences, Tohoku University · 2Unprecedented-scale Data Analytics Center, Tohoku University · 3Texas A&M University +1

  16. Preliminary Guidelines for Using and Evaluating GenAI Tools to Support Systematic Literature Reviews

    Jul 27, 2026Barbara Kitchenham, Sebastián Pizard, Lech Madeyski +3LLM EvaluationHuman-in-the-Loop Evaluation

    Keele University, United Kingdom · Universidad de la República, Uruguay · Wrocław University of Science and Technology, Poland +3

  17. LLM-SoccerArena: Benchmarking LLMs on Real-World Predictions in Sports

    Jul 27, 2026Jonas Schröder, Jonas Schweisthal, Oliver Müller +2LLM EvaluationForecasting Benchmarks

    MCML & LMU Munich Munich, Germany · Paderborn University Paderborn, Germany · University of Cologne Cologne, Germany

  18. INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models

    Jul 27, 2026Changyu Chen, Chenwei Lin, Xian XuNumerical Reasoning in Language ModelsLLM Evaluation

    Fudan University

  19. Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets

    Jul 27, 2026Zongyou Yang, Yinghan HouLLM EvaluationLLM Answer Verification

    Dyson School of Design Engineering Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom

  20. Tag Questions and the Generational Reversal of Sycophancy Across 45 Language Models

    Jul 27, 2026Tapan ParikhLLM EvaluationLLM Sycophancy

    Cornell Tech

  21. Understanding Tone-Dependent Inference Cost in Large Language Models

    Jul 27, 2026Akhil Kumar, Om DobariyaLLM Inference EfficiencyLLM Evaluation

    Pennsylvania State University University Park, PA, USA

  22. How Context Attribution Handles What the Model Already Knows

    Jul 26, 2026Quoc-Huy Trinh, Lin Zhu, Sebastian SzyllerLLM EvaluationLLM Interpretability

    Aalto University

  23. Zing: Social Mind for LLMs

    Jul 26, 2026Zing Team, Ao Xiang, Bi Jingping +56LLM EvaluationLLM Grounding

  24. Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning

    Jul 26, 2026Risako Ando, Koji MineshimaLLM EvaluationLLM Reasoning

    Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan

  25. Do LLM Debates Repeat Arguments Differently Across Languages?

    Jul 26, 2026Huiqian LaiMultilingual Language Model EvaluationLLM Evaluation

    Syracuse University

  26. Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?

    Jul 26, 2026Hai Hu, Siyuan Song, Chongtian Shao +3LLM EvaluationMemorization in Language Models

    1The Hong Kong Polytechnic University · 2The University of Texas at Austin · 3Shanghai Jiao Tong University

  27. TLA+-Bench: An Execution-Grounded Benchmark and Dataset for Natural-Language to TLA Specification Generation

    Jul 26, 2026Arslan Bisharat, Eric Spencer, Brian Ortiz +8LLM EvaluationFormal Verification

    Loyola University Chicago Chicago, Illinois, USA

  28. BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi

    Jul 25, 2026Hariom Ingle, Ronit Ghode, Ishwari Gondkar +2Named Entity RecognitionLLM Evaluation

    Department of Information Technology, PICT, Pune, India · L3Cube Labs, Pune, India · Indian Institute of Technology Madras, Chennai, India

  29. ESF-Bench: Benchmarking Challenging Slot-Filling Scenarios for Real-World Enterprise Applications

    Jul 25, 2026Toby Liang, Gopal Sarda, Sagar Davasam +1LLM Evaluation

    ServiceNow

  30. Guarantees on Dynamical System Distinguishability for LLM Token Generation

    Jul 25, 2026Mohamed Akrout, Dan WilsonLLM EvaluationClassification

    Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville, TN, USA