LLM Evaluation

LLM: Large Language Model

Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,643

All topics
CardsList

  1. Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

    Jul 9, 2026Matteo Santelmo, Xiuying Wei, Israa Fakih +5VLM EvaluationLLM Evaluation

    Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland

  2. Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench

    Jul 9, 2026Siddhartha Jain, Ameya VelingkerLLM EvaluationLong-Context Language Model Evaluation

    NVIDIA

  3. ICDAR 2026 HIPE-OCRepair Competition on LLM-Assisted OCR Post-Correction for Historical Documents

    Jul 9, 2026Maud Ehrmann, Emanuela Boros, Juri Opitz +3Historical Document AnalysisLLM Evaluation

    1 École Polytechnique Fédérale de Lausanne (EPFL), Switzerland · University of Zurich, Switzerland

  4. From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs

    Jul 9, 2026Yi Zhang, Julia RayzLLM EvaluationComputer Science Education

    School of Applied Creative Computing Purdue University West Lafayette, IN 47907, USA

  5. Validating LLMs in social science: Epistemic threats and emerging norms

    Jul 8, 2026Meera Desai, Dallas Card, Abigail Z. JacobsLLM EvaluationComputational Social Science

    School of Information, University of Michigan, Ann Arbor, USA · Center for the Study of Complex Systems, University of Michigan, Ann Arbor, USA.

  6. Predicting LLM Safety Before Release by Simulating Deployment

    Jul 8, 2026Marcus Williams, Hannah Sheahan, Cameron Raymond +8LLM EvaluationAI Risk Management

    OpenAI

  7. Measuring Intelligence Beyond Human Scale

    Jul 8, 2026Jerry Han, Rafael Moschopoulos, Ella Colby +6LLM EvaluationAI Agent Evaluation

    Princeton Superalignment

  8. Comprehensive Evaluation of Large Language Model Responses: A Multi-Factor Scoring System

    Jul 8, 2026Yiming Gai, Junde Lu, Xuefei HuangLLM EvaluationQuestion Answering

    School of Computer Science and Engineering, Beihang University, Beijing, China · Data Science and Intelligent Computing Laboratory, Hangzhou International Innovation Institute, Beihang University, Hangzhou, Zhejiang 311115, P.R.China

  9. REFORGE: A Method for Benchmarking LLMs' Reverse Engineering Capabilities in Decompiled Binary Function Naming

    Jul 7, 2026Nicolas Koller, Andreas u. SchmidtLLM EvaluationSoftware Reverse Engineering

    Systrion AG Flughafenstraße 52 22335 Hamburg, Germany · Wilhelm Büchner Hochschule Hilpertstraße 31 64295 Darmstadt, Germany

  10. Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities

    Jul 7, 2026So Hasegawa, Shailaja Keyur Sampat, Lei Liu +1Table QALLM Evaluation

    Fujitsu Research of America, Santa Clara CA 95054, USA

  11. Evaluating Neural Decompilation of Dart AOT Binaries: Fine-Tuning, Metric Validity, Specification Leakage, and Reliability

    Jul 7, 2026Raafat Abualazm, Ayman AboElhassan, Amr G. WassalSoftware EngineeringLLM Evaluation

    Cairo University, Computer Engineering Department, Faculty of Engineering, Giza, Egypt

  12. Auto-DSM Under the Lens: A Black-Box Evaluation Framework for LLM-Based DSM Generation

    Jul 7, 2026Niels Potters, Theo HofmanLLM EvaluationLLM Auditing

    Eindhoven University of Technology (TU/e), Dept. of Mechanical Engineering, Control Systems Technology section, Engineering Systems Design group, P.O.Box 513, 5600 MB Eindhoven, The Netherlands.

  13. Integrating knowledge graphs and multilingual scholarly corpora for domain-adaptive LLMs in SSH

    Jul 7, 2026Adam Faci, Alessio Miaschi, Anne Combe +4Multilingual Language ModelsLLM Evaluation

    Huma-Num, CNRS, Paris, France · CNR - Institute for Computational Linguistics “A. Zampolli” (CNR-ILC), Pisa, Italy · Inria, France +1

  14. SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

    Jul 7, 2026Yimeng Zhang, Yingying Zhuang, Ziyi Wang +12LLM EvaluationConfidence Estimation in Language Models

    Amazon · Northeastern University

  15. RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs

    Jul 6, 2026Damian Hodel, Jevin West, Aylin CaliskanLLM EvaluationSocial Bias in Language Models

    University of Washington, Seattle, USA

  16. EvalLoop: A Methodology for Evaluation-Driven Iterative Improvement of Business AI Systems

    Jul 6, 2026Kenneth Benavides, Josh Fleischer, Danti ChenLLM EvaluationLLM Reliability

    Robert Half

  17. BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

    Jul 6, 2026Abu Tyeb Azad, Ishita Sur Apan, Fahim Ahmed +8Document UnderstandingLLM Evaluation

    Wichita State University, USA · Center for Computational & Data Sciences, Bangladesh · University of Dhaka, Bangladesh +1

  18. CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

    Jul 6, 2026H. Chad Lane, Bryson KagelerLLM EvaluationIntelligent Tutoring Systems

    University of Illinois Urbana-Champaign, Champaign, IL, USA

  19. Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation

    Jul 6, 2026Sadia Kamal, Arefa Patwary, Anthony Marchiafava +2LLM EvaluationLanguage Model Generation Evaluation

    Oklahoma State University · University of North Texas

  20. Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

    Jul 6, 2026Yibo Hu, Jiaming QuLLM EvaluationNormative Conformity in Language Models

    Illinois Institute of Technology · Amazon

  21. Rating the Pitch, Not the Product: User Evaluations of LLMs Reflect Expectations More Than Performance

    Jul 6, 2026Robert Morabito, Tyler McDonald, Charitra Viswanath +4Human Preference EvaluationLLM Evaluation

    Brock University · Emory University · University of Southern California +2

  22. Evaluating Large Language Models for Antisemitic Incident Classification

    Jul 6, 2026Karina Halevy, Julia Mendelsohn, Chan Young Park +2LLM EvaluationHate Speech Detection

    Carnegie Mellon University · University of Maryland · Microsoft Research +2

  23. URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment

    Jul 6, 2026Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev +7LLM EvaluationLLM Planning

    Insilico Medicine AI Limited, Masdar City, Abu Dhabi, UAE · Independent researcher · Insilico Medicine Canada Inc., Montreal, Quebec, Canada +1

  24. evalci: A Python Library for Statistically Rigorous Comparison of Language Model Evaluations

    Jul 5, 2026Shreyas K ChandrahasLLM EvaluationPairwise Comparison

  25. Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

    Jul 4, 2026Diego Cerda-Mardini, Sarath Chandar, Sreenath MadathilLLM EvaluationProbability Calibration

    Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montréal, QC, Canada · Chandar Research Lab, Polytechnique Montréal, Montréal, H3T 1J4, Canada · Mila – Québec Artificial Intelligence Institute, Montréal, H2S 3H1, Canada +2

  26. Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth

    Jul 4, 2026Ido Amit, Ido Galil, Ran El-YanivLLM EvaluationLLM Uncertainty Estimation

    1Technion · 2Nvidia

  27. What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings

    Jul 3, 2026Teng-Ruei ChenLLM EvaluationLanguage Model Decoding

    Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu 300, Taiwan · Krixvon AI, Taiwan

  28. Spectral Signatures of Large Language Models

    Jul 3, 2026Zhuoying Zhang, Ishan V. Prasad, Yuanzhe Hu +4LLM EvaluationLanguage Model Fingerprinting

    Dartmouth College Hanover, NH, USA · University of California, Berkeley Berkeley, CA, USA · Lawrence Berkeley National Laboratory Berkeley, CA, USA

  29. Which Algorithm Specification Formats Help Language Models Implement Machine Learning Algorithms?

    Jul 3, 2026Masahiro Kato, Taka KatoLLM EvaluationProgram Synthesis

    Data Analytics Department, Mizuho-DL Financial Technology, Co., Ltd. · NP-hard Inc.

  30. Silicon Sampling via Cross-Survey Transfer

    Jul 3, 2026Chan-Tung Ku, Chan Hsu, Pei-Cing Huang +3LLM EvaluationTransfer Learning

    Department of Information Management National Sun Yat-sen University Kaohsiung, Taiwan · Institute of Political Science National Sun Yat-sen University Kaohsiung, Taiwan · Graduate Institute of Library and Information Science National Chung Hsing University Taichung, Taiwan