LLM Evaluation

LLM: Large Language Model

Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,643

All topics
CardsList

  1. VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design

    Jul 20, 2026Depeng Su, Yuyu Luo, Guobiao HuLLM Evaluation

    1Internet of Things Thrust, Information Hub 2Data Science and Analytics Thrust, Information Hub The Hong Kong University of Science and Technology (Guangzhou)

  2. Do Language Models Dream of Binding Molecules? Benchmarking LLMs under Spatial Constraints

    Jul 20, 2026Thomas MacDougall, Maksim Kuznetsov, Roman Schutski +5Constrained Generative ModelingLLM Evaluation

    Insilico Medicine Canada Inc. · Insilico Medicine Hong Kong Ltd. · Insilico Medicine AI Limited

  3. WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

    Jul 20, 2026Zhaokai Wang, Tianlin Gui, Jiayuan Rao +3LLM EvaluationForecasting Benchmarks

    Shanghai Jiao Tong University · Shanghai, China · Nanjing University +4

  4. Human Grounded Evaluation of Large Language Models for Optical Network Automation

    Jul 20, 2026Kiarash Rezaei, Omran Ayoub, Paolo Monti +1LLM Inference EfficiencyLLM Evaluation

    Department of Electrical Engineering, Chalmers University of Technology, 412 96 Gothenburg, Sweden · University of Applied Sciences and Arts of Southern Switzerland, 6928 Lugano, Switzerland

  5. Pancasila-Dilemmas: Evaluating Large Language Models on Indonesian Human Value Dilemmas Grounded in Pancasila

    Jul 20, 2026Supryadi, Irfan, Julianti +4LLM EvaluationLLM Alignment

    TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · Universitas Universal, Indonesia · Beijing Language and Culture University, Beijing, China +2

  6. Evaluating Federated Pre-Training: On the Reliability of Downstream Fine-Tuning and Intrinsic Evaluation

    Jul 20, 2026Claudia Grosser, Maike Heuer, Denis Krompass +1Language Model PretrainingLLM Evaluation

    Technical University Munich · Siemens AG

  7. HALLMARK: Diagnosing Three Failure Modes in LLM Citation Verifiers

    Jul 20, 2026Patrik Reizinger, Wieland BrendelLLM EvaluationCitation Verification

    Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen

  8. Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field

    Jul 20, 2026Tanay Aggarwal, Angelo Salatino, Francesco Osborne +1Relation ExtractionLLM Evaluation

    Knowledge Media Institute, The Open University, Milton Keynes, UK · Department of Business and Law, University of Milano-Bicocca, Milan, IT

  9. WuYu-EnvLE-Bench: A Benchmark for Evaluating Large Language Models in Environmental Law Enforcement

    Jul 20, 2026Ziliang Yang, Yi Zhang, Kaijun Lin +3LLM EvaluationLLM Grounding

    School of Environment, Tsinghua University · College of Economics and Management, Beijing University of Technology · State Key Laboratory of Iron and Steel Industry Environmental Protection, School of Environment, Tsinghua University +1

  10. Large Language Models for Citation Function Classification

    Jul 20, 2026Daniel Vodička, Jakub Šmíd, Pavel Král +1LLM EvaluationLLM Fine-Tuning

    Department of Computer Science and Engineering · NTIS – New Technologies for the Information Society University of West Bohemia in Pilsen, Faculty of Applied Sciences Univerzitní 2732/8, 301 00 Pilsen, Czech Republic · CNRS LORIA, Université de Lorraine, Nancy, France

  11. Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration

    Jul 20, 2026Jie HuLLM EvaluationLLM Answer Verification

    Research Institute of China Telecom

  12. Efficient Sequential Evaluation of Large Language Models

    Jul 19, 2026Chia-Yu Hsu, Shubhanshu ShekharLLM EvaluationConfidence Estimation in Language Models

    EECS Department, University of Michigan

  13. Literary Non-Style in LLM-Generated Text

    Jul 19, 2026Cory MassaroLLM Evaluation

  14. A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents

    Jul 19, 2026Yunze HanSupervised Fine-TuningLLM Evaluation

    Vdynamics GmbH, 80935 Munich, Germany

  15. A Consensus-Based Framework for Relative Preference Evaluation of Large Language Models

    Jul 19, 2026Mohtashim KhanLLM EvaluationLLM-as-a-Judge

  16. A Systematic Evaluation of Traditional Privacy Policy Analysis Tools Against LLMs

    Jul 19, 2026Madhav Aryal, Sudipa Saha, Sunil Manandhar +2Privacy AuditingLLM Evaluation

    University of South Florida · IBM T.J. Watson Research Center

  17. When LLMs Over-Answer: Measuring and Mitigating Quality Issues in LLM-Based Hardware Description Language Question Answering

    Jul 19, 2026Ziteng Hu, Jiachi Chen, Wenhao Lv +2LLM EvaluationElectronic Design Automation

    Micro-Electronics Research Institute, Hangzhou Dianzi University Hangzhou, Zhejiang, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University Hangzhou, Zhejiang, China · College of Information Science and Engineering, Hunan Normal University Changsha, Hunan, China

  18. Solver-Hard Is Not Model-Hard: A Hardness-Controlled Diagnostic for LLM Constraint Reasoning

    Jul 19, 2026Lucky VermaLLM EvaluationConstraint Satisfaction

    Independent Researcher

  19. JOR-Bench: Japanese Operations Research Benchmarks for Large Language Models

    Jul 18, 2026Yuu JinnaiMultilingual Language Model EvaluationLLM Evaluation

    CyberAgent Tokyo, Japan

  20. Encoding EEG Signals to Examine Human-Like Next-Word Prediction Behaviour in Language Models

    Jul 17, 2026Boi Mai Quach, Binh T. Nguyen, Cathal Gurrin +1LLM Evaluation

    School of Computing, ML-Labs, Department of Computer Science, School of Computing, ADAPT Centre, Dublin City University, Ireland · VNUHCM – University of Science · School of Computing, ADAPT Centre, Dublin City University, Ireland

  21. CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data

    Jul 17, 2026Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru +3LLM EvaluationLLM Fine-Tuning

    Scale AI

  22. StabilityBench: Benchmarking Instability in LLMs

    Jul 17, 2026Emma Kondrup, Zachary Yang, Anne Imouza +1LLM EvaluationLanguage Model Generation Evaluation

    1Mila — Quebec AI Institute · 2McGill University · 3Ubisoft La Forge

  23. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning

    Jul 17, 2026Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2LLM EvaluationBenchmark Design

    Operations, Information and Decisions Department, The Wharton School, University of Pennsylvania, Philadelphia, 19104, PA, USA · Heinz College of Information Systems and Public Policy, Carnegie Mellon University, Pittsburgh, 15213, PA, USA · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, 19104, PA, USA +1

  24. Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery

    Jul 17, 2026Tianyun Zhong, Wangyi Jiang, Wei Wang +15LLM EvaluationScientific Hypothesis Generation

    University of Chinese Academy of Sciences · Institute of Software, Chinese Academy of Sciences · 3Alibaba Group

  25. Neuro-Symbolic AI for LEED compliance: Document-Centric Benchmarking, Deterministic Numeric Checking, and When Multimodal Hurts

    Jul 17, 2026Aritro De, Juliana FelknerLLM EvaluationNeuro-Symbolic AI

    The University of Texas at Austin · The University of Texas at Austin,, Austin 78712, Texas, United States of America

  26. Can We Trust Item Response Theory for AI Evaluation?

    Jul 16, 2026Han Jiang, Sunbeom Kwon, Jinwen Luo +2LLM EvaluationItem Response Theory

    1Johns Hopkins University · University of Illinois Urbana-Champaign · University of California, Los Angeles

  27. Routing Ceilings Are Domain-Independent: Structural Prior Injection in Code Security Vulnerability Detection

    Jul 16, 2026Manuel Israel CázaresLLM EvaluationSoftware Vulnerability Detection

    Bytepro AI Mazatlán, Sinaloa, Mexico

  28. How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts

    Jul 16, 2026Steven Coyne, Diana Galvan-Sosa, Ryan Spring +4LLM EvaluationEducational Assessment

    Tohoku University, Sendai, Japan · RIKEN, Wako, Japan · ALTA Institute, Computer Laboratory, University of Cambridge, Cambridge, UK +2

  29. Controlled Reformulation Testing for Logical Consistency in Large Language Models

    Jul 16, 2026Alexander Gu, Alan ChenLLM EvaluationReasoning Consistency in Language Models

    University of Texas at Austin

  30. Instrument Effects in Language-Model Honesty Evaluation: An Auditable Single-System Demonstration

    Jul 15, 2026Justin BronderLLM EvaluationLLM Honesty

    Corabo Inc.