LLM Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,631

All topics

Fabricator or dynamic translator?
Apr 16, 2026Lisa Vasileva, Karin SimLLM Evaluation Hallucination Detection
Language Weaver
Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options
Apr 16, 2026Nahyun Lee, Guijin SonLLM Evaluation Multiple-Choice Question Answering
Chung-Ang University · Seoul National University
Prompt-Driven Code Summarization: A Systematic Literature Review
Apr 16, 2026Afia Farjana, Zaiyu Cheng, Antonio MastropaoloLLM Evaluation LLM Prompting
William & Mary, USA
PeerPrism: Peer Evaluation Expertise vs Review-writing AI
Apr 16, 2026Soroush Sadeghian, Alireza Daqiq, Radin Cheraghi +3LLM Evaluation AI-Generated Text Detection
Reviewerly Toronto, Ontario, Canada · University of Toronto, Reviewerly Toronto, Ontario, Canada · UC Berkeley, Reviewerly Berkeley, California, United States
Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces
Apr 13, 2026Manas Pathak, Xingyao Chen, Shuozhe Li +2LLM Evaluation Language Model Generation Evaluation
University of Texas at Austin
A Robust Evaluation of Probe Robustness: Lessons for Reliable OOD Uncertainty Quantification
Apr 13, 2026Joe Stacey, Hadas Orgad, Kentaro Inui +2LLM Evaluation Uncertainty Quantification
University of Sheffield · Kempner Institute at Harvard University · MBZUAI +2
Handle with CARE: Can LLMs Reproduce How Online Communities React?
Apr 12, 2026Nuan Wen, Chanbin Lim, Xuezhe MaLarge Language Model-Based Social Simulation LLM Evaluation
Information Sciences Institute University of Southern California
What Does a Sharing Question Add? Auditing LLM Survey Scores for Misinformation
Apr 8, 2026Zonghuan Xu, Xiang Zheng, Yutao Wu +1LLM Evaluation LLM Auditing
Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · City University of Hong Kong, Hong Kong SAR, China · Deakin University, Australia
LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency
Apr 7, 2026Jiachun Li, David Simchi-Levi, Will Wei SunSemiparametric Inference LLM Evaluation
Massachusetts Institute of Technology · Purdue University
LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
Apr 2, 2026Linyang He, Qiyao Yu, Hanze Dong +5Mathematical Reasoning Benchmarks LLM Evaluation
Columbia University · Microsoft Research · University of Amsterdam
Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation
Apr 1, 2026Hexuan Wang, Jingyu Zhang, Benjamin Van Durme +1LLM Evaluation LLM Grounding
Department of Computer Science, Johns Hopkins University
Measuring (some aspects of) the metacognition of AI
Mar 31, 2026Richard Servajean, Philippe ServajeanLLM Evaluation LLM Uncertainty Estimation
Center for Brain Science, RIKEN, Saitama, Japan. · National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan. · Department of Psychology, Paul-Val´ery University, Montpellier, France.
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
Mar 30, 2026Han Wang, Yifan Sun, Brian Ko +8CoT Faithfulness LLM Evaluation
University of Illinois Urbana-Champaign · University of Washington · University of California San Diego
Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP
Mar 30, 2026Urja Khurana, Michiel van der Meer, Enrico Liscio +2LLM Evaluation
Department of Intelligent Systems, Delft University of Technology · Leiden Institute of Advanced Computer Science (LIACS), Leiden University · Computational Linguistics and Text Mining Lab, Vrije Universiteit Amsterdam
Learning to Predict Future-Aligned Research Proposals with Language Models
Mar 28, 2026Heng Wang, Pengcheng Jiang, Jiashuo Sun +4LLM Evaluation Research Idea Generation
University of Illinois Urbana-Champaign
When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models
Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoLLM Inference Efficiency LLM Evaluation
Huawei Switzerland · ACS Lab, Huawei Technologies
In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts
Mar 26, 2026Matthias Busch, Marius Tacke, Sviatlana V. Lamaka +4LLM Evaluation Knowledge Conflicts in Language Models
Institute for Continuum and Material Mechanics, Technical University of Hamburg, Eißendorfer Straße, 21073 Hamburg, Germany · Institute of Material Systems Modeling, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 21502 Geesthacht, GermanyJun · Institute of Surface Science, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 215022026 Geesthacht, Germany +2
JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees
Mar 24, 2026Yuhui Wang, Zhixiong Yang, Ming Zhang +10LLM Evaluation Multi-Turn Dialogue Evaluation
1Fudan University
PROMPT2BOX: Uncovering Entailment Structure among LLM Prompts
Mar 22, 2026Neeladri Bhuiya, Shib Sankar Dasgupta, Andrew McCallum +1LLM Evaluation Representation Learning
CICS, University of Massachusetts Amherst, USA · Amazon AWS AI
Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable
Mar 20, 2026Rounak Saha, Gurusha Juneja, Dayita Chaudhuri +3LLM Evaluation AI-Generated Text Detection
♠Indian Institute of Science · ♢University of California, Santa Barbara · ♡Carnegie Mellon University
When Demonstrations Fail: Diagnosing the Limits of In-Context Learning in Large Audio-Language Models with Progressive Cue Removal
Mar 20, 2026Yen-Ting Piao, Jay Chiehen Liao, Wei-Tang Chien +5Audio-Language Model Evaluation LLM Evaluation
National Taiwan University, Taiwan
ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs
Mar 19, 2026Wanjia Zhao, Ludwig Schmidt, Yejin Choi +3LLM Evaluation AI Agent Benchmarks
Stanford University · Microsoft Research
Toward Measuring Structural Drift in LLM Communication Loops
Mar 18, 2026Wael Hafez, Amir Nazeri, Chenan WeiLLM Evaluation
Semarx Research LLC Alexandria, VA, USA
Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models
Mar 17, 2026Mohamed Adel, Bashar Alhafni, Nizar HabashLLM Evaluation Arabic NLP
Computational Approaches to Modeling Language Lab 1New York University Abu Dhabi · 2Mohamed bin Zayed University of Artificial Intelligence
SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era
Mar 17, 2026Han Jang, Junhyeok Lee, Kyu Sung ChoiLLM Evaluation Scientific Communication
Seoul National University · Seoul National University Hospital · Seoul National University College of Medicine
Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessions
Mar 12, 2026Tae-Eun SongLLM Self-Correction LLM Evaluation
Daejeon Jungang Cheonggua Co., Ltd.
Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions
Mar 4, 2026Qianyun Guo, Yibo Li, Yue Liu +1LLM Evaluation LLM Personalization
National University of Singapore, Singapore.
CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documents
Mar 4, 2026Martin Kostelník, Michal Hradiš, Martin DočekalDocument Understanding Historical Document Analysis
Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic
Verify to Amplify: Improving Reasoning via Learned Chain-of-Thought Verification
Mar 3, 2026Maria-Florina Balcan, Avrim Blum, Kiriaki Fragkia +2LLM Evaluation LLM Answer Verification
Carnegie Mellon University · Toyota Technological Institute at Chicago · Northwestern University
A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities
Mar 3, 2026Faiz Ghifari Haznitrama, Faeyza Rishad Ardi, Alice OhLLM Evaluation
School of Computing, KAIST, Daejeon, South Korea

Apr 16, 2026, cs.CLJ/K move · Enter open · S save

Fabricator or dynamic translator?

Lisa Vasileva, Karin Sim

Language Weaver

LLM Evaluation Hallucination Detection MT Evaluation

Open paper AskPDF arXiv

LLMs are proving to be adept at machine translation although due to their generative nature they may at times overgenerate in various ways. These overgenerations are different from the neurobabble seen in NMT and range from LLM self-explanations, to risky confabulations, to appropriate explanations, where the LLM is able to act as a human translator would, enabling greater comprehension for the target audience. Detecting and determining the exact nature of the overgenerations is a challenging task. We detail different strategies we have explored for our work in a commercial setting, and present our results.

Momentum

Related topics

Latest papers 1,631