LLM Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

194 papers in the last four weeks, up 87% on the four weeks before. 1.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,631

All topics

LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)
Jun 8, 2026Ankai Hao, Ke Chen, Huan Li +1LLM Evaluation Tabular ML
Zhejiang University
RTL-BenchLS: A Large-Scale Benchmark for RTL Reasoning and Generation with Large Language Models
Jun 8, 2026Jing Wang, Shang Liu, Wenji Fang +3LLM Evaluation Benchmark Design
Hong Kong University of Science and Technology
Can LLMs understand LilyPond? A benchmark for symbolic music generation and understanding
Jun 7, 2026Matteo Spanio, Mohammad Torabi, Andrea Poltronieri +1Music Generation Evaluation LLM Evaluation
Centro di Sonologia Computazionale, University of Padova, Padova, Italy · Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain
From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape
Jun 7, 2026Hao Chen, Ziyu Han, Yukun Yan +3LLM Evaluation Rubric-Based Evaluation
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology · Department of Computer Science and Technology, Institute for AI, Tsinghua University
Multilingual Fact-Checking at Scale: Fine-Tuned Compact Models vs LLMs
Jun 7, 2026Pratuat Amatya, Vinay SettyMultilingual Language Models LLM Evaluation
Factiverse AS and University of Stavanger Stavanger, Norway
Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs
Jun 7, 2026Rahul Gorijavolu, Kaushik Madapati, Pritika Vig +7LLM Evaluation LLM Sycophancy
MIT Critical Data, Massachusetts Institute of Technology, Cambridge, MA, USA · School of Medicine, Johns Hopkins University, Baltimore, MD, USA · Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA +10
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
Jun 7, 2026Antonio Franca, Alexander TongNon-Autoregressive Text Generation LLM Evaluation
AITHYRA Institute, Vienna, Austria.
Impacts of Histories and Models on LLM Grading: A Study in Advanced Software Engineering Courses
Jun 7, 2026Qilin Zhou, Zhuo Wang, Yue Li +1LLM Evaluation LLM-as-a-Judge
City University of Hong Kong, Hong Kong SAR, China
SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models
Jun 6, 2026Ayah Al-Naji, Edoardo Fazzari, Saif Alkindi +3LLM Evaluation Synthetic Benchmark Generation
Mohamed bin Zayed University of Artificial Intelligence, UAE
IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts
Jun 6, 2026Ayana Hussain, Soumya Sharma, Golnoosh Farnadi +3LLM Evaluation Privacy-Preserving Language Models
1Simon Fraser University · 3Mila · 2McGill University +1
GIScholarBench: Benchmarking LLM Overconfidence in GIS Research
Jun 6, 2026Zongrng Li, Mingzheng Yang, Lei Zou +8LLM Evaluation AI-Assisted Scientific Research
Department of Geography Texas A&M University College Station, TX, USA · Google Mountain View, CA, USA · Department of Landscape Architecture and Urban Planning Texas A&M University College Station, TX, USA
Arabic Sentence Segmentation Across Genres and Punctuation Conditions
Jun 6, 2026Mohammed Elkholy, Khalid N. Elmadani, Nizar Habash +1LLM Evaluation Arabic NLP
Mohamed bin Zayed University of Artificial Intelligence · New York University Abu Dhabi
UniQL: Towards Dialect-Universal Benchmarking for Text-to-SQL
Jun 6, 2026Jianling Gao, Chongyang Tao, Jiayuan Bai +7LLM Evaluation Text-to-SQL
1SKLCCSE, Beihang University · 2The University of Hong Kong
Summarization is Not Dead Yet
Jun 6, 2026Dongqi Liu, Chenxi Whitehouse, Zheng Zhao +3LLM Evaluation Text Summarization
Saarland University, Max Planck Institute for Informatics · Tencent YouTu Research · University of Cambridge +2
Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy
Jun 6, 2026Yuan Shen, Xiaojun Wu, Linghua YuLLM Evaluation Language Model Safety Evaluation
College of Computer Science and Technology, Zhejiang University, PR China · The First Hospital of Jiaxing, Zhejiang Province, PR China
The AI Epistemic Deference Index: A Continuous Measure of Sycophancy
Jun 5, 2026Alejandro Botas, Paul de Font-Reaulx, Luke HewittLLM Evaluation LLM Sycophancy
Independent · University of Michigan, Ann Arbor · Transluce
How reliable are LLMs when it comes to playing dice?
Jun 5, 2026Luca Avena, Gianmarco Bet, Bernardo BusoniLLM Evaluation LLM Prompting
Università degli Studi di Firenze Dipartimento di Matematica e Informatica ‘Ulisse Dini’
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification
Jun 5, 2026Sercan Karakaş, Yusuf ŞimşekLLM Evaluation Zero-Shot Text Classification
University of Chicago · Fırat University
Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm that Provably Exploits Model Similarity
Jun 5, 2026Zifan Lyu, Chahine Nejma, Tobias Wegel +2Model Selection Multi-Armed Bandits
1ETH Zurich · 2Centrale Sup´elec · 3ENS de Cachan +1
Sycophantic Praise: Evaluating Excessive Praise in Language Models
Jun 5, 2026Daniel Vennemeyer, Phan Anh Duong, Meryl Ye +2LLM Evaluation LLM Sycophancy
University of Cincinnati · Carnegie Mellon University · Texas A&M University
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Jun 5, 2026Yang Zhang, Xiao Fei, Amr Mohamed +6Multilingual Language Model Evaluation LLM Evaluation
Ecole Polytechnique · MBZUAI · ENS-PSL +1
Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinese
Jun 5, 2026Xing Yue, Yongliang Shen, Weiming LuLLM Evaluation
Zhejiang University
Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models
Jun 5, 2026Dewi Gould, Francis Rhys Ward, Anders Cairns Woodruff +18LLM Evaluation AI Safety Evaluation
Redwood Research · Astra Fellows Program · Aether Research +10
Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modeling
Jun 5, 2026Xing Yue, Linjuan Wu, Daoxin Zhang +2Reward Modeling LLM Evaluation
Zhejiang University · Xiaohongshu Inc.
MADE: Beyond Scoring via a Multilingual Agentic Diagnosing Engine for Fine-Grained Evaluation Insights
Jun 5, 2026Yilun Liu, Miao Zhang, Shimin Tao +9Multilingual Language Model Evaluation LLM Evaluation
Huawei, China
Adversarial Robustness of Activation Steering in Large Language Models
Jun 5, 2026Kien Le, Thai LeLLM Evaluation Language Model Steering
Independent Researcher · Indiana University
OpenHalDet: A Unified Benchmark for Hallucination Detection across Diverse Generation Scenarios
Jun 5, 2026Xinyi Li, Zhen Fang, Yongxin Deng +12LLM Evaluation Hallucination Detection
University of Technology Sydney · University of Bristol · University of Wisconsin–Madison +2
From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing
Jun 5, 2026Guannan Lai, Haoran Hu, Long Chen +2LLM Evaluation LLM Routing
School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University · Hong Kong University of Science and Technology +1
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
Jun 5, 2026Vladislav Smirnov, Chieu Nguyen, Sergey Senichev +14LLM Evaluation Test-Time Scaling
1MBZUAI · 7Independent Researcher · 2ETH Zürich +4
Evidence-Grounded Ensemble Diagnosis of 802.11 Packet Captures: A Multi-Stage Pipeline with Deterministic Reliability Scoring
Jun 5, 2026Jerome Henry, Swadhin Pradhan, Miroslav PopovicLLM Evaluation Evidence-Grounded Reasoning
Cisco Systems

Jun 8, 2026, cs.AIJ/K move · Enter open · S save

LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)

Ankai Hao, Ke Chen, Huan Li, Lidan Shou

Zhejiang University

LLM Evaluation Tabular ML Automated Feature Engineering

Open paper AskPDF arXiv

Feature engineering remains a cornerstone of tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for its automation, giving rise to LLM-powered Automated Tabular Feature Engineering (LATTE). However, the field lacks standardized, cost-aware evaluation platforms, and the combinatorial explosion of design choices obscures true algorithmic progress. To bridge these gaps, we systematically deconstruct 15 representative LATTE methods into a unified 6-dimensional taxonomy. Based on this abstraction, we introduce LATTEArena, a standardized, modular, and extensible benchmarking framework that decouples monolithic pipelines into reusable execution blocks. By distilling the massive combinatorial space, we evaluate 24 core LATTE configurations across 7 research questions. Our head-to-head benchmarking goes beyond predictive accuracy to quantify token efficiency and execution robustness, yielding 17 empirical findings on cost-effectiveness trade-offs. Furthermore, we provide 3 concrete recommendations for optimal real-world deployment. By enabling controlled component-level comparisons, LATTEArena shifts the paradigm from ad-hoc prompt engineering to systematic context management. All code, datasets, and over 4,000 execution logs are publicly available to foster a dynamic, community-driven benchmark. Our framework, leaderboard, and all artifacts are hosted on the LATTEArena project website at https://goodenhak.github.io/LATTEArena.

Momentum

Related topics

Latest papers 1,631