LLM Agent Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

115 papers in the last four weeks, up 140% on the four weeks before. 1.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 793

All topics

Loop-Back Authority in LLM Agent Teams: A Paired Experiment on Flat and Hierarchical Coordination
Sep 13, 2026Burak Agachan, Max van Duijn, Amirhossein ZohrehvandMulti-Agent LLM Systems Multi-Agent Coordination
Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands
Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
Sep 13, 2026Arham Sethi, Arsen Kenzhebayev, Saanvi Paturi +3Tool-Augmented Language Model Agents Deception in Language Models
The Shishukunj International School · Haileybury Astana · UWCSEA East Campus +2
DynSTEER: Dynamic Stage-wise Trajectory Evaluation and Execution-time Review for Agents
Sep 13, 2026Zhichao Shi, Xuhui Jiang, Wenjie Zhang +5LLM Agent Evaluation AI Agent Evaluation
School of Advanced Interdisciplinary Sciences, UCAS · State Key Lab of AI Safety, Institute of Computing Technology, CAS · DataArc Tech Ltd. +3
Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training
Sep 12, 2026Luming Yang, Haoxian Liu, Siqing Li +4Educational Technology LLM Agent Evaluation
Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, China +3
BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Sep 12, 2026Shenghan Zheng, Zonglin Di, Yimin Liu +19Reward Hacking LLM Agent Evaluation
Dartmouth College · Independent · Ohio State University +10
How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding
Sep 12, 2026Jeongyeon Kim, John MitchellMulti-Agent LLM Systems LLM Agent Evaluation
Stanford University, Stanford, CA, USA
K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
Sep 11, 2026Guangsheng Yu, Yanna Jiang, Qin Wang +2LLM Agent Evaluation Privacy Leakage in Language Models
University of Technology Sydney, Australia · CSIRO, Australia
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Sep 11, 2026Jiacheng Sang, Mengyuan Li, Sanxing Chen +3Agentic Search LLM Interpretability
Duke University · University of Pennsylvania
ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation
Sep 11, 2026Zesheng Wei, Mengfan Li, Wenhao Liu +3Automated Negotiation LLM Agent Evaluation
University of Science and Technology of China · Singapore Management University · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
From Document Silos to Process Intelligence: A Multi-Layer Knowledge Graph for CMC Process Development
Sep 10, 2026Reza Amirmoshiri, Faryad Sahneh, Yasser JangjouKG Construction LLM Agent Evaluation
CMC Synthetics, Sanofi US · Accelerator Data science GBU, Sanofi US
ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance
Sep 10, 2026Praphul Singh, Shanu Kumar, Akshat Agarwal +1LLM Agent Evaluation Instruction Following
Oracle Health AI · MBZUAI
The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents
Sep 9, 2026Benjamin Gruenbaum, Doron Porat, Assaf Natanzon +3Tool-Use Evaluation LLM Agent Evaluation
Eon
LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents
Sep 9, 2026Yujin Zhou, Mingxuan Zheng, Chuxue Cao +4Hallucination in Language Models LLM Agent Evaluation
Hong Kong University of Science and Technology
A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation
Sep 8, 2026Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu +6LLM Agent Evaluation Multi-Turn Dialogue Evaluation
PayPal AI
The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?
Sep 8, 2026Boyang Wang, Yunhan Wang, Yalun WuLLM Agent Evaluation LLM Agent Reliability
Independent Researcher · Beihang University · NExT++ Lab, School of Computing, National University of Singapore
Revoked but Still Authoritative: An Empirical Study of Revocation Enforcement in Agent-Memory Systems
Sep 8, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Agent Evaluation Runtime Enforcement for AI Agents
Vulcan Research, AIFT Singapore
Agentic ML Exploration (A-MLE) for Ads Ranking
Sep 8, 2026Erwin Gao, Vinodh Kumar Sunkara, Jingyi Guan +36Learning to Rank LLM Agent Evaluation
Meta Platforms, Inc. Menlo Park and New York, USA
SchemeArena: Factorized Stress Testing of Scheming in LLM Agents
Sep 8, 2026Jie Ruan, Inderjeet Nair, Amy Liu +3LLM Agent Evaluation AI Agent Monitoring
Computer Science and Engineering University of Michigan Ann Arbor, MI 48109, USA
VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities
Sep 7, 2026Jiahao Shi, Edward Tsien, Yifeng Di +10LLM Agent Evaluation Software Security
Department of Computer Science, Purdue University · Red Hat
What Does Multi-Agent Debate Actually Change?
Sep 7, 2026Chen QianLLM Agent Evaluation Multi-Agent Debate
What Does an LLM-Agent Leaderboard Rank Actually Compare?
Sep 7, 2026Wei-Jung HuangLLM Agent Evaluation AI Agent Benchmarks
Independent Researcher United States
xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Sep 7, 2026Yongchang Peng, Qingshui Gu, Liya Zhu +31Pragmatic Reasoning in Language Models LLM Agent Evaluation
ByteDance Seed · M-A-P
SkillAlign: Aligning Skill Interfaces for LLM-based Agents
Sep 7, 2026Shuo Ren, Xiaomian Kang, Jiajun ZhangLLM Agent Evaluation LLM Agent Skill Learning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Wuhan AI Research
$\tau^\tau$ -Bench: An Environment for End-To-End, Realistic Agent Construction
Sep 7, 2026Quan Shi, Keshav Dhandhania, Karthik Narasimhan +1AI Coding Agents LLM Agent Evaluation
Sierra · Princeton University
ERPBench: Evaluating LLM Agents for Enterprise Decision-Making Across Competitive Market Ecologies
Sep 7, 2026Xinran Zhang, Pengrui Lu, Lyumanshan Ye +1AI-Assisted Decision Making LLM Agent Evaluation
1Shanghai Innovation Institute · 2Beijing Institute of Technology · 4GAIR Lab +1
Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning
Sep 5, 2026Zhongan Bi, Qiwen Wang, Jianrong Jiang +11LLM Agent Evaluation Web Search Agents
Zhejiang University · Ant Group
Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Sep 4, 2026Sihan Ge, Yichen Lin, Chenyu Zhou +3Large Language Model-Guided Optimization LLM Agent Evaluation
Cardinal Operations · Shanghai Jiao Tong University, Shanghai, China
SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents
Sep 3, 2026Xin He, Yanlin Wang, Mingwei Liu +3Software Engineering Agents Automated Program Repair
School of Software Engineering, Sun Yat-sen University · College of Computer Science and Technology, Zhejiang University · School of Big Data and Software Engineering, Chongqing University +1
Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting
Sep 3, 2026Muneeb Khan, Frederic Kirstein, Terry Ruas +1LLM Agent Evaluation Conversational Agents
University of Göttingen, Germany
Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation
Sep 3, 2026Yan Tang, Tingyu Cao, Yuanbo Tang +2LLM Agent Evaluation Proactive Assistance
UniCone Team

Sep 13, 2026, cs.MAJ/K move · Enter open · S save

Loop-Back Authority in LLM Agent Teams: A Paired Experiment on Flat and Hierarchical Coordination

Burak Agachan, Max van Duijn, Amirhossein Zohrehvand

Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands

Multi-Agent LLM Systems Multi-Agent Coordination LLM Agent Evaluation

Open paper AskPDF arXiv

Does authority in AI teams improve the outcome? Organizational theory asserts that authority facilitates decision making, improving quality. Meanwhile, some nascent AI research suggests that revision under authority makes LLM output worse. Multi-agent LLM frameworks default to giving a Manager agent the authority to send a worker's output back for revision. Prior comparisons test the effect of authority using verifiable tasks. We conduct an experiment on an open-ended task, business-intelligence reporting, using a sample of 43 paired laptop products and 86 runs. Each report is written once by a hierarchical team and once by a flat team. We find that flat teams produce higher-quality reports, scoring higher on Utility (d = 0.42, p = 0.009) and Writing Clarity (d = 0.34, p = 0.030). The reports are the same length, but hierarchical team reports use 53% more hedging words such as "may" and "could", and each revision is associated with a 0.14-point drop in Writing Clarity on a 1 to 5 scale. Before any revision, the hierarchical team's first draft is indistinguishable from the flat team's report. In other words, the quality gap can be traced to revision. Authority improves quality when the Manager can verify the work, else when it can only provide feedback it has a negative effect on quality.

Momentum

Related topics

Latest papers 793