AI Agent Evaluation

Ask about this topic

Momentum

58 papers in the last four weeks, up 100% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 341

All topics
CardsList

  1. An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding

    Jul 30, 2026Yanyu Ren, Yunfeng Bai, Xizheng Wang +2Multi-Agent CoordinationMulti-Agent Orchestration

    1Tsinghua University · 2Zhongguancun Laboratory

  2. Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

    Jul 30, 2026Fouad BousetouaneAI Agent ReliabilityAI Agent Evaluation

    The University of Chicago, USA

  3. Can AI agents conduct open-ended AI research? Early evidence from two case studies

    Jul 29, 2026Peter Kirgis, Sayash Kapoor, Andrew Schwartz +21AI-Assisted Scientific ResearchAI Agent Evaluation

    Princeton University · Cornflower Labs · UK AI Security Institute +8

  4. Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation

    Jul 28, 2026Stefan Krsteski, Charlotte Meyer, Guillaume Allegre +2AI Agent EvaluationAgent Evaluation

    Andromede AI

  5. Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation

    Jul 28, 2026Zheng Tong, Yang Liu, Wanshu Fan +6AI Agent EvaluationAgentic Workflows

    School of Software Engineering, Dalian University, Dalian, China · Department of Surgery, Affiliated Zhongshan Hospital of Dalian University, Dalian, China · The Chinese University of Hong Kong, Shenzhen, China +3

  6. PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

    Jul 28, 2026Korosh Vatanparvar, Ashutosh Joshi, Maria Xenochristou +11AI Agent EvaluationAI Agent Safety

    Amazon Health AI

  7. Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

    Jul 28, 2026Abu Bakar SiddikAI Agent EvaluationCybersecurity

    Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology, Rajshahi 6204, Bangladesh

  8. GAUGE: Grading Agent-Built Financial Models Without a Golden Answer

    Jul 27, 2026Jiacheng Lu, Sinuo Wang, Wentao Zhao +12Quantitative FinanceFinancial Forecasting

    Shanghai Jiao Tong University Shanghai, China · StepFun Shanghai, China · University of Adelaide Adelaide, Australia +5

  9. Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents

    Jul 27, 2026Diandian Guo, Cong Cao, Fangfang Yuan +3AI Agent EvaluationLLM Agent Self-Improvement

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

  10. Stress-testing large language model agents in a robotic chemistry laboratory

    Jul 25, 2026Lulu Guo, Yingkai Sun, Xiaobo Li +13AI Agent ReliabilityAI Agents for Scientific Discovery

    State Key Laboratory of Precision and Intelligent Chemistry, Hefei National Research Center for Physical Sciences at the Microscale, School of Chemistry and Materials Science, University of Science and Technology of China, Hefei, China · Center for Scientific Intelligence Innovation, Hefei, China · School of Chemistry, School of Computer Science, University of Birmingham, Birmingham, UK

  11. Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI

    Jul 24, 2026Jiaqi Shao, Hanck Chen, Wei Zhang +2Computer-Use Agent BenchmarksReward Hacking

    1Hunyuan Team, Tencent · 2The Hong Kong University of Science and Technology · 3Duke Kunshan University

  12. Agentic Evaluation of Copyright Law Compliance

    Jul 23, 2026Zheng Hui, Doni Bloomfield, Noam KoltAI Agent EvaluationAI Agent Benchmarks

    Language Technology Lab, University of Cambridge, Cambridge, United Kingdom · Law School, Fordham University, New York, United States · Faculty of Law and School of Computer Science and Engineering, Hebrew University, Jerusalem, Israel

  13. pAI-Econ-claude: A Gated Human-in-the-Loop Multi-Agent Architecture for AI-Assisted Economic Theory Development

    Jul 23, 2026Chen Zhu, Xiaolu Wang, Weilong ZhangAI-Assisted Scientific ResearchAI Agent Evaluation

    China Agricultural University · University of Cambridge

  14. Frontier Financial Judgement: Can agents tell what might move a stock?

    Jul 22, 2026Joshua HarrisAI Agent EvaluationAI Agent Benchmarks

    Sonto

  15. BioSecBench-Surveillance: A Verifiable Benchmark for AI Agents in Pathogen Genomic Surveillance

    Jul 21, 2026Harmon Bhasin, Kevin Flyangolts, Dianzhuo Wang +9AI Agent EvaluationAI Agent Benchmarks

    LatchBio, San Francisco, CA · Aclid, New York, NY

  16. Do AI-Native Biotechs Need Departments? Benchmarking Company World Models for AI-Driven Drug Development

    Jul 21, 2026Yinan WangWorld ModelsAI Agent Evaluation

    Noah AI Research

  17. The Autonomous Agency Scale: A Behavioral Framework for Measuring Self-Directed Behavior in AI Systems

    Jul 20, 2026Samuel PresgravesAI Agent EvaluationAI Agent Benchmarks

    Independent researcher.

  18. A Diagnostic Framework for AI Agent Behavior

    Jul 19, 2026Xichen Zhang, Yingjie Zhang, Tianshu SunAI Agent EvaluationAI Agent Governance

    Sobey School of Business, Saint Mary’s University, Canada. · Guanghua School of Management, Peking University, China. · Cheung Kong Graduate School of Business, China.

  19. Teach it to stop, not just to click

    Jul 19, 2026Barada Sahu, Shivesh PandeyAI Agent ReliabilityAgent Reliability

    Cabal AI · Para AI

  20. OTAP: Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories

    Jul 19, 2026Babak Barazandeh, Subhabrata Majumdar, George MichailidisLLM Agent EvaluationAI Agent Evaluation

    Fortinet · IIMB · UCLA

  21. EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding

    Jul 19, 2026Yaohan Yang, Minglei Shi, Borui Zhang +2Computer-Use Agent BenchmarksGUI Agents

    Department of Automation, Tsinghua University

  22. Real-World Evaluation of an AI Agent Drafting Translational Impact Summaries

    Jul 18, 2026Mohammad Arvan, Amber E. Osterholt, Bailee Rue +5AI Agent EvaluationHuman-in-the-Loop AI

    University of Illinois Chicago, Chicago, IL, USA

  23. OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios

    Jul 16, 2026Chengyu Shen, Yujie Fu, Gangtao Xin +13Computer-Use Agent BenchmarksAI Agent Evaluation

    Huawei Cloud Post-Training Team

  24. AI Agents Do Not Fail Alone:The Context Fails First

    Jul 15, 2026Fouad BousetouaneAI Agent ReliabilityAgent Reliability

    The University of Chicago, USA

  25. DeepStress: Stress-Testing Deep Search Agents

    Jul 15, 2026Ismael Rousseau, Geraldine Damnati, Frederic BechetAI Agent ReliabilityAI Agent Evaluation

    Orange Research, Lannion, France · Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France

  26. Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors

    Jul 15, 2026Michael O. EnioladeAI Agent AuditingAI Agent Evaluation

    University of the Cumberlands

  27. Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing

    Jul 13, 2026Ananda Dhakal, Krish Neupane, Aarjan ChaudharyAI Agent EvaluationAI Agent Security Benchmarks

    Kroda Labs

  28. Quantum Circuit Vision: Cost-Aware Evaluation of Visual AI Agents for Quantum Code Generation

    Jul 11, 2026Dongping Liu, Aoyu Zhang, Luyao ZhangAI Agent EvaluationAI Agent Benchmarks

    Amazon Web Services Hong Kong, China · Amazon Web Services Beijing, China · Digital Innovation Research Center and Social Science Division, Duke Kunshan University Address: Duke Avenue No.8, Kunshan, Suzhou, Jiangsu, China, 215316

  29. SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

    Jul 9, 2026Shilin Ou, Yifan Xu, Luyao ZhangAlgorithmic AuditingAI Agent Evaluation

    Duke Kunshan University Kunshan, Jiangsu, China · Digital Innovation Research Center and Social Science Division, Duke Kunshan University, Address: Duke Avenue No.8, Kunshan, Suzhou, Jiangsu, China, 215316

  30. Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction

    Jul 9, 2026Sophia Koehler, Antonia Wüst, Inga Ibs +5AI Agent EvaluationActive Learning

    1AIML Lab, TU Darmstadt · 2Hessian Center for AI (hessian.AI) · Psychology of Information Processing, TU Darmstadt +4