Process-Based Agent Evaluation

Ask about this topic

Latest papers

CardsList

57 papers34 papers in the last four weeks, up 750% on the four weeks before. 0.2% of all new papers.

  1. DRAFT-Bench: Evaluating Frontier Agents Across the Professional Design Lifecycle

    Nov 6, 2026Rong Pan, Jianqiang WangLLM Agent EvaluationAI Agent Benchmarks

    University of Maryland, College Park · curify-AI

  2. Closed-Loop Agent Evaluation: Layered Measurement and Self-Calibration

    Nov 4, 2026Ruifeng Ren, Gun AhnAI Agent EvaluationAutomated Agent Evaluation

    Massachusetts Institute of Technology, Cambridge, MA, USA

  3. Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents

    Nov 4, 2026Milan Bulat, Gino Ardelean, Ellington KirbyAutonomous Driving Safety EvaluationAutonomous Driving Benchmarks

    Social Chaos Lab

  4. OperAItor: An Agentic Framework for Fault-Tolerant Operation of Chemical Processes

    Nov 3, 2026Javal Vyas, Mehmet Mercang{ö}zLanguage Model-Based ControlProcess-Based Agent Evaluation

    Autonomous Industrial Systems Laboratory, Department of Chemical Engineering, Imperial College London, Imperial College Road, South Kensington, London, SW7 2AZ, United Kingdom

  5. Artifact-to-Capability: Diagnosing Breaks in Capability Formation of Self-Evolving Agents in a Controlled Environment

    Nov 3, 2026Ziqian Ban, Haoxiang Yang, Yanfei Zhang +6Self-Evolving AgentsProcess-Based Agent Evaluation

    1Tianjin University · 2Independent Researcher · 3International Digital Economy Academy (IDEA)

  6. DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects

    Nov 2, 2026Yumou Liu, Mingju Chen, Zhoufutu Wen +21LLM Agent EvaluationAI Agent Benchmarks

    Humanlaya · Theseus Labs · Shanghai Jiao Tong University +4

  7. What Structurally Breaks an Agent? Auditing Difficulty in Stateful Tool-Use Workflows

    Nov 2, 2026Shereen Anand, Ju Wang, Yali DuAgent Failure AnalysisAI Agent Auditing

    PES University, India · University College London, United Kingdom · King’s College London, United Kingdom

  8. Do Agents Still Follow the Rules? Benchmarking Rule Compliance under Agent Context Compaction

    Nov 2, 2026Yiwei Zha, Qingyu Meng, Herbert Bos +1AI Agent Safety BenchmarksLLM Agent Safety

    VUSec, Department of Computer Science, Vrije Universiteit Amsterdam

  9. LUCA: A Synthetic World for Experimental Reasoning

    Nov 2, 2026Anna Gogleva, Thomas Charman, Martin Balla +1Process-Based Agent EvaluationLLM Agent Evaluation

    Relation Therapeutics

  10. CARE: Grounded Recursive Self-Improvement for Production Customer-Service Agents

    Nov 2, 2026Yanbei Jiang, Chao Lei, Rena Gao +17Recursive Self-ImprovementContinual Learning for LLM Agents

    AI and Data Team, Keeta, Meituan · University of Melbourne · Fudan University

  11. From 1D Benchmarks to Complex 3D Geometries: Can Agents Reliably Specify Problems for Automated PDE Simulation?

    Oct 30, 2026Akshat Jani, Sakhinana Sagar Srinivas, Venkataramana RunkanaLLM Agent ReliabilityProcess-Based Agent Evaluation

    Tata Research Development and Design Center, Pune, India 411057

  12. BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents

    Oct 29, 2026Shriram Vasudevan, Videet Mehta, Connor SweeneyLong-Horizon LLM AgentsProcess-Based Agent Evaluation

    Georgia Institute of Technology · Baba · Massachusetts Institute of Technology

  13. ReplayRCA: Measuring Post-hoc Diagnosis and Live Oversight of Agent Failures by Counterfactual Replay

    Oct 29, 2026Jeremy Canale, Fabien SoulisAI Agent EvaluationFailure Attribution for LLM Agents

  14. Does Recursive Knowledge Construction Help Agents Verify Consensus Claims in Science?

    Oct 29, 2026Lun Ai, Yu-Heng Wu, Jonathan Schaul +6Scientific Claim VerificationScientific Agent Benchmarks

    European Bioinformatics Institute (EMBL-EBI), European Molecular Biology Laboratory, Hinxton, UK. · Institute for Computational Biomedicine, Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Heidelberg, Germany. · EMBL Rome, European Molecular Biology Laboratory, Monterotondo, Italy.

  15. Prefix Executability: Evaluating Tool-Using Agents Beyond Final Success

    Oct 29, 2026Amir H. Rezaeian, Weiyi Sun, Yassine Benajiba +1Tool-Use EvaluationTool-Using Agents

    Oracle AI

  16. Bank-World: A Benchmark for Evaluating Agents on Banking Workflows

    Oct 29, 2026Daniel RupawallaAI Agent BenchmarksProcess-Based Agent Evaluation

    Arcanum Labs

  17. Characterizing LLM Agent Behaviors in Module Shadowing Environments

    Oct 29, 2026Shuai Yang, Enmeng Lu, Jinyu Fan +7LLM Agent SecurityAI Agent Auditing

    Gaoling School of AI, Renmin University of China, Beijing, China · Beijing Institute of AI Safety and Governance, Beijing, China · School of Humanities, University of Chinese Academy of Sciences, Beijing, China +1

  18. Repairs for Whom? Evaluator-Executor Mismatch in Selecting Repairs for Failed LLM Agents

    Oct 28, 2026Yijun Liao, Fanwei LiangLLM Agent EvaluationProcess-Based Agent Evaluation

    Independent Researcher · South China University of Technology

  19. FIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agents

    Oct 27, 2026Arther Tian, Alex Ding, Simon Wu +1AI Agent AuditingProcess-Based Agent Evaluation

    DGrid AI

  20. Benchmarking Coding Agents under Evolving Requirements

    Oct 26, 2026Jia Feng, Alyssa Wu, Cuiyun Gao +3Coding Agent BenchmarksCode Generation

    Hunyuan Team, Tencent · The Chinese University of Hong Kong

  21. ShutdownMod: Turning Any Trajectory into a Shutdown Evaluation

    Oct 26, 2026Param Biyani, Sebastien Luyasu, Cedric Potvliege +3AI Agent SafetyAI Agent Evaluation

    MATS · SEAR · University of Washington +2

  22. LongSearchEval: Evaluating Long-Horizon Web Search Capabilities of Large Language Models via a Trajectory-Aware Agent Framework with Failure-Node Localization

    Oct 26, 2026Jialiang Liu, Shusheng Li, Wenjun TanLong-Horizon Agent EvaluationAutomated Agent Evaluation

    College of Information Science and Engineering, Northeastern University, China · Key Laboratory of Intelligent Computing of Medical Images, Ministry of Education, Northeastern University, China. · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China +1

  23. When Agents Fail: Model or Harness--Who's to Blame?

    Oct 23, 2026Shao Zhang, Hangfan Zhang, Yuanyuan Shi +9LLM Agent EvaluationLLM Agent Harnesses

    Shanghai Artificial Intelligence Laboratory

  24. SCI-AGENT-EXAM: Benchmarking Agents on Scientific Research Tasks

    Oct 23, 2026Tianyun Zhong, Wei Wang, Jialin Li +17Scientific Agent BenchmarksLLM Agent Evaluation

    University of Chinese Academy of Sciences · Institute of Software, Chinese Academy of Sciences · AI Data, Alibaba Group +4

  25. DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical Tasks

    Oct 23, 2026Shan Peng, Shuaiqi Cheng, Miao Peng +2LLM Decision-MakingProcess-Based Agent Evaluation

    Xiwei Technology (Guangzhou) Co., Ltd. · The Hong Kong University of Science and Technology (Guangzhou)

  26. Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution

    Oct 23, 2026Zhiyong Cao, Hengjie Zhu, Yongxin Wang +20LLM Agent HarnessesLLM Agent Reliability

    Institute of Automation, CAS · Zhongguancun Academy · Institute of Information Engineering, CAS +2

  27. CRM-Agent-Bench: Benchmarking Tool-Using Agents in Stateful Enterprise Environments

    Oct 23, 2026Monojit Banerjee, Zhujun Lan, Defu CaoAI Agent BenchmarksEnterprise AI Agents

    Salesforce, Inc.

  28. Knowing When to Stop: The Evidence-Action Gap in Coding Agents

    Oct 22, 2026Tianyu Chen, Xinran Zhao, Vijay Viswanathan +3AI Coding AgentsProcess-Based Agent Evaluation

    The University of Texas at Austin · Carnegie Mellon University · The University of North Carolina at Chapel Hill

  29. Diagnosing Personalization Failures in Mobile GUI Agents

    Oct 22, 2026Zhixin Lin, Dongliang Xu, Jungang Li +6GUI AgentsMobile GUI Agents

    Shandong University · VIVO AI Lab · The Hong Kong University of Science and Technology (GZ) +5

  30. What the Trace Shows and What the Label Says: Auditing Behavioral Detectors in Agent Evaluation

    Oct 21, 2026Idunnuoluwa A. Adeniji, Shilpika, Michael E. PapkaLLM Agent EvaluationProcess-Based Agent Evaluation

    Department of Computer Science, University of Illinois Chicago, Chicago, IL, USA · Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, IL, USA