AI Agent Evaluation

Ask about this topic

Momentum

58 papers in the last four weeks, up 100% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 344

All topics
CardsList

  1. A 3D Characterization Framework for Intelligent Sequential Decision Making

    Oct 8, 2026Sadig Gojayev, Carolina FortunaSequential Decision MakingAI Agent Evaluation

    Department of Communication Systems, Jožef Stefan Institute, SI-1000 Ljubljana, Slovenia

  2. TRACE: Diagnosing Verifier Brittleness in Agentic Evaluation

    Oct 8, 2026Radhika GaonkarLLM Agent EvaluationAI Agent Evaluation

    Prime Intellect

  3. Closed-loop evaluation of LLM agents for embedded software development

    Oct 8, 2026Jorge García-Carrasco, Sergio García-Carrasco, Alejandro Maté +1AI Coding AgentsLLM Agent Evaluation

    Lucentia Research, Department of Software and Computing Systems, University of Alicante, Ctra. de San Vicente del Raspeig s/n, 03690 Sant Vicent del Raspeig, Spain · Universitary Institute of Materials Technology (IUTM), Universitat Politècnica de València, Plaça Ferrándiz i Carbonell, 03801 Alcoi, Spain

  4. SciExam for ENSO: Can AI Agents Build Climate Models?

    Oct 7, 2026Yinling Zhang, Langchen Liu, Dongbin Xiu +4AI for ScienceAI Agent Benchmarks

    The Ohio State University · Yale University · University of California, San Diego +2

  5. Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models

    Oct 7, 2026Tan Yu, Alexander Bukharin, Khushi Bhardwaj +19AI Coding AgentsAI Agent Evaluation

    NVIDIA · University of Minnesota – Twin Cities · University of California, Berkeley

  6. Stale, Misattributed, or Late: Where Personal Memory Fails Before Generation

    Oct 7, 2026Haonan Deng, Park SinchaisriAgent MemoryAgent Memory Management

    University of California, Berkeley · Haas School of Business, University of California, Berkeley

  7. Shared and structured inputs undermine collective random choice by reasoning AI agents

    Oct 7, 2026Takahiro Ezaki, Naoto Imura, Katsuhiro NishinariAI Agent EvaluationSelection Bias

    Research Center for Advanced Science and Technology, The University of Tokyo 4-6-1 Komaba, Meguro-ku, Tokyo 153-8904, Japan · Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan

  8. The AI Evaluation Ecosystem

    Oct 7, 2026Yash Dave, Sang T. Truong, Serena Wang +1AI Agent EvaluationBenchmark Design

    Stanford University · University of British Columbia

  9. RippleCP: Measuring Counterfactual Checkpoint Advantage in Coding Agents

    Oct 6, 2026Mayur Akewar, Ravi RanjanSoftware Engineering AgentsCounterfactual Evaluation

    Florida International University, Miami, USA

  10. Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station

    Oct 6, 2026Wenyu Du, Stephen ChungAutonomous Scientific DiscoveryAI Agent Evaluation

    DualverseAI · University of Hong Kong · University of Cambridge

  11. Agent Plasticity: Measuring Self-Improvement Through Experience

    Oct 6, 2026Harman Singh, Anton Bakhtin, Rulin Shao +8Self-Improving AgentsLLM Agent Self-Improvement

    UC Berkeley · Meta Superintelligence Labs · University of Washington +1

  12. EIO-Agents: The Missing Semantic Layer for AI Agent Evaluation

    Oct 6, 2026Fouad BousetouaneAI Agent AuditingAI Agent Evaluation

    The University of Chicago, USA

  13. Inspect Robots: Evaluating the Capabilities and Safety of Embodied AI

    Oct 5, 2026Christopher Leet, Achu Menon, Sravanthi Machcha +12AI Agent EvaluationRobotic Policy Evaluation

    Robocurve · University of Southern California · San Jose State University +7

  14. MiniCorp: The Last Mile of the AI Agent Firm

    Oct 5, 2026Jingying Zeng, Zhenwei Dai, Jinning Li +6Synthetic Data GenerationAI Agent Evaluation

    Microsoft · University of Illinois Urbana-Champaign · Princeton University +2

  15. AutoSciBench: Autonomous Benchmark Generation for Evaluating Scientific Agents

    Oct 4, 2026Dongki Kim, Namkyeong Lee, Surag Nair +7AI for ScienceAI Agent Evaluation

    Genentech · KAIST

  16. Code Owns the Simulation, Jev Owns the Evaluation

    Oct 1, 2026Yaodong Yang, Hongyao Tang, Yi Ma +4AI Agent EvaluationRobotic Policy Evaluation

    The Chinese University of Hong Kong · Tianjin University · Shanxi University +3

  17. Continuous Process-Level Evaluation for Evolving Enterprise AI Agent Skills

    Oct 1, 2026Ngoc Phuoc An Vo, Aarya Doshi, Vadim SheininTool-Use EvaluationAI Agent Evaluation

    IBM · Georgia Institute of Technology · IBM Research

  18. Agents Are Systems, Not Models: Rethinking Agentic Evaluation

    Oct 1, 2026Luis Wiedmann, Leander Girrbach, Cordelia Schmid +1AI Agent ReliabilityAI Agent Evaluation

    Technical University of Munich, MCML · Helmholtz Munich · Inria, École normale supérieure, CNRS, PSL Research University

  19. Auditing Action Settlement in LLM Agent Environments: Order, Progress, and Replay

    Oct 1, 2026Haotian Chen, Bowen Ye, Yuning Zhang +1Multi-Agent LLM SystemsAI Agent Auditing

    School of Cyber Science and Technology, University of Science and Technology of China · School of Education, Shanghai Jiao Tong University · SWJTU-Leeds Joint School, Southwest Jiaotong University

  20. Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles

    Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference EfficiencyLanguage Model Generation Evaluation

    Los Alamos National Laboratory Los Alamos, New Mexico, USA

  21. Agent Evaluation Reliability: More Tasks Won't (Always) Fix An Agent Leaderboard

    Sep 30, 2026Michael Hardy, Ruhana Azam, Anka Reuel +2AI Agent EvaluationAgent Evaluation

    Stanford University · University of Illinois Urbana-Champaign · Senior Author

  22. CompMat-Bench: Benchmarking AI Agents for Computational Materials Science

    Sep 30, 2026Chenmu Zhang, Levi Felix, Jun-Jie Zhang +6AI Agents for Scientific DiscoveryAI Agent Evaluation

    Department of Materials Science and NanoEngineering Rice University, Houston, TX 77005, USA

  23. No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents

    Sep 30, 2026Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian +1LLM Red TeamingAI Agent Evaluation

    Indiana University Bloomington, IN, USA · Rutgers University Piscataway, NJ, USA · Johns Hopkins University Baltimore, MD, USA

  24. RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models

    Sep 30, 2026Zheng Chen, Linfeng Liu, Hong Li +1Learning to RankLLM Agent Orchestration

    Meta

  25. What Should an Agent Remember? Disentangling Retention from Retrieval in Bounded-Memory Evaluation

    Sep 30, 2026Juli HuangAI Agent EvaluationMemory Agent Benchmarks

    Stanford University

  26. Risk-Aware Adaptive Evaluation: Finding High-Impact Failures Under Limited Budgets

    Sep 30, 2026Priyanath Maji, Spandan Ghose ChowdhuryThompson SamplingAI Agent Evaluation

    Georgia Institute of Technology Atlanta, GA 30332

  27. AgBench: Agentic AI Benchmarks for Personal AI Devices

    Sep 29, 2026Yizhou Han, Di Wu, Dhananjay Saikumar +1AI Agent ReliabilityAI Agent Evaluation

    University of St Andrews, UK · Zhejiang University of Technology, China

  28. Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

    Sep 29, 2026Qiankai XuContinual Learning for LLM AgentsAI Agent Evaluation

    Nanjing University

  29. Can an AI Agent Rediscover a Blaschke-Curve Invariant?

    Sep 29, 2026Yunus E. ZeytuncuAI Agents for Scientific DiscoveryAI Agent Evaluation

    University of Michigan-Dearborn

  30. CRJudgeBench: Can AI Detect Plausible but Invalid Code Reviews?

    Sep 29, 2026Yue Pan, Jiawei Li, Ziyuan Zhang +2LLM EvaluationLLM-as-a-Judge

    University College London · Amazon · Zhejiang University