Agentic Benchmarks

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

13 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Period ending 2026-09-14

13 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Period ending 2026-09-07

17 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Inside this field

Focused directions

414 papers

Latest in Agentic Benchmarks

Browse all topics →
CardsList

  1. OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents

    Aug 9, 2026Andrea Caciolai, Pere-Lluís Huguet Cabot, Chierh Cheng +11Multilingual AgentsMultilingual

  2. Compiling and Benchmarking Task-State Horizons for Embodied Agents

    Aug 8, 2026Meiqi Wang, Shichao LiAgentic BenchmarksEmbodied Agents

  3. Reason Wide, Not Deep: Amortizing the Reasoning Premium into Distilled Skills

    Aug 8, 2026Agamdeep Singh, Srishti Gautam, Priyanshu Gupta +3Agentic BenchmarksNatural Language Queries

  4. TEPA: Revoking Stale Memories for Conflict-Robust Language Agents

    Aug 7, 2026Yan Zhou, Yue Ouyang, Kaiyang Zheng +1Agent MemoryGradient Staleness

  5. A2EA^2E : An End-to-End Agent Auditing Engine

    Aug 7, 2026Haoning Wang, Mingxun Zhang, Chenyue Yu +4Agent HarnessAgentic Evaluations

  6. SkillEval: Decomposing Agent Skill Quality into Interpretable Signals

    Aug 7, 2026Jiahui Han, Qinuo Li, Ziheng Peng +6Agent Skill EcosystemsSkillaudit

  7. AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games

    Aug 6, 2026Boning Li, Yu Chen, Longbo HuangOptimal StoppingAgentic Evaluations

  8. CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks

    Aug 6, 2026Fanzhe Meng, Guoxin Chen, Jiale Zhao +6Task SynthesisAgentic Benchmarks

  9. Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents

    Aug 6, 2026Noam Koren, Roy Bar-Haim, Abigail GoldsteenAgentic BenchmarksConversational Benchmarks

  10. OpenPM: Auditable Point-in-Time Evaluation for LLM Portfolio-Management Agents

    Aug 6, 2026Xinying Cai, Minghao Guo, Jiahe Liu +7Portfolio OptimizationAgentic Evaluations

  11. Predicting Task Difficulty Without Rollouts

    Aug 6, 2026Stefan Krsteski, Charlotte MeyerAgentic BenchmarksEvaluation Metrics

  12. SkillZip: Contract-Preserving Graph Compression for Scalable Agent Skill Libraries

    Aug 6, 2026Xingyu Tan, Xiaoyang Wang, Qing Liu +4Reusable Agent SkillsAgentic Benchmarks

  13. EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents

    Aug 6, 2026Jie Wu, Ming Gong, Feixiang Cheng +1Opti-Agent-BenchLarge Language Model Agents

  14. Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks

    Aug 5, 2026Nathan S Johnson, Ian AbshireAgentic BenchmarksMicroscopy

  15. Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports

    Aug 5, 2026Haobin Li, Ping Deng, Weizhong Qian +4BugCoding Agents

  16. What Is a Skill Worth? Structure-Aware Shapley Valuation of Agent Skills

    Aug 5, 2026Tao Li, Junfeng Liu, Qinghua Zhao +5Agentic EvaluationsValue

  17. DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces

    Aug 4, 2026Boyan Li, Zhuowen Liang, Yupeng Xie +11Agentic BenchmarksData Lakes

  18. GISAgentBench: A Practitioner-Sourced Benchmark for Evaluating LLM Agents on GIS Tasks

    Aug 3, 2026Abhinav Pothuri, Zhe Jiang, Zelin Xu +1GeospatialOpti-Agent-Bench

  19. PATH-Bench: Path-Dependent Evaluation of Lifelong Agents

    Aug 2, 2026Xidong Yang, Xingyi Zhang, Wenhao Li +7Long-Horizon AgentsAgentic Benchmarks

  20. What Could the Agent See at 19:05? Generating Temporal Enterprise Scenarios from Real Research and Replaying Them to Evaluate Agents

    Aug 2, 2026Tezan Sahu, Himani AroraAgentic EvaluationsArtificial Intelligence Agents

  21. AgentSLABench: Evaluating and Benchmarking Agentic Systems Under Resource Constraints

    Aug 1, 2026Meher Bhaskar Madiraju, Meher Sai Preetam MadirajuAgentic BenchmarksAgentic Evaluations

  22. OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

    Aug 1, 2026Yunhao Chen, Xin Wang, Yixu Wang +6Agentic EvaluationsRed-Teaming

  23. DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable

    Aug 1, 2026Pu Cao, Qingye Kong, Xuedan Yin +5Image EditingWorkflow Generation

  24. CurveShift: Is Agent Progress Scalar? Separating Level from Shape

    Jul 31, 2026Hanwen Xing, Pengyun Wang, BingXu Meng +8Agentic BenchmarksHard

  25. AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

    Jul 31, 2026Dong Yan, Jian Liang, Dapeng Hu +4Self-Evolving AgentsLarge Language Model Agents

  26. MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows

    Jul 31, 2026Nithishwer Mouroug Anand, Wei-Tse Hsu, Kyle Vaccaro +6Coding AgentsMolecular Dynamics

  27. Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation

    Jul 31, 2026Goutham Ramakrishnan, Megha SharmaAgentic BenchmarksSmall Language Models

  28. Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark

    Jul 31, 2026Natan Vidra, Alina Kapanova, Arun Kanhai +1One-Shot RoutingAgentic Benchmarks

  29. Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

    Jul 30, 2026Fouad BousetouaneAgentic EvaluationsArtificial Intelligence Agents

  30. Evaluating Agentic Bioinformatics through Function, Evidence, and Validation

    Jul 30, 2026Phuc Pham, Truong-Son HyScientific WorkflowsAgentic Workflows

  31. Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks

    Jul 29, 2026Jeff Mohl, Nelson Gardner-Challis, Magda Dubois +6Algorithm AuditingAgentic Benchmarks

  32. SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements

    Jul 29, 2026Pengyu Xue, He Yang Yuan, Xin Wang +6Coding AgentsAgentic Benchmarks

  33. Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data

    Jul 29, 2026Lingyang Zeng, Guangze Chen, Kaichen Yu +9Agentic BenchmarksAgent Memory

  34. Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification

    Jul 28, 2026Chenrui Shi, Yuwei Wu, Yang Liu +5Graphical User Interface AgentsGraphical User Interface

  35. RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement

    Jul 28, 2026Fanqing Meng, Lingxiao Du, Qiguang Chen +4Agentic BenchmarksRecursive Self-Improvement

  36. HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following

    Jul 28, 2026Liudas Panavas, Sebastian Minus, Bradley Monton +4Agentic BenchmarksAgentic Tool Use

  37. Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

    Jul 28, 2026Abu Bakar SiddikCybersecurityArtificial Intelligence Safety

  38. When Do Agent Loops Mistake Stagnation for Progress? Self-Evaluation Bias and Externally Grounded Verification in Long-Running Autonomous LLM Agent Loops

    Jul 27, 2026Hyundoo Park, Byungho ChoiAgent LoopAgentic Evaluations

  39. Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents

    Jul 27, 2026Diandian Guo, Cong Cao, Fangfang Yuan +3VerifierAgentic Evaluations

  40. Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents

    Jul 27, 2026Bowen Qin, Yi XieRepositoryCoding Agents

  41. Scaling GUI Agents with Visual State Transitions

    Jul 27, 2026Xiangyan Liu, Kaixin Li, Haonan Wang +6Graphical User Interface AgentsGraphical User Interface

  42. Success Is Not Self-Explanatory: Auditing Success Provenance in Agent Evaluation

    Jul 27, 2026Jingkun Luo, Da-Tian PengAgentic EvaluationsContent Provenance

  43. TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI

    Jul 24, 2026Ritik Raj, Souvik Kundu, Sarbartha Banerjee +3Large Language Model RoutingAdaptive Routing

  44. Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI

    Jul 24, 2026Jiaqi Shao, Hanck Chen, Wei Zhang +2Agentic BenchmarksExploitation

  45. Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model

    Jul 24, 2026Nanbeige Lab, :, Chen Yang +23Agentic ReasoningAgentic Benchmarks

  46. Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents

    Jul 24, 2026Suman Navaratnarajah, Taehyoung Kim, Jona Ruthardt +5Embodied AgentsAgentic Benchmarks

  47. Agentic Evaluation of Copyright Law Compliance

    Jul 23, 2026Zheng Hui, Doni Bloomfield, Noam KoltRightsLarge Language Model Agents

  48. GuardianAgentBench: Where Agents Fail and How to Guard Them

    Jul 23, 2026Vishal Ishwar Naik, Chenyu Xu, Donna Dong +5Agentic BenchmarksLarge Language Model Agents

  49. ArbiGraph: Arbitrarily Scalable Verifiable Task Graphs for Evaluating Context Management

    Jul 22, 2026Pavel Golikov, Evgenii Opryshko, Gennady Pekhimenko +1Reasoning GraphsAgentic Benchmarks

  50. Frontier Financial Judgement: Can agents tell what might move a stock?

    Jul 22, 2026Joshua HarrisFinance BenchmarksMarket

  51. AutoWorldModel-Bench: A State-Centric Benchmark for Automated World-Model Research

    Jul 20, 2026Marjan Moodi, Xuankang Zhu, Fernando De Mesentier Silva +2World ModelsAgentic Benchmarks

  52. Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventions

    Jul 20, 2026Pin Qian, Su Wang, Yihang Chen +5Agentic BenchmarksAgent Memory

  53. EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding

    Jul 19, 2026Yaohan Yang, Minglei Shi, Borui Zhang +2Agentic Benchmarks

  54. OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios

    Jul 16, 2026Chengyu Shen, Yujie Fu, Gangtao Xin +13Agentic BenchmarksArtificial Intelligence Agents

  55. Reward-Free Evolving Agents via Pairwise Validator

    Jul 15, 2026Minghao Liu, Yu Wang, Jiayun Wang +1Self-Evolving AgentsAgentic Evaluations

  56. Copy-on-Write Scoring: Application-Specific Agent Evaluations

    Jul 15, 2026Joanna Roy, Sven HoelzelAgentic EvaluationsScoring

  57. AI Agents Do Not Fail Alone:The Context Fails First

    Jul 15, 2026Fouad BousetouaneAgentic EvaluationsArtificial Intelligence Agents

  58. AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities

    Jul 15, 2026Kai Chen, Zichen Ding, Jiaye Ge +20Agentic EvaluationsLlm-Based Agent

  59. Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents

    Jul 15, 2026Jiwen Zhou, Xiang Liu, Mingming Li +5Recommender SystemsSequential Recommendation

  60. Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors

    Jul 15, 2026Michael O. EnioladeArtificial Intelligence SafetySecurity Analysis