Agentic Benchmarks

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

13 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Period ending 2026-09-14

13 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Period ending 2026-09-07

17 new papers

A weekly snapshot of new work published in Agentic Benchmarks.

Inside this field

Focused directions

414 papers

Latest in Agentic Benchmarks

Browse all topics →
CardsList

  1. Trip+: Benchmarking Agents in Personalized Interactive Travel Planning

    Jun 19, 2026Junle Chen, Wei Chen, Yehong Xu +6TourLarge Language Model Planning

  2. AgentMeter: Evaluating Model-CLI Matching for CLI-Based Local Task-Solving Agents

    Jun 19, 2026Han Chi, Jiaxin Qi, Yan Cui +2Agentic Benchmarks

  3. Power Systems Agent Benchmark: Executable Evaluation of AI Agents in Electric Power Engineering

    Jun 18, 2026Sergei TrashchenkovPower SystemsAgentic Benchmarks

  4. Contagion Networks: Evaluator Preference Propagation in Multi-Agent LLM Systems

    Jun 18, 2026Zewen LiuMulti-Agent Large Language Model SystemsAgentic Evaluations

  5. ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments

    Jun 18, 2026Tingyue Pan, Mingyue Cheng, Daoyu Wang +4Information Retrieval MetricsAcademic Papers

  6. Measuring Biological Capabilities and Risks of AI Agents

    Jun 18, 2026Patricia Paskov, Jeffrey Lee, Kyle Brady +1Artificial Intelligence SafetyAgentic Evaluations

  7. ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?

    Jun 18, 2026Jiajun Li, Mingshu Cai, Yixuan Li +5Opti-Agent-BenchMulti-Agent Orchestration Frameworks

  8. Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents

    Jun 18, 2026Dhaval C. Patel, Kaoutar El Maghraoui, Shuxin Lin +58Agentic EvaluationsAgentic Benchmarks

  9. IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows

    Jun 17, 2026Ahmad Salimi, Wentao Ma, Yuzhi Tang +3Voice AgentsAgentic Benchmarks

  10. RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models

    Jun 17, 2026San Kim, Daechul Ahn, Reokyoung Kim +3Agentic BenchmarksReasoning Benchmark

  11. SWE-Future: Forecast-Conditioned Data Synthesis for Future-Oriented Software Engineering Agents

    Jun 17, 2026Qiao Zhao, JianYing Qu, Jun Zhang +3Task SynthesisCoding Agents

  12. Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation

    Jun 17, 2026Seyed Alireza Azimi, Homayoon Farrahi, Abhishek Naik +2Action SpaceManipulation

  13. Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Models

    Jun 16, 2026Jasmine Brazilek, Joel Christoph, Maheep Chaudhary +4Agentic BenchmarksTour

  14. A Framework for Evaluating Agentic Skills at Scale

    Jun 16, 2026Maksim Shaposhnikov, Nicolas Fortuin, Simon Stipcich +3Agentic EvaluationsLarge Language Model Agents

  15. EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent

    Jun 16, 2026Zeyao Du, Tong Li, Haibo ZhangAgentic CommerceOpti-Agent-Bench

  16. Dissecting model behavior through agent trajectories

    Jun 16, 2026Gaurav Gupta, Vatshank Chaturvedi, Jun Huan +1Agentic BenchmarksProblem Solving Capability

  17. Human-on-the-Bridge: Scalable Evaluation for AI Agents

    Jun 15, 2026Fouad BousetouaneAgentic EvaluationsBridge

  18. VisualClaw: A Real-Time, Personalized Agent for the Physical World

    Jun 15, 2026Haoqin Tu, Jianwen Chen, Zijun Wang +14Agentic Video GenerationMultimodal Agents

  19. Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft

    Jun 14, 2026Juheon Yi, Jinglu Wang, Xiaoyi Zhang +1CollaborationAgentic Benchmarks

  20. Can Agents Read the Room? Benchmarking Visual Social Intelligence in Multimodal Simulation

    Jun 13, 2026Shijun Wan, Xuehai Wu, Jiwen Zhang +2Multimodal AgentsAgentic Benchmarks

  21. CacheRL:Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward

    Jun 12, 2026Md Amirul Islam, Sumiran Thakur, Huancheng Chen +3Agentic BenchmarksAgentic Tool Use

  22. Formalizing Numerical Analysis: An Agent Pipeline and Quality Audit Beyond Kernel Acceptance

    Jun 12, 2026Theodore Meek, Siyuan Ge, Di Qiu Xiang +2AutoformalizationTheorem

  23. Dialogue SWE-Bench: A Benchmark for Dialogue-Driven Coding Agents

    Jun 12, 2026Brendan King, Jeffrey FlaniganCoding AgentsSwe-Bench Verified

  24. AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility

    Jun 11, 2026Xiaoyuan Liu, Jianhong Tu, Yuqi Chen +26Agentic EvaluationsReproducibility

  25. EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

    Jun 11, 2026Yunhan Wang, Jiaan Wang, Lianzhe Huang +2Multi-Hop Question Answering BenchmarksAgentic Benchmarks

  26. Benchmarking AI Agents for Addressing Scientific Challenges Across Scales

    Jun 10, 2026Tianyu Liu, Allen Xin Wang, Antonia Panescu +30Agentic BenchmarksAutoresearch

  27. Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

    Jun 10, 2026Mengyu Zheng, Kai Han, Boxun Li +13Swe-Bench VerifiedCoding Agents

  28. Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness

    Jun 10, 2026Sawyer Zhang, Alexander Wang, Sophie LeiAgentic EvaluationsLarge Language Model Agents

  29. Search Discipline for Long-Horizon Research Agents

    Jun 9, 2026Adithya Srinivasan, Devesh ParagiriAutoresearchAgentic Evaluations

  30. Agent Skill Evaluation and Evolution: Frameworks and Benchmarks

    Jun 9, 2026Kexin Ding, Yang Zhou, Can Jin +3Skill EvolutionAgentic Evaluations

  31. ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity

    Jun 9, 2026Andrew Bo Liu, Samira Nedungadi, Bryce Cai +3Agentic BenchmarksLarge Language Model Agents

  32. TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning

    Jun 9, 2026Heming Zou, Qi Wang, Yun Qu +9Agentic Reinforcement-Learning FrameworkRollout

  33. VISTA: A Versatile Interactive User Simulation Toolkit for Agent Evaluation

    Jun 9, 2026Yunan Lu, Ryan Shea, Yusen Zhang +1Agentic EvaluationsUser Simulation

  34. T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains

    Jun 9, 2026Genta Indra Winata, Amartya Chakraborty, Yuzhen Lin +12Agentic BenchmarksMulti-Agent Systems

  35. Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages

    Jun 9, 2026Aman Sharma, Sushrut Thorat, Paras ChopraCoding AgentsAgentic Benchmarks

  36. STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios

    Jun 9, 2026Sirui Liang, Bohan Yu, Peiyu Wang +8Agentic BenchmarksClaws

  37. CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries

    Jun 8, 2026Vasudha Varadarajan, Akhila Yerukola, Mona T. Diab +1Cultural KnowledgeAdherence

  38. Bittensor Agent Arenas as a Trajectory Primitive: Distilling a Shopping Agent from ShoppingBench Subnet Traces

    Jun 8, 2026Shardul Bansal, Seth Schilbe, Jarrod BarnesAgentic BenchmarksAgentic Commerce

  39. OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics

    Jun 8, 2026Mingxian Lin, Shengju Qian, Yuqi Liu +9Agentic BenchmarksArena

  40. SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks

    Jun 8, 2026Hongcheng Gao, Hailong Qu, Jingyi Tang +18Spatial ReasoningMultimodal Agents

  41. Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops

    Jun 8, 2026Ziqian Zhong, Ivgeni Segal, Ivan Bercovich +3HackingAdversarial Evaluation

  42. Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing Human

    Jun 8, 2026Emre TuranScalable OversightAgentic Evaluations

  43. Building Customer Support AI Agents at 100M-User Scale: An Evaluation-Driven Framework

    Jun 7, 2026Aman Gupta, Kevin Rossell, Edesio Alcobaça +8Agentic EvaluationsArtificial Intelligence Agents

  44. ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents

    Jun 7, 2026Lu Jia, Haibo Tong, Feifei Zhao +5Large Language Model SafetyLarge Language Model Agents

  45. Online Agent-as-a-Judge: Situation-Generating Evaluation for Interactive Agents

    Jun 6, 2026Hyogon Ryu, Jeonghwan Kim, Yewon Lim +3Agentic EvaluationsLlm-As-A-Judge

  46. Closing the Sim-to-Real Gap: An Evaluation Framework for Autonomous Cyber Defense Configuration of Commercial EDR

    Jun 6, 2026Kerri Prinos, Lilianne BrushDefense StrategiesAgentic Evaluations

  47. PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

    Jun 6, 2026Zayx ShawnSelf-Evolving AgentsSelf-Evolution

  48. Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems

    Jun 5, 2026Yiyang Zhao, Zhuo Zhang, Qingxuan Le +2Agentic BenchmarksMulti-Agent Systems

  49. Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

    Jun 5, 2026Jiayu Wang, Weijiang Lv, Bowen Fu +8Agentic BenchmarksArtificial Intelligence Scientists

  50. Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests

    Jun 5, 2026Thanawat Lodkaew, Johannes Ackermann, Soichiro Nishimori +3Coding AgentsAgentic Evaluations

  51. SWE-Explore: Benchmarking How Coding Agents Explore Repositories

    Jun 5, 2026Shaoqiu Zhang, Yuhang Wang, Jialiang Liang +8Repository-Level Code UnderstandingCoding Agents

  52. The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective

    Jun 5, 2026Xiaoou Liu, Tiejin Chen, Weibo Li +2Sim-To-Real GapAgentic Evaluations

  53. SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?

    Jun 5, 2026Rishi Desai, Jesse Hu, Joan Cabezas +23Agentic BenchmarksLong-Horizon Tasks

  54. Benchmark Everything Everywhere All at Once

    Jun 4, 2026Shiyun Xiong, Dongming Wu, Peiwen Sun +5Agentic BenchmarksAgentic Systems

  55. AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

    Jun 4, 2026Jiayu Liu, Cheng Qian, Zhenhailong Wang +10Large Language Model PlanningClassical Planning

  56. TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework

    Jun 4, 2026Bobby Yan, Fredrik KjolstadAgentic BenchmarksCoding Agents

  57. Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents

    Jun 4, 2026Kaixuan Liu, Guojun Xiong, Weinan Zhang +1Large Language Model AgentsAutoregressive Rollout

  58. ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developer

    Jun 4, 2026Jintao Huang, Xiaomin Li, Gaurav Mittal +1Llm-Based AgentAgentic Benchmarks

  59. Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents

    Jun 3, 2026Dae Yon Hwang, Raunaq Suri, Valentin Villecroze +4Agentic Reinforcement-Learning FrameworkLarge Language Model Agents

  60. AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?

    Jun 3, 2026Zhangchen Xu, Junda Chen, Yue Huang +16Long-Horizon TasksAgentic Optimization