AI Agent Benchmarks

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,013

All topics
CardsList
  1. BrickBench: Evaluating Agentic Brick Design

    Oct 8, 2026Peter Kulits, Yiqing Xu, R. Kenny Jones +2AI Agent BenchmarksAI Coding Agents

  2. Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition

    Oct 8, 2026Kaisen Yang, Qingle Liu, Kejin Wang +24Game-Playing AgentsSelf-Improving Agents

  3. DataSense-Bench: The First Step Toward an AI Scientist

    Oct 8, 2026Yudi Zhang, Mingyu Cao, Lu Yin +2Training Data SelectionAI Agent Benchmarks

  4. A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

    Oct 8, 2026Ming Chen, Rong-Xi Tan, Ke Xue +8Black-Box OptimizationAI Agent Benchmarks

  5. SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction

    Oct 8, 2026Hexuan Deng, Yue Wang, Wenyu Jiang +13Coding AgentsAI Agent Benchmarks

  6. Closed-loop evaluation of LLM agents for embedded software development

    Oct 8, 2026Jorge García-Carrasco, Sergio García-Carrasco, Alejandro Maté +1AI Coding AgentsLLM Agent Evaluation

  7. Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

    Oct 8, 2026Yuxuan Cao, Junlong Li, Hao Li +1AI Agent BenchmarksSpatial Reasoning Benchmarks

  8. When Interfaces Speak: Data-Aware Generative UI Harness for Active Interaction

    Oct 8, 2026Xiaolong Li, Xiaohan Xu, Jinyang Li +5Human-AI InteractionLLM Agent Harnesses

  9. On the Clock: Towards Punctual and Productive Time-Budgeted AI Agents

    Oct 7, 2026Aaron Wang, Neelabh Madan, Vlad Sobal +5Runtime Enforcement for AI AgentsLLM Agent Harnesses

  10. SciExam for ENSO: Can AI Agents Build Climate Models?

    Oct 7, 2026Yinling Zhang, Langchen Liu, Dongbin Xiu +4AI for ScienceAI Agent Benchmarks

  11. RSIGym: A Flexible Environment for Recursive Self-Improvement

    Oct 7, 2026Fanqing Meng, Lingxiao Du, Haocheng Lu +6LLM Agent Self-ImprovementRecursive Self-Improvement

  12. AgentTime: Can Agents Estimate and Control Their Own Runtime?

    Oct 7, 2026Michael Ofengenden, Maksym AndriushchenkoAI Agent BenchmarksLong-Horizon Agent Evaluation

  13. LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets

    Oct 7, 2026Jun Zhao, Leiming Fu, Yanbo Wen +9LLM Agent EvaluationAI Agent Benchmarks

  14. Coding-Agent Benchmarks Should Match Their Users' Task Flows

    Oct 7, 2026Igor Slinko, Yaroslav Golubev, Sergey TitovLLM Agent EvaluationAI Agent Benchmarks

  15. DrugTargetWorld: A Synthetic Biobank for Training and Benchmarking AI Scientists

    Oct 7, 2026Samuel Margolis, Paul Schmiedmayer, Alan Huang +12Drug DiscoveryAI Agent Benchmarks

  16. RSI-Forge: From Research Papers to Environments for Recursive Self-Improvement

    Oct 7, 2026Renxiong Wang, Darvin Yi, Abril Herrlein +16Self-Improving AgentsRecursive Self-Improvement

  17. DUDA-Bench: Benchmarking LLM Agents on Multimodal Data-Driven Urban Diagnosis

    Oct 7, 2026Yizhi Song, Hang Ni, Weijia Zhang +1Data Analysis AgentsAI Agent Benchmarks

  18. ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation

    Oct 6, 2026Arkajyoti Chakraborty, Aryan Tayal, Ishika Agarwal +5Tool-Using AgentsAI Agent Benchmarks

  19. GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks

    Oct 6, 2026Gabriel Diaz-Ireland, Diego Prieto-Herráez, Mario García Peces +3Tool-Use EvaluationLLM Agent Evaluation

  20. ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding

    Oct 6, 2026Hanjun Luo, Xiucheng Zhang, Zhuoning Xu +4AI Coding AgentsAI Risk Management

  21. ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?

    Oct 6, 2026Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2Continual Learning for LLM AgentsLLM Agent Evaluation

  22. PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence

    Oct 5, 2026Dheeraj Varghese, Anna Vettoruzzo, Walter Simoncini +5AI Agent Benchmarks

  23. ArtifactArena: Evaluating Models by What They Build in the Physical World

    Oct 5, 2026Kushagra Tiwary*, David Mayo*, Nikhil Behari +5Robotics SimulationRobotics

  24. What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents

    Oct 5, 2026Zhongxiang Sun, Jiahao Yan, Hongkang Zhao +5Software Engineering AgentsHuman-in-the-Loop Evaluation

  25. InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems

    Oct 5, 2026Enxin Song, Suhao Yu, Yifei Xu +7Streaming Video UnderstandingVideo-Language Model Evaluation