Artificial Intelligence Benchmarks

Ask about this topic

Momentum

3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 41

All topics
CardsList

  1. ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds

    Sep 24, 2026Ming Zhang, Zhenghao Xiang, Peizhong Gao +17Artificial Intelligence BenchmarksExploration

    Fudan University

  2. The Convention Gap: Towards Measuring Implicit Communication in Cooperative AI Evaluation

    Sep 10, 2026Makoto Fukushima, Hua-Dong Xiong, Ehsan Moradi PariAgentic EvaluationsArtificial Intelligence Benchmarks

    Honda Research Institute Japan · Honda Research Institute USA

  3. Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation

    Sep 10, 2026Koutian Wu, Junjie Zhou, Ergan Shang +6Artificial Intelligence BenchmarksLarge Language Model Benchmarks

    Earth-Space-AI · Tacite AI · Hangzhou Dianzi University +6

  4. K-Bench: measuring model performance on real scientific agent requests

    Aug 21, 2026Aubrey M. Brueckner, Darshil Patel, Yuhuan He +1Artificial Intelligence BenchmarksLlm-As-A-Judge

    1K-Dense, Inc.

  5. DiG-bench: Discovery in Games

    Aug 12, 2026Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan +13Agentic BenchmarksArtificial Intelligence Benchmarks

    1Thinking About Thinking · 2Independent · University of Oxford +5

  6. Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Framework

    Aug 12, 2026Avinash Agarwal, Vridhi JainArtificial Intelligence BenchmarksMulti-Turn Benchmark

    Unique Identification Authority of India, New Delhi, India

  7. LexKairos: Benchmarking Legal Temporal Capabilities in LLMs

    Aug 10, 2026Chenyang Li, Zejia Feng, Yuqin Huang +2Legal Reasoning TasksLegal Domain

    University of Oxford · Sun Yat-Sen University · Tsinghua University

  8. What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)

    Aug 6, 2026Ro Encarnación, Tina Behzad, Emma Lurie +1Artificial Intelligence BenchmarksArtificial Intelligence Safety

    University of Pennsylvania · Stony Brook University

  9. Adversarial Fast-Moving Real-World Domains as Test Beds for Benchmarking AI Scientist Capabilities

    Aug 4, 2026William Bolton, Philip TorrScientific IdeationArtificial Intelligence Benchmarks

    Department of Engineering Science, University of Oxford, Oxford, United Kingdom.

  10. On the missing benchmarks layer and a potential solution

    Aug 4, 2026Francis F Daniel, Mauro Ibañez, Francis Perelman +1Artificial Intelligence BenchmarksArtificial Intelligence Systems

    1SURUS · 2Independent

  11. On the missing data layer and a potential solution

    Aug 3, 2026Francis F Daniel, Mauro Ibañez, Francis Perelman +1Artificial Intelligence InfrastructureLarge Databases

    1SURUS · 2Independent

  12. InfoOps Bench: A live information operations safety benchmark

    Jul 30, 2026Dorian Quelle, Lisa-Maria Neudert, Jonathan Bright +1Artificial Intelligence BenchmarksFact-Checking

  13. Evaluating RAG for French immigration law: a benchmark and baseline study

    Jul 27, 2026Annia Abtout, Julien Delaunay, Monika Ewa RakoczyLuxembourgishArtificial Intelligence Benchmarks

    Talan, Paris, France

  14. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning

    Jul 17, 2026Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2Artificial Intelligence BenchmarksHuman-Annotated Benchmark

    Operations, Information and Decisions Department, The Wharton School, University of Pennsylvania, Philadelphia, 19104, PA, USA · Heinz College of Information Systems and Public Policy, Carnegie Mellon University, Pittsburgh, 15213, PA, USA · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, 19104, PA, USA +1

  15. Can We Trust Item Response Theory for AI Evaluation?

    Jul 16, 2026Han Jiang, Sunbeom Kwon, Jinwen Luo +2Artificial Intelligence BenchmarksItem Response Theory

    1Johns Hopkins University · University of Illinois Urbana-Champaign · University of California, Los Angeles

  16. MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection

    Jul 16, 2026Goktug OzkanArtificial Intelligence SafetyClinician Trust

    Department of Internal Medicine, Kutahya Emet Dr. Fazil Dogan State Hospital, Kutahya, Turkiye

  17. Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

    Jul 9, 2026Matteo Santelmo, Xiuying Wei, Israa Fakih +5Artificial Intelligence BenchmarksArtificial Intelligence Models

    Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland

  18. Two AI Metrics Diverged: Will it Make All the Difference?

    Jul 1, 2026Alex Fogelson, Zachary A. Brown, Hans Gundlach +2Artificial Intelligence BenchmarksPerformance Metric

    MIT FutureTech, CSAIL, Cambridge, MA, USA.

  19. BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases

    Jun 23, 2026Qi Chen, Wenxuan Li, Pedro R. A. S. Bassi +14Cancer DetectionArtificial Intelligence Benchmarks

    Johns Hopkins University · German Cancer Research Center · University Hospital Basel +10

  20. Generative Engine Optimization at Scale: Measuring Brand Visibility Across AI Search Engines

    Jun 18, 2026Pratyush KumarBrand-As-MemoryAgentic Search

    Ranqo

  21. ForecastBench-Sim: A Simulated-World Forecasting Benchmark

    Jun 17, 2026Jaeho Lee, Nick Merrill, Ezra KargerArtificial Intelligence Benchmarks

    1Forecasting Research Institute · 2UC Berkeley

  22. Legal Reasoning Is Not Lawyering: Rethinking Legal Benchmarks for Pro Se Access to Justice

    Jun 16, 2026Andrew Lou, David ShinLegal DomainArtificial Intelligence Benchmarks

    Yale Law School, USA

  23. Bounded Difference Concentration for Infinitely Exchangeable Sequences with Applications to AI Benchmark Uncertainty

    Jun 16, 2026Fangyuan Lin, Spencer Frei, Victor H. de la PenaConcentration BoundsArtificial Intelligence Benchmarks

    Department of Statistics, Columbia University · Google DeepMind

  24. AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Ranges

    Jun 12, 2026Fengyu Liu, Jiarun Dai, Yihe Fan +11Artificial Intelligence BenchmarksAdversarial Evaluation

    Fudan University

  25. Benchmarking AI for low-resource contexts: Thinking beyond leaderboards

    May 27, 2026Aakash Pant, Kavya Shah, Apoorv Agnihotri +3Artificial Intelligence BenchmarksArtificial Intelligence Evaluation

    Wadhwani AI Global

  26. Automated Benchmark Auditing for AI Agents and Large Language Models

    May 25, 2026Junlin Wang, Federico Bianchi, Shang Zhu +4Artificial Intelligence BenchmarksAgentic Benchmarks

    Duke University · Together AI · Stanford University

  27. AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems

    May 24, 2026Michael Hardy, Anka Reuel, Lijin Zhang +6Artificial Intelligence BenchmarksLeaderboard

    Stanford University, Stanford, CA, USA · BroadMetrics, Edison, NJ, USA

  28. RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges

    May 24, 2026Ruize Li, Zhibin Wen, Tao Han +5Artificial Intelligence Weather ModelsNumerical Weather Prediction

    The Hong Kong University of Science and Technology · Nanjing University · Southern University of Science and Technology +2

  29. Teaching AI Through Benchmark Construction: QuestBench as a Course-Based Practice for Accountable Knowledge Work

    May 20, 2026Haiyang Shen, Jiuzheng Wang, Taian Guo +9Artificial Intelligence EducationArtificial Intelligence Benchmarks

    1Peking University · 2Advanced Institute of Big Data

  30. Position: State-of-the-Art Claims Require State-of-the-Art Evidence

    May 17, 2026YongKyung OhArtificial Intelligence BenchmarksEvidence Conflict

    Medical & Imaging Informatics (MII), University of California, Los Angeles (UCLA)

  31. The Evaluation Trap: Benchmark Design as Theoretical Commitment

    May 13, 2026Theodore J KalaitzidisArtificial Intelligence BenchmarksSelf-Evaluation

    Brown University

  32. Unsteady Metrics and Benchmarking Cultures of AI Model Builders

    May 13, 2026Stefan Baack, Christo Buschek, Maty BohacekArtificial Intelligence BenchmarksMulti-Dimensional Evaluation

    Independent Researcher · Mozilla Fellowship · Stanford University

  33. The Scaling Law of Evaluation Failure: Why Simple Averaging Collapses Under Data Sparsity and Item Difficulty Gaps, and How Item Response Theory Recovers Ground Truth Across Domains

    May 11, 2026Jung Min KangArtificial Intelligence BenchmarksItem Response Theory

    Independent Researcher · Seoul, South Korea

  34. NeuralBench: A Unifying Framework to Benchmark NeuroAI Models

    May 8, 2026Hubert Banville, Stéphane d'Ascoli, Simon Dahan +12Small-Sample Electroencephalogram DatasetsArtificial Intelligence Benchmarks

    Lucy · Brain & AI team, Meta FAIR · MIND, Inria +2

  35. Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference

    May 1, 2026Yuxuan Gao, Megan Wang, Yi Ling YuArtificial Intelligence BenchmarksArtificial Intelligence Systems

    University of Pennsylvania · OpenMesh AI · Columbia University

  36. Railway Artificial Intelligence Learning Benchmark (RAIL-BENCH): A Benchmark Suite for Perception in the Railway Domain

    Apr 24, 2026Annika Bätz, Pavel Klasek, Seo-Young Ham +3RelightingAutonomous Driving Perception

    Karlsruhe Institute of Technology, Engler-Bunte-Ring 21, 76131 Karlsruhe, Germany · Deutsches Zentrum für Schienenverkehrsforschung (DZSF), August-Bebel-Straße 10, 01219 Dresden, Germany · DB InfraGO AG, EUREF-Campus 17, 10829 Berlin, Germany

  37. E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

    Apr 20, 2026Koya Sakamoto, Taiki Miyanishi, Daichi Azuma +6Viewpoint-Dependent Active PerceptionMultiple Vision Tasks

    The University of Tokyo, Japan · National Institute of Informatics, Japan · Institute of Science Tokyo, Japan +3

  38. MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition

    Apr 17, 2026Farhad Abtahi, Abdolamir Karbalaie, Eduardo Illueca-Fernandez +1Artificial Intelligence BenchmarksMetacognition

    Department of Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institutet, 17177 Stockholm, Sweden · Department of Clinical Physiology, Karolinska University Hospital, 17176 Stockholm, Sweden · Department of Biomedical Engineering and Health Systems, KTH Royal Institute of Technology, 14157 Huddinge, Sweden +2

  39. Riemann-Bench: A Benchmark for Moonshot Mathematics

    Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Research-Level MathematicsOlympiad-Level Problem

    Surge AI

  40. When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation

    Feb 18, 2026Mubashara Akhtar, Anka Reuel, Prajna Soni +34Artificial Intelligence BenchmarksLarge Language Model Benchmarks

    ETH Zurich · ETH AI Center · Stanford University +23

  41. BRIDGE: Predicting Human Task Completion Time From Model Performance

    Feb 6, 2026Fengyuan Liu, Jay Gala, Nilaksh +3Artificial Intelligence BenchmarksPsychometric Properties

    Mila - Quebec AI Institute · McGill University · Polytechnique Montréal +3