Benchmark Design

Ask about this topic

Momentum

28 papers in the last four weeks, up 65% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 285

All topics
CardsList

  1. UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation

    Aug 2, 2026Binshuang LiConditional Average Treatment Effect EstimationBenchmark Design

    Independent Researcher USA

  2. SDDMO-Bench: A Benchmark Suite for Streaming Data-Driven Dynamic Multi-Objective Optimization

    Aug 1, 2026Wenjie Xiao, Hui Bai, Junhao ChenBenchmark DesignMulti-Objective Evolutionary Optimization

    School of Xiangtan University Xiangtan, China

  3. A reproducible and extensible framework for benchmarking competing risks survival models

    Jul 31, 2026Begoña B. Sierra, Colin McLean, Peter S. Hall +2Benchmark DesignSurvival Analysis

    Cancer Research UK Scotland Centre · Institute of Genetics and Cancer · University of Edinburgh +2

  4. DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat

    Jul 31, 2026Ismayil Ismayilov, Atakan Kara, Kaan OktayLogical ReasoningGame-Playing Agents

    fal

  5. ModelEquivBench: Certifying Multi-Relational Evaluation of LLM-Generated Optimization Models

    Jul 31, 2026Penglin Zhu, Jungang XuLLM EvaluationPairwise Comparison

    School of Computer Science and Technology, University of the Chinese Academy of Sciences

  6. InferQ: A Database-Oriented Benchmark for Quantum Circuits Simulation

    Jul 31, 2026Andrei Ilinescu, Aadi Patwardhan, Rihan HaiBenchmark DesignSynthetic Benchmark Generation

    Delft University of Technology, Delft, The Netherlands

  7. LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data

    Jul 30, 2026Debopam Sanyal, Hongjie Chen, Alexey Tumanov +1Benchmark DesignCloud Computing

    Georgia Tech · Dolby Labs

  8. Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

    Jul 29, 2026Jiwon Jang, Kisu Yang, Heuiseok Lim +1AI Agent ReliabilityBenchmark Design

    VAIV Company · Seoul National University · Korea University

  9. NeoRacer: An Open, Standardized 1:12 Scale Autonomous Race Car for Benchmarking and Education

    Jul 29, 2026Koneshka Bandyopadhyay, Ansh Mehta, Bassel El Mabsout +1Benchmark DesignAutonomous Racing

    Neobotics Foundation, Inc., Cambridge, MA, USA · Boston University, Boston, MA, USA

  10. PlatformBid: An Auto-Bidding Benchmark from a Unified Advertising Platform's Perspective

    Jul 29, 2026Shengtian Yang, Yewen Li, Peng Jiang +5Benchmark DesignAutomated Bidding

    Nanyang Technological University Singapore, Singapore

  11. When benchmark inferences do not compose: Projectibility in AI evaluation

    Jul 28, 2026Brett ReynoldsBenchmark DesignBenchmark Validity

    Humber Polytechnic & University of Toronto

  12. AMPBench-MT: A Homology-Controlled Benchmark for Antimicrobial Peptide Potency, Spectrum, and Safety Prediction

    Jul 28, 2026Ziheng Zhou, Huiyu Luo, Xiaohu Zhu +4Benchmark DesignToxicity Detection

    Shanghai Ocean University, Shanghai, China · Tongji University, Shanghai, China · Center for Safe AGI, Shanghai, China +1

  13. GraphRareBench: An Auditable Graph-Evidence Benchmark for Phenotype-Driven Rare-Disease Diagnosis

    Jul 27, 2026Guiling Guo, Jia Yang, Jiahao Xu +3Rare Disease DiagnosisBenchmark Design

    National Institute for Data Science in Health and Medicine, Xiamen University · Institute of Artificial Intelligence, Xiamen University · School of Medicine, Xiamen University +1

  14. DBA-Bench: A Production-Fidelity Benchmark for LLM-Based Database Operations Agents

    Jul 24, 2026Junming Chen, Junyang Jiang, Xu Chen +2Benchmark DesignAI Agent Benchmarks

    University of Electronic Science and Technology of China Chengdu, China

  15. CEL: Comprehensive Counterfactual Explanations Library and Benchmark

    Jul 24, 2026Oleksii Furman, Łukasz Lenkiewicz, Marcel Musiałek +1Counterfactual EvaluationExplainable Artificial Intelligence

    Wrocław University of Science and Technology, Wrocław, Poland

  16. Evaluation design conditions the expert-vs-auto MeSH gap: a controlled comparison of bag-of-words and BiomedBERT on the Cohen benchmark

    Jul 23, 2026Samuel M. Okoe-MensahBenchmark DesignEvidence Selection

    Accra, Ghana

  17. WaveformQA: Benchmarking LLM Temporal Reasoning on Digital Waveforms

    Jul 22, 2026Yichuan Liu, Daniel Cummings, Nick VadlamudiLLM EvaluationTime Series QA

    Tenstorrent Austin, TX, USA

  18. KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?

    Jul 22, 2026Peiyu Zang, Jian Tao, Jialing Zhang +4GPU Kernel OptimizationBenchmark Design

    Beijing Normal University · BAAI · Beijing Jiaotong University +2

  19. CLOSER-Bench: Evaluating Budgeted Cross-Stage Design Closure for Hardware Agents

    Jul 18, 2026Peilong Zhou, Zhirong Chen, Cangyuan Li +4Benchmark DesignElectronic Design Automation

    1SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences

  20. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning

    Jul 17, 2026Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2LLM EvaluationBenchmark Design

    Operations, Information and Decisions Department, The Wharton School, University of Pennsylvania, Philadelphia, 19104, PA, USA · Heinz College of Information Systems and Public Policy, Carnegie Mellon University, Pittsburgh, 15213, PA, USA · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, 19104, PA, USA +1

  21. CFM-Bench: A Unified Multi-Domain, Multi-Task Benchmark for Channel Foundation Models

    Jul 16, 2026Yuan Gao, Wenjun Yu, Jun Jiang +3Wireless Foundation ModelsBenchmark Design

    School of Communication and Information Engineering, Shanghai University, Shanghai, China · Xi’an Jiaotong-Liverpool University, Suzhou, China

  22. The 2nd International StepUP Competition for Biometric Footstep Recognition: From Steps to Strides

    Jul 15, 2026Robyn Larracy, Anant Gupta, Gourav Gupta +8Biometric IdentificationBenchmark Design

    University of New Brunswick, Canada · ArogyaPandit Private Limited, India · Universit´e de Haute-Alsace, IRIMAS UR 7499, France +1

  23. How Many Tasks Are Enough for Agent Benchmark Decisions? A Replay Analysis of Public LLM Agent Benchmarks

    Jul 14, 2026Wei-Jung HuangPairwise ComparisonBenchmark Design

    Independent Researcher · United States

  24. Good Benchmarks

    Jul 13, 2026Ivan BercovichBenchmark DesignSoftware Engineering Benchmarks

  25. AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification

    Jul 13, 2026Lingkai Kong, Zijian Wu, Yuzhe Gu +11Mathematical Reasoning BenchmarksAutomated Theorem Proving

    Shanghai AI Laboratory · Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong +1

  26. AutoMatBench: An Automatic Optimization Toolkit for the Acceleration of Material Properties Prediction Benchmarking

    Jul 13, 2026Hongxiao Li, Wanling GaoBenchmark DesignOOD Generalization

    Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

  27. An Empirical Study for Android-to-OpenHarmony GUI Test Migration

    Jul 13, 2026Yakun Zhang, Xinjia Chen, Yiyun Chen +6Mobile GUI AutomationAutomated Software Testing

    Harbin Institute of Technology, Shenzhen, China · Beijing Institute of Technology, China · Beihang University, China +1

  28. Latency-Aware Bid Acceptance under Operational Feasibility: A Public Benchmark with Hindsight Ceilings

    Jul 8, 2026Aswin ChandrasekaranBenchmark DesignVehicle Routing

    Bubba AI

  29. BatteryLake: Agentic, Physics-Grounded Curation of Heterogeneous Battery Aging Data and Benchmarking

    Jul 6, 2026Tianwen Zhu, Hao Wang, Yonggang WenRUL EstimationBenchmark Design

    College of Computing and Data Science, Nanyang Technological University, Singapore

  30. ClassicLogic: A Knowledge-Driven Benchmark of Classic Puzzle Games for Evaluating Compositional Generalization

    Jul 6, 2026Mahnoor Shahid, Hannes RotheLogical ReasoningBenchmark Design

    Universität Duisburg-Essen Germany