Benchmark Design

Ask about this topic

Momentum

28 papers in the last four weeks, up 65% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 285

All topics
CardsList

  1. HamQASBench: A Hamiltonian-Informed Diagnostic Benchmark for Evaluating Quantum Architecture Search

    Jul 6, 2026Jiayang Niu, Akib Karim, Yan Wang +5Benchmark ConstructionBenchmark Design

    School of Computing Technologies RMIT University · Quantum Systems School of Computing Technologies RMIT University · Data61, CSIRO

  2. TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data

    Jul 4, 2026Jialin Zhang, Fenghao Dong, Yajie Zhou +2Benchmark DesignSynthetic Tabular Data Generation

    †University of Hong Kong ¶Tongji University · ‡Carnegie Mellon University · §University of Maryland, College Park

  3. Scientific Code Search at Scale: A Multi-Domain Dataset and Benchmark

    Jul 3, 2026Nishan Pantha, Pranath Reddy Kumbam, Sajil Awale +8Benchmark DesignInformation Retrieval

    The University of Alabama in Huntsville (UAH), Huntsville, AL, USA · Universities Space Research Association (USRA), USA · NASA Marshall Space Flight Center, Huntsville, AL, USA

  4. Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation

    Jul 3, 2026Alberto Foresti, Ivan Butakov, Alexander Tolmachev +3Copula ModelsBenchmark Design

    Department of Data Science EURECOM 450 Route des Chappes, 06410 Biot, France · Applied AI Institute Presnenskaya naberezhnaya, 12, 121205 Moscow, Russia · Moscow Independent Research Institute of Artificial Intelligence Bolshaya Cheremushkinskaya Street, 20, Building 4, Premises 3/1, 117218 Moscow, Russia

  5. TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution

    Jul 2, 2026Jiale Amber Wang, Kaiyuan Wang, Pengyu NieAutomated Software TestingBenchmark Design

    University of Waterloo · 2Google

  6. PACE: A Proxy for Agentic Capability Evaluation

    Jul 2, 2026Yueqi Song, Lintang Sutawika, Jiarui Liu +8Benchmark DesignLLM Agent Evaluation

    Carnegie Mellon University · Salesforce AI Research

  7. MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models

    Jul 2, 2026Yuanzhi Liu, Shousheng Zhao, Bo Zhou +2VLM EvaluationBenchmark Construction

    1Beijing University of Posts and Telecommunications, Beijing, China

  8. Meta-Benchmarks for Financial-Services LLM Evaluation

    Jul 2, 2026Blair HudsonFinancial ServicesLLM Evaluation

    Commonwealth Bank of Australia

  9. Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?

    Jul 1, 2026Zhi Chen, Zhensu Sun, Yuling Shi +2Benchmark AuditingBenchmark Design

    Singapore Management University · Singapore, Singapore · Shanghai Jiao Tong University +1

  10. Optimization Algorithms for Joint OFDM Waveform Design and RIS Configuration in 6G Networks: From Convex Relaxation to Foundation Models

    Jun 30, 2026Ahmet Kaplan6G NetworksBenchmark Design

    Department of AI Engineering, Istanbul Medipol University, Istanbul, Türkiye

  11. Exploring Differences Between Tabular Enterprise Data and Public Benchmarks

    Jun 29, 2026Myung Jun Kim, Maximilian Schambach, Frank Essenberger +2Benchmark DesignTabular ML

    SAP SE, Germany.

  12. CAN We Trust Your Results? A Cross-Dataset Study of Automotive IDS Evaluation

    Jun 29, 2026Beatrix Koltai, Gergely Acs, Andras GazdagBenchmark DesignNetwork Intrusion Detection

    Laboratory of Cryptography and System Security (CrySyS Lab), Budapest University of Technology and Economics, Hungary · HUN-REN-BME Information Systems Research Group, Hungary

  13. MaDI-Bench: An End-to-End Data Integration Benchmark

    Jun 29, 2026Aaron Steiner, Ralph Peeters, Christian BizerBenchmark Design

    Data and Web Science Group University of Mannheim Mannheim, Germany

  14. Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark

    Jun 29, 2026Matthias Blaschke, Daniel Kienzle, Zsuzsanna Koczor-Benda +3Molecular OptimizationBenchmark Design

    University of Augsburg · CAAPS† · Germany +2

  15. SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

    Jun 29, 2026Jian Zhu, Yuzheng Zhang, Zeyao Ma +11Benchmark DesignSpreadsheet Automation

    School of Information, Renmin University of China · Aptura AI · Shortcut AI +1

  16. SFBench: The SciFy Scientific Feasibility Benchmark

    Jun 28, 2026Cash Costello, James Mayfield, Elsbeth Turcan +7LLM EvaluationBenchmark Design

    Johns Hopkins Applied Physics Laboratory

  17. Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving

    Jun 28, 2026Pawan Sasanka Ammanamanchi, Siddharth Bhat, Stella BidermanBenchmark DesignBenchmark Validity

    Eleuther AI · Cambridge

  18. Benchmarking on Tasks That Matter: Dataset Selection for Preserving Model Rankings

    Jun 26, 2026Rostislav Gusev, Alexey ZaytsevBenchmark Design

    Applied AI Institute Moscow, Moscow region, Russian Federation

  19. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerAutomatic DifferentiationPDE Solving

    Pasteur Labs, Brooklyn, NY, USA

  20. Evaluating LLMs on Real-World Software Performance Optimization

    Jun 24, 2026Ezgi Sarıkayak, Wenchao Gu, Hesham Ghonim +1Software EngineeringLLM Evaluation

    Siemens AG · Technical University of Munich Munich, Germany · Technical University of Munich Heilbronn, Germany

  21. Test-Input Generation for Tensor Programs: What Actually Finds Kernel Bugs

    Jun 23, 2026Dipankar SarkarAutomated Software TestingBenchmark Design

    Arizona State University, USA

  22. BCoughBench: Benchmarking Respiratory Acoustic Foundation Models Under Body-Coupled Wearable Sensor Conditions

    Jun 23, 2026Mayur Sanap, Prasanna Desikan, Edgar LobatonBenchmark DesignWearable Sensing

    Centific Global Solutions Inc. · USA · North Carolina State University

  23. PROTECT-90: A Fault Dataset for Power System Protection

    Jun 23, 2026Julian Oelhaf, Georg Kordowich, Christian Bergler +3Benchmark DesignSynthetic Benchmark Generation

    1Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Electrical Energy Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Department of Electrical Engineering, Media and Computer Science, Ostbayerische Technische Hochschule Amberg-Weiden, Amberg, Germany

  24. BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

    Jun 23, 2026Jin Huang, Yutong Xie, Wanli Song +4Benchmark DesignHuman Behavior Simulation

    University of Michigan · MobLab · Stanford University +1

  25. EEG Benchmarking Needs a Task Specification Layer: NeuroDoc for Rulebook-Guided, Executable Benchmark Construction

    Jun 22, 2026Chengxuan Qin, Zhige Chen, Shu Peng +9ElectroencephalographyBenchmark Construction

    School of Advanced Technology, Xi’an Jiaotong-Liverpool University · School of Electrical Engineering, Electronics and Computer Science, University of Liverpool · Department of Data Science and Artificial Intelligence, Hong Kong Polytechnic University +1

  26. HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions

    Jun 22, 2026Yuichi TatenoBenchmark DesignText Embedding Evaluation

  27. LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation

    Jun 19, 2026Xingyu Chen, Rui Wang, Zhaopeng Tu +1LLM EvaluationBenchmark Design

    Shanghai Jiao Tong University, Tencent · Shanghai Jiao Tong University · Tencent

  28. JAMER: Project-Level Code Framework Dataset and Benchmark on Professional Game Engines

    Jun 18, 2026Jianwen Sun, Chuanhao Li, Zizhen Li +5Benchmark DesignCode Generation

    1Alaya Studio · 2Nankai University, Tianjin, China · 3Shanghai Innovation Institute, Shanghai, China

  29. MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discovery

    Jun 17, 2026Hongxuan Liu, Roman Bushuiev, Ivy Lightheart +12Benchmark DesignBenchmark Contamination

    1Massachusetts Institute of Technology · Institute of Organic Chemistry and Biochemistry of the Czech Academy of Sciences · 3Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague +2

  30. StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns

    Jun 17, 2026Vlad Sobal, Shuo Yang, Yuting Zhang +2AI Coding AgentsBenchmark Design

    AWS Agentic AI