Benchmark Design

Ask about this topic

Momentum

28 papers in the last four weeks, up 65% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 285

All topics
CardsList

  1. The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models

    Oct 27, 2025Timo Freiesleben, Sebastian ZezulkaBenchmark DesignBenchmark Validity

    LMU Munich, MCMP & MCML · University of Tübingen

  2. oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

    Oct 9, 2025Ruiling Xu, Yifan ZhangLLM EvaluationBenchmark Design

    University of Illinois Urbana-Champaign

  3. TOPO-Bench: An Open-Source Topological Mapping Evaluation Framework with Quantifiable Perceptual Aliasing

    Oct 5, 2025Jiaming Wang, Jizhuo Chen, Diwen Liu +1Benchmark DesignMap-Based Localization

  4. fev-bench: A Realistic Benchmark for Time Series Forecasting

    Sep 30, 2025Oleksandr Shchur, Abdul Fatir Ansari, Caner Turkmen +5Benchmark DesignTime Series Forecasting

    AWS · Keystone AI · Prior Labs +2

  5. IMProofBench: Benchmarking AI on Research-Level Mathematical Proof Generation

    Sep 30, 2025Johannes Schmitt, Gergely Bérczi, Jasper Dekoninck +57Mathematical Reasoning BenchmarksLLM Evaluation

  6. WirelessMathBench-XL: An Auditable Benchmark for Wireless Mathematical Reasoning

    Sep 27, 2025Xin Li, Mengbing Liu, Yiyang Zhu +4Mathematical Reasoning BenchmarksBenchmark Auditing

    Nanyang Technological University

  7. PDFBench: A Benchmark for De novo Protein Design from Function

    May 25, 2025Jiahao Kuang, Nuowei Liu, Jie Wang +3Benchmark DesignMolecular Generation

    School of Computer Science and Technology, East China Normal University · Institute of Artificial Intelligence (TeleAI), China Telecom · College of Foreign Languages and Literatures, Fudan University

  8. Multimodal Language Models as Text-to-Image Model Evaluators

    May 1, 2025Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat +5LLM-as-a-JudgeBenchmark Design

    1FAIR at Meta · University of Texas at Austin · ∗Work done at Meta +3

  9. BiasBench: A reproducible benchmark for tuning the biases of event cameras

    Apr 25, 2025Andreas Ziegler, David Joseph, Thomas Gossard +2Event-Based VisionBenchmark Design

    Cognitive Systems Group, University of Tübingen, Germany

  10. Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks

    Apr 10, 2025Pavel Chizhov, Anton Changalidis, Vishnu Prasad Vijaya Kumar +4Benchmark AuditingReasoning Evaluation

    1CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt · 2PleIAs, Paris, France

  11. BigO(Bench): Can LLMs Generate Code with Controlled Time and Space Complexity?

    Mar 19, 2025Pierre Chambon, Baptiste Roziere, Benoit Sagot +1Benchmark DesignCode Generation

    FAIR at Meta · Inria · Work done at Meta, now working at Mistral AI

  12. Is Your Benchmark Still Useful? Dynamic Benchmarking for Code Language Models

    Mar 9, 2025Batu Guan, Xiao Wu, Yuanyuan Yuan +1Benchmark DesignData Contamination in Language Models

    The Chinese University of Hong Kong · Huazhong University of Science and Technology · ETH Zurich

  13. GraphChase: A Platform and Benchmark for Urban Network Security Games

    Jan 29, 2025Shuxin Zhuang, Shuxin Li, Tianji Yang +4Benchmark DesignAI Agent Benchmarks

    City University of Hong Kong CAIR, Hong Kong Institute of Science & Innovation, CAS Hong Kong, Hong Kong · Nanyang Technological University Singapore, Singapore · Georgia Institute of Technology Atlanta, Georgia, United States +2

  14. Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

    Jan 18, 2025Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12Benchmark ConstructionBenchmark Design

    The Hong Kong University of Science and Technology · Guangzhou HKUST Fok Ying Tung Research Institute · The Chinese University of Hong Kong +6

  15. What does the model actually see? Evaluation protocols and input availability in data-driven prediction of room acoustic parameters

    Date pendingAk\in OktavBenchmark DesignRoom Acoustics

    Vibration and Acoustics Laboratory (VAL) and Department of Mechanical Engineering, Alanya Alaaddin Keykubat University, 07425, Antalya, Türkiye