Benchmark Construction

Ask about this topic

Momentum

8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 51

All topics
CardsList

  1. WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses

    Sep 29, 2026Haomin Qi, Xiangzhe Xu, Yiming Huang +2Software Engineering AgentsBenchmark Construction

    University of California, San Diego · Purdue University · National University of Singapore

  2. TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces

    Sep 27, 2026Dehai Min, Daoan Zhang, Yiming Zeng +13Benchmark ConstructionLLM Agent Evaluation

    ByteDance Inc., USA · University of Illinois at Chicago

  3. A comparative assessment of global building and settlement datasets across geographic and settlement contexts

    Sep 23, 2026Rufai Omowunmi Balogun, Caroline Margaux Gevaert, Capucine Riom +5Benchmark ConstructionBuilding Footprint Extraction

    The World Bank, Global Facility for Disaster Reduction and Recovery · Center for Geospatial Analytics, Clark University, Worcester, MA, USA · University of Twente, Enschede, Twente, The Netherlands +2

  4. The Endless Exam: Mathematical Constructions from Today's Models toward Superintelligence

    Sep 21, 2026Muhan ZhangMathematical Reasoning BenchmarksBenchmark Construction

    The University of Texas at Arlington

  5. Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts

    Sep 16, 2026Guojun Zhu, Xunheng Huang, Peng Yin +3Counterfactual EvaluationBenchmark Construction

    School of Mathematical Sciences, University of Chinese Academy of Sciences, China · Department of Statistics & Data Science, National University of Singapore, Singapore · Institute of Automation, Chinese Academy of Sciences, China

  6. Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation

    Sep 14, 2026M. Ali BayramMultilingual Language Model EvaluationLLM Evaluation

    Department of Computer Engineering, Yıldız Technical University İstanbul, Turkey

  7. Embodied-BenchForge: A Closed-Loop Agentic Workflow for Embodied Benchmark Construction

    Sep 14, 2026Baoyang Jiang, Fengchun Zhang, Leyuan Wang +9Benchmark ConstructionEmbodied QA

    QiYuan Lab · University of Electronic Science and Technology of China · Beijing University of Posts and Telecommunications +3

  8. Field-level prediction of mid-plane stress tensor fields in concrete target penetration: a cross-velocity graph neural operator surrogate

    Sep 9, 2026Wenpu Du, Peng Zhou, Yunlong Xia +5Neural Surrogate ModelingBenchmark Construction

    School of Environment and Safety Engineering, North University of China, Taiyuan, 030051, China

  9. A Large Open Multi-Energy Corpus of Soil Compaction Tests, with Machine-Learning Baselines

    Sep 3, 2026Sompote Youwai, Chana Phutthananon, Warat KongkitkulBenchmark ConstructionMaterials Property Prediction

    AI Research Group, Department of Civil Engineering King Mongkut’s University of Technology Thonburi 126 Pracha Uthit Road, Bang Mod, Thung Khru, Bangkok 10140, Thailand

  10. ToolGate: An Executable Acceptance Pipeline for Tool-Dependent Scientific Benchmark Construction

    Sep 2, 2026Ke Zhang, Yankang Liu, Roya Zandi +1Tool-Use EvaluationBenchmark Construction

    University of California, Riverside

  11. CADEngBench: It Looks Like CAD, but Does It Work? Evaluating Parametric Design, Assembly Reasoning, and Physics Simulation

    Aug 10, 2026Harmanjot Singh, Abhra Dubey, Jorge Alejandro Amador HerreraBenchmark ConstructionParametric CAD Modeling

    Mohamed Bin Zayed University of Artificial Intelligence · Abu Dhabi, UAE

  12. ArchEGraph: A Large-Scale Graph Dataset for Geometry-Topology-Physics Aligned Building Energy Modeling

    Aug 7, 2026Yihui Li, Yihui Chen, Kaidi Zha +7Benchmark ConstructionBuilding Energy Management

    Tsinghua University · UC Berkeley · 1Tsinghua University 2UC Berkeley 3MIT +4

  13. BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests

    Aug 3, 2026Zetong Xiong, Qiao Zhao, Jun Zhang +20Software Engineering AgentsBenchmark Construction

    Baidu Beijing, China

  14. Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms

    Aug 2, 2026Tezan Sahu, Aritra Das, Pankaj Mittal +1Benchmark ConstructionLLM Agent Evaluation

    Microsoft Hyderabad, India · Indian Institute of Technology Roorkee Roorkee, India

  15. Benchmarks Are Not Monolithic: Sample-Level Auditing and Orchestration for LLM Evaluation

    Jul 30, 2026Philipp D. Siedler, Jordan SassoonLLM EvaluationBenchmark Auditing

    Aleph Alpha Research, Heidelberg, Germany.

  16. SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining

    Jul 29, 2026Yi Cui, Zilin Wang, Yijie Xu +5Multimodal RobustnessBenchmark Construction

    Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · The Hong Kong University of Science and Technology(Guangzhou) Guangzhou, China · School of Management Northwest Polytechnical University Xi’an Xi’an, China +4

  17. HamQASBench: A Hamiltonian-Informed Diagnostic Benchmark for Evaluating Quantum Architecture Search

    Jul 6, 2026Jiayang Niu, Akib Karim, Yan Wang +5Benchmark ConstructionBenchmark Design

    School of Computing Technologies RMIT University · Quantum Systems School of Computing Technologies RMIT University · Data61, CSIRO

  18. MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models

    Jul 2, 2026Yuanzhi Liu, Shousheng Zhao, Bo Zhou +2VLM EvaluationBenchmark Construction

    1Beijing University of Posts and Telecommunications, Beijing, China

  19. EEG Benchmarking Needs a Task Specification Layer: NeuroDoc for Rulebook-Guided, Executable Benchmark Construction

    Jun 22, 2026Chengxuan Qin, Zhige Chen, Shu Peng +9ElectroencephalographyBenchmark Construction

    School of Advanced Technology, Xi’an Jiaotong-Liverpool University · School of Electrical Engineering, Electronics and Computer Science, University of Liverpool · Department of Data Science and Artificial Intelligence, Hong Kong Polytechnic University +1

  20. CFAgentBench: A Reproducible Environment and Benchmark for Autonomous Construction-Finance Agents

    Jun 20, 2026Rishi SrivastavaBenchmark ConstructionLLM Agent Evaluation

    Beiing Human

  21. CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage

    Jun 19, 2026Zhangwei Cao, Shuhan Fan, Yuting Wei +5Benchmark ConstructionReasoning Evaluation

    University of International Relations · 2Kedge Business School · 3Peking University +3

  22. Intelligent Automation for Embodied Benchmark Construction: Pipelines, Embodiments, Simulators, and Trends

    Jun 10, 2026Jinshan Lai, Jianwei Hu, Baoyang Jiang +7Benchmark ConstructionBenchmark Design

    University of Electronic Science and Technology of China, Chengdu, China · Qiyuan Lab, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China +2

  23. ClusBench: The Clustering Benchmark Data Resource You've All Been Waiting For (?)

    Jun 9, 2026David P. HofmeyrBenchmark ConstructionClustering

    School of Mathematical Sciences, Lancaster University

  24. Towards Event-Robust Acoustic Scene Classification

    Jun 5, 2026Yiqiang Cai, Bohan Hu, Yu Yang +3Benchmark ConstructionAudio Classification

    Xi’an Jiaotong-Liverpool University, Suzhou, China · Zhongdian Zhiheng Information Technology Service Co., Ltd, Nanjing, China · China Telecom Jiangsu Branch, Nanjing, China +1

  25. Brick-Composer: Using MLLMs for Assembly with Diverse Bricks

    Jun 3, 2026Jiateng Liu, Bingxuan Li, Zhenhailong Wang +8Visual Spatial ReasoningBenchmark Construction

    UIUC1 · Stevens Institute of Technology2 · Northwestern University3

  26. SC3: The Multi-Solvent Solubility Challenge and Benchmark

    Jun 3, 2026Vansh Ramani, Har Ashish Arora, Dhairya Kuchhal +4Benchmark ConstructionBenchmark Validity

    Indian Institute of Technology Delhi, India · Kurnakov Institute of General and Inorganic Chemistry RAS, Russia

  27. Position: Every Ground Truth is a Human Construction, not an Objective Truth

    May 28, 2026Charlotte Högberg, Ericka Johnson, Kiri L. WagstaffBenchmark Construction

    Lund University, Lund, Sweden · Linköping University, Linköping, Sweden · Oregon State University, Corvallis, OR, USA.

  28. A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

    May 27, 2026Tomer Keren, Nitay Calderon, Asaf Yehudai +3Computer-Use Agent BenchmarksTool-Use Evaluation

    Faculty of Data and Decision Science Technion · IBM Research

  29. ATLAS: All-round Testing of Long-context Abilities across Scales

    May 27, 2026Deli Huang, Cunguang Wang, Hongyin Tang +15LLM EvaluationBenchmark Construction

    Meituan · Fudan University

  30. Constraint acquisition needs better benchmarks

    May 25, 2026Rafał Stachowiak, Tomasz P. PawlakBenchmark ConstructionBenchmark Design

    Institute of Computing Science, Poznan Univeristy of Technology, Piotrowo 2, Poznań, Poland