Human-Annotated Benchmark

Ask about this topic

Momentum

18 papers in the last four weeks, up 125% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 87

All topics
CardsList

  1. PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data

    Oct 5, 2026Yaohui Zhang, Binxu Li, Haoyi Duan +7Human-Annotated BenchmarkQuantifying

    Generative Expert Labs, Inc · Stanford University · Princeton University

  2. GNN-CB: A Graph Neural Network Competition Benchmark for Human and LLM Evaluation

    Oct 4, 2026Murad Hossen, Tasneem Selim, Gurur Gamgam +22Human-Annotated Benchmark

    University of Houston, Texas,USA · BASIRA Lab, Department of Computing, Imperial College London, United Kingdom · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt +23

  3. A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models

    Oct 1, 2026Yingzhu Zhao, Vlad Pandelea, Han Yuan +4Financial Question AnsweringHuman-Annotated Benchmark

    Global Decision Science, American Express · Singapore Decision Science Center of Excellence, American Express, 1 Marina Boulevard, 018989, Singapore.

  4. R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing

    Sep 30, 2026Xin Wang, Zichuan Ying, Xinna Lin +8Chemical StructuresMoleculenet

    Westlake University · The University of Hong Kong · Shanghai Innovation Institute +5

  5. EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses

    Sep 30, 2026Zhixuan Tan, Pengjie Gu, Zhao Li +4Coding AgentsAgentic Benchmarks

    The Chinese University of Hong Kong, Shenzhen · MemoraX AI

  6. OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models

    Sep 28, 2026Hao Wang, Tao Yu, Liuzhou Zhang +13Video World ModelsMultimodal Memory

    USTC · Infrec · CASIA +10

  7. MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

    Sep 23, 2026Yongjun Jeong, Hanbum Ko, Ye Rin Kim +6Molecular DesignHuman-Annotated Benchmark

    Department of Artificial Intelligence, Korea University · LG AI Research · Department of Statistics, Korea University

  8. GroundedGEO: Auditing the Evidence Gap in Generative Search Rankings

    Sep 21, 2026Yihan Xia, Huiling Fan, Kangrong Zhong +1Evidence-Grounded QuestionsEvidence Conflict

    Shenzhen University Shenzhen, China

  9. ReSTI: A Source-Grounded Audit and Repair of STI-Bench

    Sep 21, 2026Pengzhan Sun, Ramanathan Rajaraman, Shiu-Hong Kao +2Stable Spatial UnderstandingHuman-Annotated Benchmark

    National University of Singapore · University of Science and Technology of China

  10. Mind the Gaps: A Curated Benchmark for Form Field Detection

    Sep 20, 2026Iheb Brini, Omar Moured, Hamza Gbada +1Field EvaluationHuman-Annotated Benchmark

    Luleå University of Technology, Luleå, Sweden · Karlsruhe Institute of Technology, Karlsruhe, Germany · LATIS-Laboratory of Advanced Technology and Intelligent Systems, National Engineering School of Sousse (ENISo), University of Sousse, Tunisia

  11. Omni2Web: Benchmarking Audiovisual Website Development

    Sep 20, 2026Minghao Han, Zhenghao Xing, Xize Cheng +7OmniHuman-Annotated Benchmark

    FDU · Alibaba Token Hub, Alibaba Group · CUHK +2

  12. DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering

    Sep 17, 2026Luca De Grandis, Silvia Cappelletti, William Raccagni +3Human-Annotated BenchmarkDocument

    University of Modena and Reggio Emilia, Modena, Italy · University of Modena and Reggio Emilia Modena, Italy · University of Pisa, Pisa, Italy +1

  13. TeochewBench: A Human-Reviewed Benchmark for Teochew Hanzi Translation

    Sep 16, 2026Jianan WuMachine Translation QualityMultilingual Benchmark

    Peking University

  14. Map2Route: Benchmarking Compositional Language-Grounded Route Planning over Semantic Maps

    Sep 15, 2026Muyi Bao, Hang Xu, Jingfan Tang +5Semantic MappingHuman-Annotated Benchmark

    Carnegie Mellon University, Pittsburgh, PA 15213, USA · Nanyang Technological University, Singapore 639798

  15. KnowBench: Effort Reduction as a Unified, Deployment-Grounded Benchmark for Clinical AI

    Sep 14, 2026Jocelyn Kang, Caroline ZhangClinician TrustClinical Notes

    Knowtex Inc.

  16. MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

    Sep 14, 2026Hasan Iqbal, Sarfraz Ahmad, Hyunjae Kim +5FactsHuman-Annotated Benchmark

    Mohamed bin Zayed University of Artificial Intelligence · Yale University · Korea University +2

  17. NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment

    Sep 12, 2026Guoqiang Zhang, Kexin Tan, Ming Zhang +12Human-Annotated BenchmarkHuman Judgment

    College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Graduate School of Engineering, The University of Tokyo, Tokyo, Japan · AtomInnoLab 4Atom Infinite Pte. Ltd., Singapore +1

  18. ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

    Sep 12, 2026Yizhan Li, Jianxin You, Mengyang Xiong +5Embodied Artificial IntelligenceHuman-Annotated Benchmark

    Université de Montréal · Mila – Quebec AI Institute · McGill University +2

  19. SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code

    Sep 8, 2026Wentao Li, Yibo Wu, Yizhe Chen +5Scientific FigureImage-To-Code Generation

    Tsinghua University

  20. TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents

    Sep 4, 2026Zhibo Yang, Chen Zhang, Yuewei Zhang +1Artificial Intelligence ScientistsHuman-Annotated Benchmark

    TruthInsight-AI

  21. GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

    Sep 3, 2026Linh Le, Melanie Bui, My Chiffon Nguyen +2Policy EvaluationMulti-Agent Simulations

    1Lida Safety · 2ERA

  22. Can LLMs Discover Scientific Laws in Real and Parallel Worlds?

    Sep 1, 2026Yiming Huang, Ziche Liu, Junxia Cui +11Scientific DiscoveryHuman-Annotated Benchmark

    University of California San Diego · University of California, Irvine · Cushing Academy +1

  23. SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos

    Aug 31, 2026Zheyu Huang, Zijing Shi, Haozhe Luo +4Social ReasoningReasoning Benchmark

    AAII, University of Technology Sydney · Northeastern University · Fudan University +1

  24. HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

    Aug 13, 2026Dairu Liu, Zekun Qi, Jiayu Zeng +11Human MotionMotion Imitation

    1Nankai University · 3Galbot · 2Tsinghua University +3

  25. BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavarian

    Aug 13, 2026Jophin John, Michael Hoffmann, Jan Fillies +2Multilingual BenchmarkDialects

    Leibniz Supercomputing Centre (LRZ), Garching, Germany · Stanford University, Stanford, California, USA · Freie Universität Berlin, Germany +2

  26. SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information

    Aug 11, 2026Junjie Ye, Zhuohui Sheng, Shaofan Liu +12Human-Annotated BenchmarkDevice

    1Fudan University · 2Tencent Hunyuan Team

  27. ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering

    Aug 11, 2026Akrin Zheng, Alexander Wu, Alaia LiuKnowledge GraphsHuman-Annotated Benchmark

  28. MRBench: A Comprehensive Benchmark for Human Motion-Text Retrieval

    Aug 8, 2026Fulong Liu, Liang Xu, Chengqun Yang +3Motion-Language AlignmentHuman-Annotated Benchmark

    Shanghai Jiao Tong University · Beijing Zhongguancun Academy

  29. FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings

    Aug 7, 2026Sasan Mansouri, Daniel Saad, Mark Wahrenburg +2Financial Question AnsweringHuman-Annotated Benchmark

    University of Groningen, Groningen, Netherlands · Goethe University Frankfurt, Frankfurt am Main, Germany · DataNXT GmbH, Frankfurt am Main, Germany +2

  30. Recipes for Creativity: Iterative Generation and Evaluation in Large Language Models

    Aug 7, 2026Rens Anderson, Tessa Verhoef, Amirhossein ZohrehvandCreativityLarge Language Model Evaluation

    LIACS Leiden University Einsteinweg 55 2333 CC Leiden The Netherlands

  31. TRACE: A Multi-Layer Benchmark for Human AI Controller Coordination Under Drift and Failure

    Aug 7, 2026Joshua Zuniga, Srinivasan Subramanian, Ramya Madhuri Narapureddy +1TracesArtificial Intelligence Control

    College of Computing and Software Engineering Kennesaw State University Marietta, GA, USA

  32. GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models

    Aug 6, 2026Shuai Wang, Yaxin Feng, Xuekun Jiang +12Physics SimulationVideo World Models

    1Shanghai Artificial Intelligence Laboratory · 2Hong Kong University of Science and Technology · 3Shanghai Jiao Tong University +1

  33. M3^3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

    Aug 6, 2026Hong Jiang, Junnan Zhu, Jingwang Huang +9MetaphorsHuman-Annotated Benchmark

    School of Computer Science, Chongqing University, Chongqing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore +3

  34. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks

    Aug 5, 2026Pau Arnal, Khaled Denfir, Danylo Smahliuk +2Expert DemonstrationsHuman-Annotated Benchmark

    Sovrano AI

  35. FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation

    Aug 5, 2026Yinghao Tang, Tan Zhenwei, Yiyao Wang +4ReportModel Auditing

    State Key Laboratory of CAD&CG, Zhejiang University · Ant Group

  36. COMEX: A Composition-Grounded Benchmark and Learning Framework for Explainable Aesthetic Image Cropping

    Aug 4, 2026Rui Yang, Wei Zhou, Dingyong Gou +5CroppingMulti-Image Editing Benchmarks

    State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation · School of Data Science and Institute of Artificial Intelligence, Chang’an University

  37. CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Models

    Aug 2, 2026Xiaocui Yang, Xican Tan, Shoujie Chen +3Legal Reasoning TasksHuman-Annotated Benchmark

    School of Computer Science and Engineering, Northeastern University, Shenyang, China · Institute of Intelligent Computing, University of Electronic Science and Technology of China, Chengdu, China · School of Civil and Commercial Law, Southwest University of Political Science and Law, Chongqing, China +1

  38. MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents

    Jul 26, 2026Gengyang Xu, Dongwei Xiao, Yiteng Peng +1Embodied AgentsHuman-Annotated Benchmark

    Hong Kong University of Science and Technology, China

  39. GPE: Evaluating Robust Evidence Aggregation for Fact Verification under Controllable GEO-Style Poisoning

    Jul 22, 2026Zhaoqi Wang, Zijian Zhang, Xiaomei Yuan +4Fact-CheckingEvidence-Grounded Questions

    School of Cyberspace Science and Technology, Beijing Institute of Technology · School of Computer Science, The University of Auckland

  40. Benchmarking Human and Automatic Speech Recognition of Diverse Speech: Initial Results

    Jul 21, 2026Ilse Huisman, Rares Popa, Yuanyuan Zhang +1Automatic Speech Recognition EvaluationAutomatic Speech Recognition

    Multimedia Computing Group Delft University of Technology Delft, the Netherlands

  41. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning

    Jul 17, 2026Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2Artificial Intelligence BenchmarksHuman-Annotated Benchmark

    Operations, Information and Decisions Department, The Wharton School, University of Pennsylvania, Philadelphia, 19104, PA, USA · Heinz College of Information Systems and Public Policy, Carnegie Mellon University, Pittsburgh, 15213, PA, USA · Department of Computer and Information Science, University of Pennsylvania, Philadelphia, 19104, PA, USA +1

  42. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    Jul 16, 2026Saima Afrin, Alessandro Midolo, Camilo Escobar-Velásquez +5Code GenerationMultilingual Benchmark

    William & Mary, VA, USA · University of Catania, Italy · Universidad de los Andes, Colombia +2

  43. Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging

    Jul 11, 2026Jinglan Gong, Jiefan Lu, Hewei Guo +5Dialogue BenchmarksUser Simulation

    SenseTime Research · University of Science and Technology of China · Tsinghua University

  44. Evaluating Neural Decompilation of Dart AOT Binaries: Fine-Tuning, Metric Validity, Specification Leakage, and Reliability

    Jul 7, 2026Raafat Abualazm, Ayman AboElhassan, Amr G. WassalModel Fine-TuningHuman-Annotated Benchmark

    Cairo University, Computer Engineering Department, Faculty of Engineering, Giza, Egypt

  45. Overview of the TalentCLEF 2026: Skill and Job Title Intelligence for Human Capital Management

    Jun 30, 2026Luis Gasco, Hermenegildo Fabregat, Laura García-Sardiña +8JobsSkills

    Avature Machine Learning, Spain · TechWolf, Belgium · NLP & IR Group at UNED, Madrid, Spain

  46. A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models

    Jun 27, 2026Nuo Chen, Lulin Liu, Zihao Li +12Physics SimulationCollision Prediction

    Texas A&M University · University of Minnesota · Marquette University +6

  47. HG-Bench: A Benchmark for Multi-Page Handwritten Answer-Region Grounding in Automated Homework Assessment

    Jun 24, 2026Chuangxin Zhao, Boyan Shi, Yanling Wang +7Human-Annotated BenchmarkContextual Grounding

  48. AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning

    Jun 23, 2026Honglin Guo, Qi Zhang, Yu Zhang +7Human-Annotated BenchmarkTool-Grounded Reasoning

    Fudan University · Zhejiang University · Shanghai Qiji Zhifeng Co., Ltd.

  49. Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

    Jun 15, 2026Xingyu Tan, Shiyuan Liu, Xiaoyang Wang +5Multi-Hop ReasoningHuman-Annotated Benchmark

    University of New South Wales, Australia · 2CSIRO, Australia · University of Technology Sydney, Australia

  50. Human-Centered Benchmarking of Driver Monitoring Models

    Jun 6, 2026Ruben Dario Florez-ZelaMobilenetv2Human-Annotated Benchmark

    Universidad Nacional de San Agustin de Arequipa (UNSA) Arequipa, Peru

  51. Impostor: An Agent-Curated Benchmark for Realistic AIGC Manipulation Localization

    Jun 3, 2026Zhenliang Li, Yutao Hu, Qixiong Wang +5Human-Annotated BenchmarkLocalization

    Southeast University · Xiaohongshu Inc · Xiaohongshu Inc. +1

  52. LCSHBench: A Multilingual, Consensus-Grounded Benchmark for Library of Congress Subject Heading Assignment

    Jun 3, 2026Kwok Leong TangCatalogsHuman-Annotated Benchmark

  53. OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs

    Jun 2, 2026Yifei Li, Pengyiang Liu, Yuhang Zang +4Stable Spatial UnderstandingMultimodal Large Language Models

    Tsinghua University · Shanghai AI Laboratory · Beihang University

  54. InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark

    Jun 1, 2026Shiyu Wang, Ziyu Liu, Chaoyi Yu +6Emotion RecognitionHuman-Annotated Benchmark

    East China Normal University, Shanghai, China

  55. "I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise

    Jun 1, 2026Matthew Khoriaty, David Williams-King, Shi FengDiversityCreativity

    ERA Fellowship · George Washington University

  56. Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration

    May 29, 2026Jun Wang, Xiaohao Xu, Xiaonan HuangHuman-Robot CollaborationHuman-Robot Interaction

    University of Michigan, Ann Arbor

  57. Position: Every Ground Truth is a Human Construction, not an Objective Truth

    May 28, 2026Charlotte Högberg, Ericka Johnson, Kiri L. WagstaffGround TruthHuman-Annotated Benchmark

    Lund University, Lund, Sweden · Linköping University, Linköping, Sweden · Oregon State University, Corvallis, OR, USA.

  58. AstroMind: A High-Fidelity Benchmark for Spacecraft Behavior Reasoning Based on Large Language Models

    May 23, 2026Hao Liu, Siyuan Yang, Qinglei Hu +1Human-Annotated BenchmarkHigh-Fidelity

    Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · KTH Royal Institute of Technology, Stockholm, Sweden · School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China +1

  59. From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding

    May 21, 2026Yandi Wang, Libin Zhan, Ziwei Huang +5Visual Document RetrievalInformation Extraction

    Zhejiang University, China

  60. Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?

    May 19, 2026Animesh Maheshwari, Divyansh Sahu, Nishit Verma3D Spatial ReasoningDiverse Occlusion-And-Revelation Scenarios

    Deccan AI