Human-Annotated Benchmark

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

11 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Period ending 2026-09-14

38 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Period ending 2026-09-07

15 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Inside this field

Focused directions

233 papers

Latest in Human-Annotated Benchmark

Browse all topics →
CardsList

  1. InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk

    Jul 31, 2026Yuan Gao, Zeren Yang, Junnan Li +6InfrastructureArtificial Intelligence Agents

  2. LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data

    Jul 30, 2026Debopam Sanyal, Hongjie Chen, Alexey Tumanov +1MediaFirst Comprehensive Benchmark

  3. DataClawEval: A Benchmark for Data Engineering Agents in Real Industrial Harness

    Jul 30, 2026Debin Meng, Jiaming Yang, Zefang Zong +4Data LakesFirst Comprehensive Benchmark

  4. PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology

    Jul 29, 2026Xiaohan Li, Xinyu Liu, Chang Liu +4Diagnostic BenchmarkMedical Vision-Language Models

  5. MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

    Jul 29, 2026Yihao Chen, Shi Chang, Khaled Chawa +4Mock-Interface SynthesisForge

  6. SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining

    Jul 29, 2026Yi Cui, Zilin Wang, Yijie Xu +5Safety EvaluationHazard

  7. When benchmark inferences do not compose: Projectibility in AI evaluation

    Jul 28, 2026Brett ReynoldsArtificial Intelligence EvaluationBenchmark Suite

  8. MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents

    Jul 26, 2026Gengyang Xu, Dongwei Xiao, Yiteng Peng +1Embodied AgentsEmbodied Spatial Intelligence

  9. ParasGB: A Graph Benchmark Suite for Parasitic Estimation on AMS Circuits

    Jul 25, 2026Jiajun Zou, Jiawei Liu, Ao Liu +8And Mixed-Signal CircuitsCircuits

  10. NL2SHACL-Bench: A Benchmark Suite for Natural Language to SHACL Translation

    Jul 24, 2026Yuchen Zhou, Niels Bobet, Maribel AcostaNatural Language QueriesKnowledge Graphs

  11. Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

    Jul 23, 2026Xu Wang, Kaixiang Yao, Miao Pan +4Embodied Spatial IntelligenceVisual Generation

  12. KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding

    Jul 22, 2026Zeyu Liu, Zhangzhe Zhu, Yang Zhang +3Pose-Guided Kinematic ReasoningVideo World Models

  13. Benchmarking Human and Automatic Speech Recognition of Diverse Speech: Initial Results

    Jul 21, 2026Ilse Huisman, Rares Popa, Yuanyuan Zhang +1Human-Annotated BenchmarkLuxembourgish

  14. An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation

    Jul 19, 2026Zhanbo Li, Shifeng Wu, Xiangjin Meng +1OntologyCounterfactual Reasoning

  15. Can Multimodal Large Language Models Understand OCT?

    Jul 18, 2026Baochen Fu, Wenzhi Deng, Baihao Jin +5Optical Coherence TomographyMultimodal Benchmarks

  16. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning

    Jul 17, 2026Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2Artificial Intelligence BenchmarksHuman-Annotated Benchmark

  17. Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Data

    Jul 17, 2026Kiwan Kwon, Kangmin Kim, Hojin Lee +5Synthetic Tabular DataBenchmark Suite

  18. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    Jul 16, 2026Saima Afrin, Alessandro Midolo, Camilo Escobar-Velásquez +5Code GenerationMultilingual Benchmark

  19. When Directional Accuracy Lies: A Base-Rate-Honest Benchmark for LoRA-Adapted TimesFM on Equity Forecasting

    Jul 14, 2026Taizhen CheungTime Series Foundation ModelsFinance Benchmarks

  20. AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification

    Jul 13, 2026Lingkai Kong, Zijian Wu, Yuzhe Gu +10Mathematical ReasoningReasoning Benchmark

  21. AutoMatBench: An Automatic Optimization Toolkit for the Acceleration of Material Properties Prediction Benchmarking

    Jul 13, 2026Hongxiao Li, Wanling GaoData-Driven Materials DiscoveryMle-Bench Lite

  22. Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging

    Jul 11, 2026Jinglan Gong, Jiefan Lu, Hewei Guo +5Human-Annotated BenchmarkUser Simulation

  23. UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks

    Jul 9, 2026Zhekai Chen, Chengqi Duan, Kaiyue Sun +4Agentic BenchmarksReal-World Tasks

  24. Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

    Jul 9, 2026Matteo Santelmo, Xiuying Wei, Israa Fakih +5Artificial Intelligence ModelsBlind Spots

  25. Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics

    Jul 7, 2026Pavel Snopov, German MagaiPrivate BenchmarkLarge Language Model Agents

  26. PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation

    Jul 7, 2026Yuhang Wu, Shuxiang Zhang, Wee Hian Ching +2Text-To-ImageText Generation

  27. AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Ranking

    Jul 7, 2026Zhiyuan Chen, Jing Hu, Junzhe Wang +4AntibodyStructure-Based Drug Design

  28. HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation

    Jul 5, 2026Yaozu Wu, Wei-Chieh Huang, Jizhou Guo +11Human OversightLlm-Based Agent

  29. Robust Feasible Route Construction through Collaborative Partition Optimization

    Jul 4, 2026Oguzhan Karaahmetoglu, Hyong KimVehicle Routing ProblemConstruction

  30. SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

    Jul 3, 2026Huajun Bai, Weiwei Lv, Huichuan Zheng +2Self-SpeculativeSpeculative Decoding

  31. PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

    Jul 3, 2026Tianyi Yang, Dawei Fu, Youpeng Wu +5Human-Annotated BenchmarkScientific Discovery

  32. PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

    Jul 3, 2026Xianren Zhang, Delvin Ce Zhang, Dongwon Lee +1Large Language Model UnlearningEfficient Privacy Loss

  33. Prior Bias in Vision Language Models on UML Diagram Interpretation

    Jul 3, 2026Zaiyu Cheng, Khai-Nguyen Nguyen, Antonio MastropaoloDiagramsLarge Vision Language Models

  34. K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

    Jul 2, 2026Khush Attarde, Yusuf Ali, Megha Thukral +3Video DatasetMultimodal Reasoning

  35. A2^{2}utoLPBench: An Auto-Generated, Agent-Friendly LP Benchmark via Inverse-KKT Construction

    Jul 2, 2026Shuo Ren, Yaohui Han, Yifan Shi +6Physics BenchmarksLlm-Based Agent

  36. PACE: A Proxy for Agentic Capability Evaluation

    Jul 2, 2026Yueqi Song, Lintang Sutawika, Jiarui Liu +8Agentic BenchmarksAgentic Evaluations

  37. Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds

    Jun 30, 2026Dong ZhangLlm-As-A-JudgePhysics Benchmarks

  38. CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes

    Jun 30, 2026Yuchen Huang, Xiang Li, Zhenqing Ling +5RefinementProgrambench

  39. Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation

    Jun 30, 2026Dong Yeong Kim, JunGyu Lee, Jaewon Choi +5Spatio-Temporal CoherenceUltrasound

  40. HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents

    Jun 30, 2026Qianchu Liu, Sheng Zhang, Guanghui Qin +16Opti-Agent-BenchBenchmark Suite

  41. A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models

    Jun 27, 2026Nuo Chen, Lulin Liu, Zihao Li +12Physics SimulationWorld Models

  42. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerPartial Differential Equation BenchmarksMosaic

  43. HG-Bench: A Benchmark for Multi-Page Handwritten Answer-Region Grounding in Automated Homework Assessment

    Jun 24, 2026Chuangxin Zhao, Boyan Shi, Yanling Wang +7Human-Annotated BenchmarkComputerized Adaptive Testing

  44. AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning

    Jun 23, 2026Honglin Guo, Qi Zhang, Yu Zhang +7Multi-Hop Question Answering BenchmarksHuman-Annotated Benchmark

  45. HOLMES: Evaluating Higher-Order Logical Reasoning in LLMs

    Jun 22, 2026Yucheng Wu, Jundong Xu, Mingzhen Ju +4First Comprehensive Benchmark

  46. HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions

    Jun 22, 2026Yuichi TatenoInformation Retrieval MetricsProgrambench

  47. How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures

    Jun 20, 2026Abhijit Sinha, Hemant Kumar Kathania, Mohit Joshi +3Self-Supervised Speech ModelsWav2Vec

  48. RedactionBench

    Jun 17, 2026Sean Brynjólfsson, Shashvat Jayakrishnan, Esha Sali +2Human-Annotated BenchmarkNamed-Entity Recognition

  49. MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents

    Jun 15, 2026Lawrence Keunho Jang, Andrew Keunwoo Jang, Jing Yu Koh +1Computer-Use AgentsArtificial Intelligence Assistants

  50. Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

    Jun 15, 2026Xingyu Tan, Shiyuan Liu, Xiaoyang Wang +5Biomedical Knowledge GraphMulti-Hop Reasoning

  51. Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages

    Jun 14, 2026Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar +1Audio Deepfake DetectionNeural Audio Codecs

  52. HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning

    Jun 13, 2026Thinh T. H. Nguyen, Le-Tuan Nguyen, Minh-Duong Nguyen +4Efficient Continual LearningContinual Learning Methods

  53. Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models

    Jun 12, 2026Rituraj Sharma, Tu VuHidden StatesCross-Entropy Losses

  54. BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems

    Jun 12, 2026Leonhard Waibl, Felix Michalak, Hadrien MariacciaJailbreak DetectionFirst Comprehensive Benchmark

  55. LessonBench-V1: A Benchmark Dataset for Evaluating AI Lesson Generation Agents

    Jun 12, 2026Ravidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle +2TutorsHuman-Annotated Benchmark

  56. OR-Action: Multi-Role Video Understanding with Fine-Grained Actions

    Jun 11, 2026Felix Tristram, Ege Özsoy, Christian Benz +3Open-Vocabulary Action RecognitionFine-Grained Actions

  57. Prefill Awareness in Large Language Models

    Jun 10, 2026Andy Wang, Parv Mahajan, David Demitri Africa +3Large Language Model SafetyFrontier Models

  58. Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction

    Jun 10, 2026Baoyang Jiang, Fengchun Zhang, Leyuan Wang +7Embodied Spatial IntelligenceEmbodied Agents

  59. MPC-Patch-Bench: Security-Aware LLM Code Patch for Multi-Party Computation

    Jun 9, 2026Yukuan Zhang, Mengxin Zheng, Qian LouLarge Language Model SafetyProgrambench

  60. SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks

    Jun 8, 2026Hongcheng Gao, Hailong Qu, Jingyi Tang +18Spatial ReasoningMultimodal Agents