Human-Annotated Benchmark

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

11 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Period ending 2026-09-14

38 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Period ending 2026-09-07

15 new papers

A weekly snapshot of new work published in Human-Annotated Benchmark.

Inside this field

Focused directions

Focused topic

Human-Annotated Benchmark

Within Human-Annotated Benchmark

85 papersExplore

Focused topic

Programbench

Within Human-Annotated Benchmark

42 papersExplore

Focused topic

Benchmark Suite

Within Human-Annotated Benchmark

31 papersExplore

Focused topic

First Comprehensive Benchmark

Within Human-Annotated Benchmark

30 papersExplore

Focused topic

Private Benchmark

Within Human-Annotated Benchmark

26 papersExplore

Focused topic

Physics Benchmarks

Within Human-Annotated Benchmark

20 papersExplore

233 papers

Latest in Human-Annotated Benchmark

Browse all topics →

Human-Centered Benchmarking of Driver Monitoring Models
Jun 6, 2026Ruben Dario Florez-ZelaMobilenetv2 Human-Annotated Benchmark
When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing
Jun 5, 2026Siyuan Xu, Yibing Liu, Peilin Chen +3Image Editing Multimodal Knowledge Editing
MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios
Jun 5, 2026Haitian Wang, Ruoxi Sun, Quantong Qiu +5Multimodal Embeddings Multimodal Memory
ShallowBench: Benchmarking Generative Drug Design Models on Shallow-Pocket Targets
Jun 4, 2026Saket Reddy, Shiwei LiuStructure-Based Drug Design Drug Discovery
Impostor: An Agent-Curated Benchmark for Realistic AIGC Manipulation Localization
Jun 3, 2026Zhenliang Li, Yutao Hu, Qixiong Wang +5Imprint Human-Annotated Benchmark
LCSHBench: A Multilingual, Consensus-Grounded Benchmark for Library of Congress Subject Heading Assignment
Jun 3, 2026Kwok Leong TangReal Datacite Dois Human-Annotated Benchmark
FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs
Jun 2, 2026Eshika Khandelwal, Jingjing Pan, Mingfang Zhang +3Multimodal Benchmarks Multi-Modal Large Language Models
HighTide: An Agent-Curated Open-Source VLSI Benchmark Suite
Jun 2, 2026Benjamin Goldblatt, Paolo Pedroso, Farhad Modaresi +2Vhdl Benchmark Suite
OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs
Jun 2, 2026Yifei Li, Pengyiang Liu, Yuhang Zang +4Embodied Spatial Intelligence Human-Annotated Benchmark
A Benchmark for Semi-supervised Multi-modal Crowd Counting
Jun 2, 2026Haoliang Meng, Xiaopeng Hong, Yabin Wang +1Semi-Supervised Learning Unlabeled Data
TSQAgent: Rating Time Series Data Quality via Dedicated Agentic Reasoning
Jun 2, 2026Shunyu Wu, Dan Li, Haozheng Ye +6Data Quality Agentic Reasoning
Any2Poster: Any-Source Poster Generation Across Modalities and Domains
Jun 1, 2026Amogh Vinaykumar, Aiden Li, Suozhi Huang +1Human-Annotated Benchmark Human-Agent Collaborative Paper-To-Page
RescueBench: Can Embodied Agents Save Lives in the Wild ?
Jun 1, 2026Kui Wu, Beiyu Guo, Hao Chen +6Embodied Agents Agentic Benchmarks
"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise
Jun 1, 2026Matthew Khoriaty, David Williams-King, Shi FengDiversity Surprisal
DEMM-Bench: A Cross-Regime Benchmark for Agent-Runtime Governance-Evidence Sufficiency
May 30, 2026Oleg SolozobovContent Provenance Agent Decisions
Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration
May 29, 2026Jun Wang, Xiaohao Xu, Xiaonan HuangHuman-Robot Collaboration Explicit Visual Grounding
Office Comprehension Benchmark
May 29, 2026Firoz Shaik, Mateus Picanço Lima Gomes, Tanvir Aumi +17First Comprehensive Benchmark Large Language
TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation
May 29, 2026Yeil Jeong, Youngjin Yoo, Jiyoung Bae +5Human-Annotated Benchmark Model Evaluation
LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis
May 28, 2026Kewei Xu, Xiaoben Lu, Shuofei Qiao +4Long-Horizon Longitudinal Data
Architecture-Sensitive Supervised Fine-Tuning for Screen-Conditioned Action Prediction: A PiSAR Benchmark
May 28, 2026Rahul Bissa, Abhishek Vyas, Yash JainSupervised Fine-Tuning Fine-Grained Actions
Continual Model Routing in Evolving Model Hubs
May 27, 2026Jack Bell, Giacomo Carfì, Gerlando Gramaglia +1Efficient Continual Learning Independent Anchor Network
SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agents
May 27, 2026Yubin Qu, Yi Liu, Gelei Deng +4Coding Agents Benchmark Suite
Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation
May 26, 2026Unggi Lee, Hoyoung Ahn, Yoon Choi +16Longvideobench Pedagogy
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
May 26, 2026Hwiwon Lee, Jiawei Liu, Dongjun Kim +3Vulnerability Detection & Repair Large Language Model Safety
Anchor: Mitigating Artifact Drift in Agent Benchmark Generation
May 25, 2026Maksim Ivanov, Abhijay RanaArtificial Intelligence Agents Workflow Generation
Deployment-complete benchmarking
May 25, 2026El Mustapha Mansouri, Keigo AraiBenchmark Suite Algorithm Auditing
RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges
May 24, 2026Ruize Li, Zhibin Wen, Tao Han +5Artificial Intelligence Weather Models Numerical Weather Prediction
GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration
May 23, 2026Junjie Zhao, Jingyi Liang, Zhenyang Cai +22Clinical Reasoning Benchmarking Large Language Models
AstroMind: A High-Fidelity Benchmark for Spacecraft Behavior Reasoning Based on Large Language Models
May 23, 2026Hao Liu, Siyuan Yang, Qinglei Hu +1Mars Human-Annotated Benchmark
What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA
May 21, 2026Xinjie He, Zhiyuan Lin, Su Liu +4Deep Reinforcement Learning Curriculum
From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding
May 21, 2026Yandi Wang, Libin Zhan, Ziwei Huang +5Visual Document Retrieval Human-Annotated Benchmark
SGR-Bench: Benchmarking Search Agents on State-Gated Retrieval
May 21, 2026Ningyuan Li, Haiyang Shen, Mugeng Liu +4Search Agents Agentic Benchmarks
SDGBiasBench: Benchmarking and Mitigating Vision--Language Models' Biases in Sustainable Development Goals
May 21, 2026Zihang Lin, Huaiyuan Qin, Muli Yang +1Large Vision Language Models Large Language Model Bias
Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?
May 19, 2026Animesh Maheshwari, Divyansh Sahu, Nishit Verma3D Spatial Reasoning Spatial Reasoning
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
May 19, 2026Yujie Wei, Yujin Han, Zhekai Chen +20Joint Audio-Video Generation Evaluation Benchmarks
FineBench: Benchmarking and Enhancing Vision-Language Models for Fine-grained Human Activity Understanding
May 19, 2026Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh +2Fine-Grained Video Understanding Video Question Answering
EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design
May 19, 2026Gioele Molinari, Florian Felten, Soheyl Massoudi +1Engineering Multi-Agent Systems
CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision
May 19, 2026Pengcheng Wang, Haoxiang Liu, Yang Dai +5Captcha-Capable Native Graphical User Interface Agent Visual Reasoning
HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models
May 19, 2026Emmy Liu, Varun Gangal, Michael Yu +4Hallucinations Human-Annotated Benchmark
ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents
May 18, 2026Jiawei He, Jie Jia, Chenbo Liu +4Coding Agents Programbench
TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents
May 16, 2026Zhiqiang Liu, Wenhui Dong, Yilang Tan +3Tool Omni-Modal
GPF-LiveNews: A Streaming Evaluation Protocol for Group-Conditioned Framing in Large Language Models
May 16, 2026Mohd Ariful Haque, Fahad Rahman, Kishor Datta Gupta +1Large Language Model Bias Large Language Model Evaluation
MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models
May 15, 2026Weixin Liu, Congning Ni, Shelagh A. Mulvaney +4Benchmarking Large Language Models Mental Health
The Evaluation Trap: Benchmark Design as Theoretical Commitment
May 13, 2026Theodore J KalaitzidisArtificial Intelligence Benchmarks Benchmark Suite
RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning
May 13, 2026Andrea MorandiLlm-As-A-Judge Judges
KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models
May 13, 2026Richard Sproat, Stefano PeluchettiMultimodal Benchmarks Textual Dataset
GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language
May 13, 2026Jinwoong Kim, Rui Yang, Huishuai ZhangRepresentational Geometry Geoquery
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
May 12, 2026Hao Wang, Hanchen Li, Qiuyang Mang +3Programbench Adversaries
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender
May 12, 2026Leonor Veloso, Hinrich SchützeGender Bias Gender
MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling
May 12, 2026Zhong Li, Qi Huang, Yuxuan Zhu +6Llm-Based Optimization Modeling Multi-Modal Large Language Models
Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
May 12, 2026Kyosuke Takami, Yuka Tateisi, Satoshi Sekine +1Multimodal Benchmarks Computerized Adaptive Testing
GeoR-Bench: Evaluating Geoscience Visual Reasoning
May 12, 2026Yushuo Zheng, Zicheng Zhang, Huiyu Duan +7Geological Semantic Understanding Geoquery
On the Impact of Crossover in Many-Objective Optimization: A Runtime Analysis of NSGA-III
May 11, 2026Andre OprisMulti-Objective Optimization Evolutionary Search
The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies
May 11, 2026Gabriel GarciaChain-of-Thought Reasoning Correctness
PlantMarkerBench: A Multi-Species Benchmark for Evidence-Grounded Plant Marker Reasoning
May 11, 2026Sajib Acharjee Dip, Song Li, Liqing ZhangBiomedical Text Human-Annotated Benchmark
MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding
May 10, 2026Xiaoyu Yuan, Niklas Heikkala, Tiina Törmänen +3Theory-Of-Mind Reasoning Multi-Agent Reasoning
MedMeta: A Benchmark for LLMs in Synthesizing Meta-Analysis Conclusion from Medical Studies
May 10, 2026Huy Hoang Ha, Benoit Favre, Francois PortetSystematic Review Clinical Reasoning
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5Several Approximate Unlearning Strategies Multi-Modal Large Language Models
SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation
May 8, 2026Tianfei Ren, Zhipeng Yan, Yiming Zhao +13Image Generation Text-To-Image
Text-to-CAD Evaluation with CADTests
May 8, 2026Dimitrios Mallis, Marco Wang, Ahmet Serdar Karadeniz +3Text-To-Cad Computer-Aided Design

Jun 6, 2026 · cs.CVJ/K move · Enter open · S save

Human-Centered Benchmarking of Driver Monitoring Models

Ruben Dario Florez-Zela

Universidad Nacional de San Agustin de Arequipa (UNSA) Arequipa, Peru

Vision-based driver monitoring systems are increasingly deployed in safety-critical intelligent transportation settings, yet they are almost always compared on classification accuracy alone. This paper argues that accuracy is insufficient to characterize a model's fitness for real-world deployment, and proposes the Human-Centered Benchmarking Framework (HCBF), which evaluates models across four dimensions: accuracy, explainability, efficiency, and robustness. The framework is applied to four representative lightweight architectures, MobileNetV3, ShuffleNetV2, EfficientNet-B0, and DeiT-Tiny, on the MRL Eye Dataset for eye-state classification. While the models are nearly indistinguishable on clean-set accuracy, each leads in exactly one dimension, and all four lie on the Pareto frontier. A Human-Centered Score computed under three deployment-oriented weighting scenarios ranks ShuffleNetV2 first throughout. However, this aggregate winner retains less than half of its performance under sensor noise and fails by classifying closed eyes as open, whereas the transformer remains robust. These findings show that aggregate ranking can mask dimension-specific vulnerabilities that are operationally decisive, underscoring the value of multi-dimensional, human-centered evaluation.

Mobilenetv2 Human-Annotated Benchmark

Open paper Ask about it PDF