cs.AIJul 5, 2026

HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation

Authors: Yaozu WuWei-Chieh HuangJizhou GuoDongyuan LiRenhe JiangHenry Peng ZouChunyu MiaoShanghao Li+6 more

Organizations: The University of Tokyo · University of Illinois Chicago · MBZUAI · McGill University · Zhejiang University

Abstract

Large language models increasingly operate in settings where humans are active collaborators rather than passive task providers. We introduce HAS-Framework, a graph-based framework that represents humans and LLM-powered agents as first-class participants with explicit roles, permissions, communication paths, and action authority. Building on this framework, HAS-Bench evaluates Human-Agent Systems under configurable human participation across agency levels, interaction channels, and persona policies. The benchmark measures both task outcomes and process-level collaboration behavior, including clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost. Experiments across six domains show that human participation can substantially improve task completion and failure recovery, but the gains depend on when, how, and by whom human input is exercised.

Explore similar work

CardsList
  1. HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants

    Sep 10, 2025Benjamin Sturgeon, Daniel Samuelson, Jacob Haimes +1