Do we have the knowledge we need? Rethinking human-AI decision-making in corporations
Authors: Anne S. R. Marx, Ricardo M. Avelino, Torbjørn Netland, Mennatallah El-Assady
Organizations: Department of Computer Science & ETH AI Center, ETH Zurich, Switzerland · Department of Computer Science & Architecture, ETH Zurich, Switzerland · Department of Management, Technology, and Economics, ETH Zurich, Switzerland · Department of Computer Science, ETH Zurich, Switzerland
Organizational knowledge is fragmented across a variety of software systems, tacit expertise, and manual documents that have traditionally been designed for human consumption. As AI systems are increasingly deployed and granted decision-making roles, they require access to this knowledge. This raises two questions: how should organizations store and maintain knowledge so that it remains accessible to both humans and future AI systems, and how should agency be allocated between humans and AI across tasks with different risks and levels of uncertainty? In this position paper, we describe how organizational knowledge evolves and contribute a framework that maps task attributes and knowledge availability to recommended agency allocations and control mechanisms. We illustrate the applicability of the framework on two different manufacturing tasks: a routine operation (visual quality inspection) and a one-off strategic decision (factory location), and conclude with opportunities for future research.
Figures & tables
Figure 1. Our mixed agency model for decision-making in manufacturing consists of a) grounding on available knowledge, b) problem characteristics that impact c) the level of agency. Teaser describing our framework for mixed agency In human-AI collaboration, an orchestrator can dynamically distribute agency between user and system. The agency scale has three anchors: User Agency, where the system assists the user; Mixed Agency, where the system collaborates with the user; and System Agency, where the system can act independently of users. The orchestrator makes this decision based on Grounding and Problem. Grounding refers to the basis on which we make decisions. It distinguishes between descriptive knowledge, which includes historical, semantic, and predictive knowledge, and prescriptive knowledge, which includes procedural and normative knowledge. Predictive knowledge is informed by historical and semantic knowledge and, in turn, informs prescriptive knowledge. The problem can be characterized by four attributes: The task can be routine or non-routine. The goal can be measurable or abstract. The solution space can be bounded or unbounded. The consequences can be forgiving or unforgiving. All these influence the orchestrator's decision to distribute the Agency. Keywords: knowledge theory,