Healthy skepticism in AI: a data visualization research agenda
Authors: G. Elisabeta Marai, Marc Baaden, Michael Behrisch, Michael Krone, Pere-Pau Vázquez
Organizations: University of Illinois Chicago, Chicago, IL, 60607, U.S.A. · CNRS (French National Centre for Scientific Research), Paris, 75794, France · Utrecht University, Utrecht, 3584CC, The Netherlands · Stuttgart Technical University of Applied Sciences, Stuttgart, 70174, Germany · Universitat Politècnica de Catalunya, Barcelona 08034, Spain
Research in data visualization of artificial intelligence (AI) models has historically focused on enhancing trust through visual explanations of AI. The trustworthiness line of work was built at least partially on an assumption that humans were critical users unlikely to adopt AI technology. It is increasingly clear that human trust levels in AI span, in fact, a wide range from critical to over-reliant. There is an urgent need to support both trust and healthy skepticism in AI solutions. We argue that it is healthy for humans to adopt a skeptical view both on the results of AI models and on the use of such AI models. We share our thoughts on the rising phenomenon of over-reliance on AI models, the risks and opportunities in using AI models, and the role of data visualization in over-reliance situations where humans are not motivated to engage in critical thinking.
Figures & tables
Figure 1: Visualization for healthy skepticism and visualization for explainable AI (XAI). The space of reliance and trust is a spectrum, from cautious or avoidant (left) to over-reliant, nearly blind trust (right). Visualization for XAI is focused on the left end—its mission is to increase trust, and it tends to serve model builders, co-builders, or early adopters, who have an interest in the inner workings of the AI model and are motivated to interact with it and explore. Visualization for healthy skepticism aims to move in the opposite direction, from the opposite end, and needs to address a broader, usually deployment (as opposed to development) audience. Most broader audiences have as much interest in XAI as we, the authors, have in understanding spam filtering. Visualization can help in over-reliance situations where humans are not motivated to engage in critical thinking.
Figure 2: Explanatory design for healthy skepticism from Wentzel et al. [ 12 ] . A yellow thumbs up/down icon indicates the AI model limitations: how much data support the digital twin model had when making that recommendation. Along with the digital twin AI model prediction in purple, a decoy AI model prediction based on nearest neighbors is shown in green, to further encourage the clinician to pause and reflect.
Department of Computer Science and Engineering, IIT Delhi · Department of Philosophy, Duke University · Department of Computer Science, Carnegie Mellon University +1