Advancing Trustworthy AI in Healthcare Through Meta-Research: Results of an Interdisciplinary Design-Thinking Workshop
Authors: Valerie Bürger, Marlie Besouw, Jana Fehr, Riana Minocher, Emma Moorhead, Isabel Velarde, Louis Agha-Mir-Salim, Julia Amann, Alexandra Bannach-Brown, David B. Blumenthal, Kaitlyn Hair, Bert Heinrichs, Moritz Herrmann, Elizabeth Hofvenschiöld, Sune Holm, Anne A. H. de Hond, Sara Kijewski, Stuart McLennan, Timo Minssen, Marco S. Nobile, Nico Pfeifer, Jessica L. Rohmann, Tony Ross-Hellauer, Marija Slavkovik, Karin Tafur, Eleonora Viganò, Magnus Westerlund, Tracey Weissgerber, Vince I. Madai
Organizations: QUEST Center for Responsible Research, Berlin Institute of Health (BIH) at Charité – Universitätsmedizin Berlin, Berlin, Germany · Department of Medical Imaging, Radboud University Medical Centre, Nijmegen, The Netherlands · Institute of Medical Informatics, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany · Einstein Center Digital Future, Berlin, Germany · Strategy & Innovation, Careum Foundation, Zurich, Switzerland · Institute of Social and Preventative Medicine, University of Bern, Bern Switzerland · Biomedical Network Science Lab, Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg · EPPI Centre, Social Research Institute, University College London, United Kingdom · Institute of Neuroscience and Medicine: Brain and Behaviour (INM-7), Forschungszentrum Jülich, Jülich, Germany; Institute of Science and Ethics (IWE), University of Bonn, Bonn, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine, LMU Munich, Munich, Germany · ESB Business School, Reutlingen University, Germany · Department of Food and Resource Economics, University of Copenhagen, Frederiksberg, Denmark · Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands · Department of Health Sciences and Technology, Chair of Bioethics, ETH Zurich, Zurich, Switzerland · Institute of History and Ethics in Medicine, Department of Preclinical Medicine, TUM School of Medicine and Health, Technical University of Munich · Centre for Advanced Studies in Bioscience Innovation Law (CeBIL), Faculty of Law, University of Copenhagen, Denmark · Department of Environmental Sciences, Informatics and Statistics, Ca’ Foscari University of Venice, Venice, Italy · Institute for Bioinformatics and Medical Informatics, University of Tübingen, Tübingen, Germany · Open and Reproducible Research Group, Know Center Research GmbH, Graz, Austria · Department of Information Science and Media Studies, University of Bergen, Bergen, Norway · Independent AI Researcher (AI Law and Ethics); Founder and Principal AI Advisor in AI Governance (Policy and Regulation), Paris, France · Digital Society Initiative, University of Zurich; Swiss Institute for Entrepreneurship, University of Applied Sciences of the Grisons, Chur, Switzerland · Department of Business and Economics, and Law, Abo Academy University, Finland; Arcada University of Applied Sciences, Finland · CNC-UC, Center for Neuroscience and Cell Biology, University of Coimbra, Coimbra, Portugal; CIBB, Center for Innovative Biomedicine and Biotechnology, University of Coimbra, Coimbra, Portugal · Faculty of Computing, Engineering and the Built Environment, School of Computing and Digital Technology, Birmingham City University, Birmingham, United Kingdom
Meta-research and Trustworthy AI (TAI) share common goals, namely improving evidence, robustness, and transparency, yet there is very little interplay between the two fields. To investigate the potential benefits of closer collaboration between the domains of TAI in healthcare and meta-research, we convened an interdisciplinary workshop funded by the Volkswagen Foundation in February 2025. The workshop aimed to collaboratively examine key challenges in translating AI ethics principles into practice and to identify potential solutions informed by meta-research approaches. A Design Thinking-informed co-creation approach was followed by an inductive descriptive analysis of the outputs. Our results demonstrate how meta-research can offer concrete contributions to address pressing challenges of TAI in healthcare. These challenges include the dynamic and complex nature of TAI ethical requirements and principles, common terminology and understanding of TAI, ensuring robustness, replicability, and reproducibility, choosing adequate evaluation metrics, lack of transparency, advancing preclinical biomedical research, and validation in real-world clinical environments. We present a catalog of ideas and a roadmap for future research, which synthesize existing interconnections and identify concrete next steps and open research gaps, thereby serving as a foundation for future interdisciplinary efforts.
Responsible AI studies how to design, evaluate, and govern AI systems to promote fairness, safety, transparency, accountability, privacy, and sustainability.
Building trustworthy AI systems for mental health support is a shared priority across stakeholders from multiple disciplines. However, "trustworthy" remains loosely defined and inconsistently operationalized. AI research often focuses on technical criteria (e.g., robustness, explainability, and safety), while therapeutic practitioners emphasize therapeutic fidelity (e.g., appropriateness, empathy, and long-term user outcomes). To bridge the fragmented landscape, we propose a three-layer trust framework, covering human-oriented, AI-oriented, and interaction-oriented trust, integrating the viewpoints of key stakeholders (e.g., practitioners, researchers, regulators). Using this framework, we systematically review existing AI-driven research in mental health domain and examine evaluation practices for ``trustworthy'' ranging from automatic metrics to clinically validated approaches. We highlight critical gaps between what NLP currently measures and what real-world mental health contexts require, and outline a research agenda for building socio-technically aligned and genuinely trustworthy AI for mental health support.
Xin Sun, Yue Su, Yifan Mo +9
National Institute of Informatics (NII), Japan · Vrije Universiteit Amsterdam, the Netherlands
With the development of artificial intelligence (AI), the landscape of meta-ethics, which has largely centred on human ethics, faces pressures that may significantly reconfigure it. In particular, if future AI systems were to exhibit sufficiently integrated capacities for moral reasoning, moral intentionality, and moral reflection, novel meta-ethical questions would arise concerning what I call "AI's own ethics", as distinct from ethical principles merely imposed on AI by human designers. This paper offers a conditional and methodological framework for identifying the questions that would emerge if such AI systems were to arise. On that basis, the paper distinguishes four domains of meta-ethical inquiry in the era of AI: questions about the nature of human ethics from the human perspective; questions about the nature of AI's own ethics from the human perspective; questions about the nature of human ethics from the AI perspective; and questions about the nature of AI's own ethics from the AI perspective. The paper then considers how some existing mainstream meta-ethical theories (such as cognitivism and non-cognitivism, error theory and success theory, relativism, and objective realism) might illuminate these domains, while arguing that many familiar human-centred formulations of those theories may not transfer straightforwardly to AI cases without substantial revision. The overall conclusion is that the emergence of AI's own ethics would place significant pressure on current frameworks and may require substantial refinement, reconstruction, or reconceptualisation.
Shang Lu
School of Computer Science and Engineering, UNSW Sydney, Sydney, Australia
Trustworthy AI encompasses many aspirational aspects for aligning AI systems with human values, including fairness, privacy, robustness, explainability, and uncertainty quantification. The ultimate goal of Trustworthy AI research is to achieve all aspects simultaneously. However, efforts to enhance one aspect often introduce unintended trade-offs that negatively impact others. In this position paper, we review notable approaches to five aspects and systematically consider every pair, detailing the negative interactions that can arise. For example, applying differential privacy to model training can amplify biases, undermining fairness. Drawing on these findings, we take the position that current research practices of improving one or two aspects in isolation are insufficient. Instead, research on Trustworthy AI must account for interactions between aspects and adopt a holistic view across all relevant axes at once. We offer recommendations for how researchers can work towards integrated trust, and provide guidance for practitioners to manage interactions today.