Reciprocal Trust and Distrust in Artificial Intelligence Systems: The Hard Problem of Regulation

Martino Maggetti

cs.AIApr 7, 2026

Reciprocal Trust and Distrust in Artificial Intelligence Systems: The Hard Problem of Regulation

Authors: Martino Maggetti

Organizations: Institute of Political Studies (IEP), University of Lausanne

Abstract

Policy makers, scientists, and the public are increasingly confronted with thorny questions about the regulation of artificial intelligence (AI) systems. A key common thread concerns whether AI can be trusted and the factors that can make it more trustworthy in front of stakeholders and users. This is indeed crucial, as the trustworthiness of AI systems is fundamental for both democratic governance and for the development and deployment of AI. This article advances the discussion by arguing that AI systems should also be recognized, as least to some extent, as artifacts capable of exercising a form of agency, thereby enabling them to engage in relationships of trust or distrust with humans. It further examines the implications of these reciprocal trust dynamics for regulators tasked with overseeing AI systems. The article concludes by identifying key tensions and unresolved dilemmas that these dynamics pose for the future of AI regulation and governance.

Figures & tables

	Trustee is trustworthy
	Yes	No
Trustor is trustful (Yes)	(1) Warranted trust	(3) Unwarranted trust
Trustor is trustful (No)	(2) Unwarranted distrust	(4) Warranted distrust

Table 1: Alignment of trust and trustworthiness

Context	Costlier error	Who gets more and/or final authority?	Key regulatory instrument
Nuclear launch, strategic warning	False positive	Human	Statutory redundancy rule, mandatory uncertainty display
Reactor, chemical process, flight control	False negative	AI system	Non-overridable shutdown logic, regulator-audited firmware

Table 2: Allocate autonomy to the costlier error

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Feature	Human Operators (1986)	Modern AI System
Data Processing	Minutes (Slow)	Milliseconds (Real-time)
Protocol Adherence	Subjective / Negotiable	Absolute / Rigid
Fatigue/Stress	High (24-hour shift)	Zero
Physics Intuition	Limited by instrumentation	Comprehensive (Simulated)

Appendix

Table 4

Trust in AI

Trust in artificial intelligence concerns when people and institutions should rely on AI, and how systems can make that reliance warranted and safe.

See all 168 papers

Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems
Mar 25, 2026Adeela Bashir, Zhao Song, Ndidi Bianca Ogbo +21Trust in AI Human Oversight of AI
School Computing, Engineering and Digital Technologies, Teesside University · School of Mathematics, University of Birmingham · Informatics Institute, University of Amsterdam +7
Regulating autonomous and agentic AI
Jul 23, 2026Chris Reed, Alex Austria, Anmol Bharuka +3AI Risk Management AI Agent Governance
Professor of Electronic Commerce Law at the Centre for Commercial Law Studies, Queen Mary University of London · postgraduate student researchers for the Cloud Legal Project
Stability of AI Governance Systems: A Coupled Dynamics Model of Public Trust and Social Disruptions
Mar 10, 2026Jiaqi Lai, Hou Liang, Weihong HuangHawkes Processes AI Governance
Nanyang Technological University, Singapore

Mar 25, 2026J/K move · Enter open · S save

Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems

Adeela Bashir, Zhao Song, Ndidi Bianca Ogbo, Nataliya Balabanova+20

School Computing, Engineering and Digital Technologies, Teesside University · School of Mathematics, University of Birmingham · Informatics Institute, University of Amsterdam+7

Trust in AI Human Oversight of AI AI Safety AI Governance

Open paper AskPDF arXiv

As the capabilities and adoption of Artificial Intelligence (AI) systems grow, trust in these AI systems is an increasingly urgent concern. Much research has focused on models of AI governance and has primarily examined incentives for safe development and effective regulation. Hence they typically represented users trust as a one-shot adoption choice rather than as a dynamic, evolving process shaped by repeated interactions. We instead model trust as the dynamic choice of reduced monitoring in a repeated, asymmetric interaction between users and AI developers, where checking developers' behaviour is costly. Using evolutionary game theory, we study how users' strategies of trust and developers' strategies of providing safe (compliant) or unsafe (non-compliant) AI co-evolve under different levels of monitoring cost and institutional regimes. We conduct the analysis on both imitation-based and learning-based perspectives, with the stochastic finite-population dynamics, the infinite-population replicator analysis and the reinforcement learning analysis. We find three robust long-run regimes: no adoption by users while developers provide unsafe AI, unsafe but widely adopted systems, and safe systems that are widely adopted. Only the last is desirable, and it arises when penalties for unsafe behaviour exceed the extra cost of safety and users can still afford to monitor at least occasionally. Our results formally support governance proposals that emphasise transparency, low-cost monitoring, and meaningful sanctions, and they show that neither regulation alone nor blind user trust is sufficient to prevent the drift towards unsafe or low-adoption outcomes.

Abstract

Trust in AI

Explore similar work

Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems

Regulating autonomous and agentic AI

Stability of AI Governance Systems: A Coupled Dynamics Model of Public Trust and Social Disruptions