AI Model Design and Physician Liability

Shujie Luan; Shubhranshu Singh; Tinglong Dai

cs.AIAug 12, 2026

AI Model Design and Physician Liability

Authors: Shujie Luan, Shubhranshu Singh, Tinglong Dai

Organizations: The University of Western Australia, UWA Business School, Perth, Australia · Carey Business School, Johns Hopkins University, Baltimore, Maryland 21202

Abstract

Clinical algorithms can perform differently across patient groups, a concern that grows as these algorithms increasingly take the form of artificial intelligence (AI) models. In the United States, a 2024 federal rule bars healthcare providers that receive federal funds from discriminating through their use of clinical algorithms, which exposes them to liability when such use disadvantages a protected patient group. We examine how such liability considerations reshape (i) an AI firm's model design decisions that drive group-specific accuracy and (ii) a physician's decisions to use AI in healthcare delivery. The firm designs an AI model for two patient groups, and improving accuracy for the disadvantaged group is more costly. The physician (who remains the accountable decision maker) then decides whether to consult AI, weighing the reduction in clinical uncertainty against expected liability exposure when AI errors disproportionately affect the disadvantaged group. We find that this liability can induce disparate use of AI: the physician may reduce AI use overall and, over an intermediate range of liability, rely on AI less for disadvantaged patients. The effect is non-monotone. As liability increases, the physician's use of AI for disadvantaged patients first declines, then rises as the firm reallocates investment toward reducing disparity or switches to an equal-accuracy design. Mandating equal algorithmic accuracy across patient groups can then inadvertently harm both groups, because a uniform accuracy requirement distorts the firm's investment incentives and the physician's equilibrium AI-use decisions.

Figures & tables

	Meaning	Example	Payment Flow
$c$	Patient’s cost of AI use	Out-of-pocket expenses (copayments, deductibles, coinsurance) or disutility (distrust, privacy concerns)	Borne by patient
$r$	Provider’s per-use revenue	Insurer reimbursement (for example, CMS, private insurers) and reputational benefit, if any, from AI use	Accrued by provider
$f$	AI firm’s per-use fee	Payment from provider, hospital, or health system to AI firm for technology use or licensing	Provider $\rightarrow$ AI firm

Table 1: Meaning, Examples, and Payment Flow for Parameters

c

r

, and

f

	Physician’s decision
AI signal	$T_{1}$	$T_{2}$
$s_{t}=T_{1}$	$Q_{1\|1}\cdot b-c$	$Q_{2\|1}\cdot b-c$
$s_{t}=T_{2}$	$Q_{1\|2}\cdot b-c$	$Q_{2\|2}\cdot b-c$

Table 2: The Patient’s Expected Utility When the Physician Uses AI

	Physician’s decision
AI signal	$T_{1}$	$T_{2}$
$s_{t}=T_{1}$	$Q_{1\|1}\cdot b-c+\theta(r-Q_{2\|1}\cdot\mathbf{1}_{t=y}\,\ell)$	$Q_{2\|1}\cdot b-c+\theta r$
$s_{t}=T_{2}$	$Q_{1\|2}\cdot b-c+\theta r$	$Q_{2\|2}\cdot b-c+\theta(r-Q_{1\|2}\cdot\mathbf{1}_{t=y}\,\ell)$

Table 3: The Physician’s Expected Payoff When Treating an Individual Patient Using AI

Figure 1: Timing of the Game. The AI firm’s first-stage choice is between a disparate design (

\rho_{x}>\rho_{y}

) and an equal-accuracy design (

\rho_{x}=\rho_{y}

Figure 2: Parameter region in which type- $y$ AI use is non-monotone in liability ( Proposition 2 ).

Figure 3: Parameter region in which type- $y$ AI use switches between overuse and underuse ( Proposition 3 (c)).

Figure 4: AI use for type-

y

patients: for a given

\ell

, the physician uses AI when

\alpha

lies between the cutoffs. Dash-dot lines correspond to an altruistic physician (

\theta=0

); solid lines correspond to an impurely altruistic physician.

	Physician’s decision
AI signal	$T_{1}$	$T_{2}$
$s_{t}=T_{1}$	$Q_{1\|1}\cdot b-c+Q_{2\|1}\cdot\mathbf{1}_{t=y}\,\ell$	$Q_{2\|1}\cdot b-c$
$s_{t}=T_{2}$	$Q_{1\|2}\cdot b-c$	$Q_{2\|2}\cdot b-c+Q_{1\|2}\cdot\mathbf{1}_{t=y}\,\ell$

Table 4: The Patient’s Expected Utility When the Physician Uses AI and Liability Transfers Are Included

	Physician’s decision
AI signal	$T_{1}$	$T_{2}$
$s_{t}=T_{1}$	$Q_{1\|1}\cdot b-c+\theta r-(\theta-1)Q_{2\|1}\cdot\mathbf{1}_{t=y}\,\ell$	$Q_{2\|1}\cdot b-c+\theta r$
$s_{t}=T_{2}$	$Q_{1\|2}\cdot b-c+\theta r$	$Q_{2\|2}\cdot b-c+\theta r-(\theta-1)Q_{1\|2}\cdot\mathbf{1}_{t=y}\,\ell$

Table 5: The Physician’s Expected Payoff When Treating an Individual Patient Using AI and Liability Transfers Are Included

Panel of Figure 5 — Figure 5: Aggregate welfare as a function of liability $\ell$ . In both examples, the upward-sloping segment is welfare under the firm’s preferred disparate design; the flat segment is welfare once the firm switches to an equal-accuracy design, which does not vary with $\ell$ .

Panel of Figure 11 — Figure OA1 : Welfare effects of an equal-accuracy requirement for each patient type (for panel (a): $b=2$ , $\theta=1.1$ , $c=0.6$ , $r=0.5$ , $\kappa_{x}=1.1$ , $\kappa_{y}=1.3$ , $\ell=0.1$ ; for panel (b): $b=1.6$ , $\theta=1$ , $c=0.6$ , $r=0.5$ , $\kappa_{x}=1.3$ , $\kappa_{y}=1.4$ , $f=0.6$ ). In the hatched region, the firm serves type- $x$ patients only under the disparate design. Regions labeled “Equal-accuracy AI is optimal” are those in which the equal-accuracy design is optimal.

Panel of Figure 12 — Figure OA2 : Welfare effects of an equal-accuracy requirement under exogenous and endogenous pricing. Panel (a) reproduces the baseline with exogenous $(c,f)=(0.6,0.6)$ ; panel (b) endogenizes $f$ and imposes $c=f$ . Both panels use $b=1.6$ , $\theta=1$ , $r=0.5$ , $\kappa_{x}=1.3$ , and $\kappa_{y}=1.4$ . In the hatched regions, the firm serves type- $x$ patients only under the disparate design. Regions labeled “Equal-accuracy AI is optimal” are those in which the equal-accuracy design is optimal.

Figure OA3 : Impact of mandating equal-accuracy AI on expected welfare for each patient type ( $\underline{\alpha}=0.4$ , $\overline{\alpha}=0.88$ , $b=0.9$ , $\theta=1$ , $c=0.36$ , $r=0.3$ , $\kappa_{x}=0.4$ , $\kappa_{y}=0.55$ , $\tilde{f}=0.28$ ). In the hatched region, the firm serves type- $x$ patients only under the disparate design.

Human-AI Decision Making

Human-AI decision making studies how people and AI systems share, support, or delegate decisions across tasks and institutions.

See all 173 papers

Optimal Liability Design for Medical AI
Aug 4, 2026Rui Mao, Tingliang Huang, Houcai ShenMechanism Design AI Accountability
School of Management & Engineering, Nanjing University, Nanjing 210093, China · Haslam College of Business, University of Tennessee, Knoxville, TN 37996, USA
Algorithmic Authority and the Clinical Standard of Care
Apr 29, 2026Aizierjiang AiersilanAI Accountability Healthcare
The George Washington University
Whose doctor does the AI recommend? An algorithm audit of reputation and demographic signals in large language model-assisted physician choice
Aug 14, 2026Syeda Anshrah Gillani, Mirza Samad Ahmed BaigLarge Language Model-Based Recommendation LLM Auditing
Heidelberg University, Heidelberg, Germany · Fandaqah, Al Khobar, Saudi Arabia

Aug 4, 2026J/K move · Enter open · S save

Optimal Liability Design for Medical AI

Rui Mao, Tingliang Huang, Houcai Shen

School of Management & Engineering, Nanjing University, Nanjing 210093, China · Haslam College of Business, University of Tennessee, Knoxville, TN 37996, USA

Mechanism Design AI Accountability Principal-Agent Theory Clinical Decision-Making

Open paper AskPDF arXiv

Artificial intelligence (AI) is increasingly integrated into medical decision-making, yet its liability implications remain complex, particularly when physicians differ in diagnostic skills and their quality is unobservable. This paper develops a principal-agent model in which a social planner designs medical liability to regulate a physician with private quality information who chooses between a standard treatment, a personalized judgment-based treatment, or following an imperfect AI recommendation. Our analysis yields several novel insights. First, we show that the optimal mechanism under asymmetric information is surprisingly simple: a uniform, one-size-fits-all liability level for all physician types who deviate from the standard of care. Despite physician heterogeneity, this simple policy often achieves the full-information first-best outcome, particularly when standard care is reliable or AI is highly accurate. Second, the relationship between AI accuracy and optimal liability is non-monotonic. Contrary to common intuition, better AI does not always imply more relaxed liability. As AI accuracy increases, the optimal liability either decreases monotonically or follows an inverted-U pattern, depending on the uncertainty of the standard treatment. Third, asymmetric information does not universally reduce social welfare. Welfare loss arises only when standard care is unreliable and AI accuracy is too low; even then, its magnitude follows an inverted U-shape, initially increasing as AI complicates the regulatory problem, but declining as more accurate AI helps mitigate it. Finally, we find that information asymmetry is a double-edged sword in the presence of AI, and greater transparency does not benefit all stakeholders equally.

Abstract

Human-AI Decision Making

Explore similar work

Optimal Liability Design for Medical AI

Algorithmic Authority and the Clinical Standard of Care

Whose doctor does the AI recommend? An algorithm audit of reputation and demographic signals in large language model-assisted physician choice