Concordance Plausibility Check for Intervention-Conditioned Human Digital Twins

Saeed Farzi; Christopher Kipps; John McBeth; Gabriela Czanner; Sofia Michopoulou; Benjamin Macarthur; Sarah Ennis; Paul Elkington

cs.AIOct 17, 2026

Concordance Plausibility Check for Intervention-Conditioned Human Digital Twins

Authors: Saeed Farzi, Christopher Kipps, John McBeth, Gabriela Czanner, Sofia Michopoulou, Benjamin Macarthur, Sarah Ennis, Paul Elkington,

Organizations: School of Electronics and Computer Science, University of Southampton, Southampton, UK · Faculty of Medicine, University of Southampton, Southampton, UK · University Hospital Southampton NHS Foundation Trust, Southampton, UK · Primary Care Research Centre, Faculty of Medicine, University of Southampton, Southampton, UK · Digital Health and Biomedical Engineering Group, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton, UK · Human Development and Health, Faculty of Medicine, University of Southampton, Southampton, UK · Mathematical Sciences, University of Southampton, Southampton, UK · Human Genomic Informatics Group, Faculty of Medicine, University of Southampton, Southampton, UK · Institute for Life Sciences, University of Southampton, Southampton, UK · Clinical and Experimental Sciences Academic Unit, Faculty of Medicine, University of Southampton, Southampton, UK · NIHR Biomedical Research Centre: Southampton, University of Southampton and University Hospital Southampton NHS Foundation Trust, Southampton, UK · Lifecourse Epidemiology Centre, University of Southampton, Southampton, UK

Abstract

Intervention-conditioned human digital twins predict treatment alternatives that cannot be observed in the same patient. Ordinary predictive accuracy therefore cannot establish counterfactual credibility. We propose the Concordance Plausibility Check (CPC), a claim-centered framework combining real-world evidence, qualified simulation and clinical knowledge. CPC matches evidence to the causal claim, audits contradictions and shared dependencies, and reports Pass, Fail or reason-coded Indeterminate.

Figures & tables

Figure 1: The double-invisibility problem. After a treatment decision, only one path is realized for a real patient, and that factual trajectory may itself be incompletely observed. The alternative intervention trajectory is not observed for the same patient. Evaluation must therefore distinguish evidence about the observed course from evidence that can constrain the unobserved alternative. Filled circles represent observed values, while open circles indicate missing values. The branching point marks where alternative interventions are applied

Claim family	Clinical question	Main evidential boundary
Intervention response ( $\mathsf{CC}$ -O, $\mathsf{CC}$ -T)	What would happen under a specified treatment or treatment policy?	Requires a well-defined intervention, adequate data support and credible handling of longitudinal observation and treatment history.
Population effect ( $\mathsf{CC}$ -ATE, $\mathsf{CC}$ -CATE)	How would two interventions differ on average, or within a defined subgroup?	Population or subgroup effects may be estimable or bounded, but they do not reveal the missing outcome for a particular patient.
Individual counterfactual ( $\mathsf{CC}$ -ITE)	What is the realized difference for patient $i$ under two interventions?	The expected contrast for patients sharing $i$ ’s history is CC-CATE; a realized CC-ITE needs cross-world assumptions.
Structural or attributional ( $\mathsf{CC}$ -STR, $\mathsf{CC}$ -PS, $\mathsf{CC}$ -ATTR)	Through which pathway, mechanism or subgroup structure would an effect arise?	Mechanistic or attributional conclusions require extra structural and pathway assumptions beyond those needed for a population effect.
Decision or policy ( $\mathsf{CC}$ -V)	Which treatment policy should be preferred?	Requires evidence for the relevant treatment histories, outcomes and clinical utility over the population where the policy will be used.

Table 1: The CPC claim families. The same digital twin may support several claim types, but each must be evaluated separately.

Figure 2: The Three-Reality Concordance Framework. Observed Reality asks whether the claim is compatible with real-world evidence; Virtual Reality (qualified-simulation evidence) tests recovery in declared simulations; Knowledge Reality asks whether the result is clinically and mechanistically admissible. Compatibility strengthens claim-specific credibility only within each reality’s stated assumptions and authority. The surrounding labels denote population, time, support, structure, paired testing, benchmarking, authority, grounding and adjudication rubric.

Reality	Useful contribution	Limit that must remain explicit
Observed	Factual performance, real-world causal estimates or bounds, support and transport checks, and negative controls.	Cannot reveal the missing paired outcome for the same real patient.
Virtual	Direct recovery tests for paired outcomes, trajectories or pathways under a declared simulator and intervention structure.	Demonstrates compatibility with that simulator, not clinical counterfactual truth.
Knowledge	Clinical, temporal, pharmacological and mechanistic admissibility based on qualified external evidence and expert review.	Can reject implausible behavior but cannot supply the missing counterfactual outcome.

Table 2: What each CPC reality contributes and where its authority ends.

Figure 3: CPC claim-level evaluation workflow. Evaluation begins by defining the clinical question and context of use. The claim is then checked for factual credibility, counterfactual plausibility, contradiction, uncertainty and evidence dependence before a final Pass, Fail or Indeterminate status is reported.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Prior approach	Established contribution	Elements adopted in CPC	Additional operational role within CPC
ASME V&V40 [ 2 ] ; ICH E9(R1) [ 15 ]	Risk-informed, context-of-use model credibility; prespecification of clinical estimands.	Registration of the intended population, intervention, outcome, horizon and context of use.	Links each registered counterfactual claim to its applicable causal identification assumptions and mandatory evidence checks. An unmet mandatory requirement prevents an unqualified Pass.
Vallée [ 39 , 41 , 40 ]	Clinical-claim-based validation, decision relevance and guidance for causal digital twins.	Alignment of validation evidence with the intended clinical claim and its decision context.	Connects nine formal counterfactual claim classes to claim-specific identification requirements, qualified evidence streams and explicit Pass, Fail or reason-coded Indeterminate decisions.
Cornish et al. [ 6 ]	Causal falsification using testable implications and potential contradictions.	Prespecified falsification checks for assessing compatibility with causal claims.	Incorporates qualified contradictions as non-compensatory Fail triggers for mandatory criteria. Inconclusive or insufficiently informative tests cannot establish Pass.
Laudy [ 18 ]	Distinction between identifiable marginal quantities and cross-world dependence; partial identification and bounds.	Explicit separation of marginal, joint and individual counterfactual quantities.	Links identification limitations to claim-specific decision rules. Unresolved cross-world assumptions produce reason-coded Indeterminate outcomes; simulator-based paired contrasts do not establish clinical counterfactual truth.

Appendix

Table S1: Established methodological foundations and their distinct roles within the proposed CPC evaluation framework. CPC builds on these approaches by linking claim-specific causal requirements, qualified evidence and falsification to auditable, non-compensatory decisions.

Class	Representative target functional	Principal requirements	Main real-world constraint / residual limitation
$\mathsf{CC}$ -O	$\mathbb{E}(Y_{q}^{g})$ or $F_{Y_{q}^{g}}$	A1–A3; A4 when transported	Interventional mean, distribution or bounds; support and identification uncertainty may remain.
$\mathsf{CC}$ -T	$P(X_{b+1:t}^{g}\mid h)$	Sequential A1–A3; A4 when transported	Longitudinal interventional distribution; sparse support and long-horizon uncertainty may remain.
$\mathsf{CC}$ -ATE	$\mathbb{E}(Y^{g_{1}}-Y^{g_{0}})$	A1–A3; A4 when transported	Population causal contrast, sensitivity region or bounds; no individual cross-world effect is implied.
$\mathsf{CC}$ -CATE	$\mathbb{E}(Y^{g_{1}}-Y^{g_{0}}\mid H_{b}=h)$	A1–A3; A4 when transported	Conditional causal contrast under local support; sparse histories may remain unsupported.
$\mathsf{CC}$ -ITE	Realized difference for patient $i$ : $Y_{i}^{g_{1}}-Y_{i}^{g_{0}}$	A1–A3 for marginals plus A5	An individualized conditional mean is CC-CATE, not CC-ITE; joint dependence requires A5 and a realized individual contrast remains unobserved.
$\mathsf{CC}$ -STR	$\mathbb{E}[Y^{a_{1},M^{a_{1}}}-Y^{a_{1},M^{a_{0}}}]$	A1–A3 plus A6; A5 when cross-world contrasts are used	Pathway-specific constraints; the asserted pathway remains assumption-dependent.

Appendix

Table S2: Full CPC counterfactual claim taxonomy, representative target functionals, principal identification requirements and real-world constraints.

Claim	Counterfactual toy question	Domain / classification
$\mathsf{CC}$ -O	What would five-year dementia risk be under fixed statin intervention $a_{S}$ ?	Intervention Response; population; single-point.
$\mathsf{CC}$ -T	How would cognition, cardiovascular state and dementia status evolve under $a_{S}$ ?	Intervention Response; personalized; longitudinal.
$\mathsf{CC}$ -ATE	What is the average five-year risk difference under $a_{S}$ versus $a_{0}$ ?	Population Effect; population; single-point.
$\mathsf{CC}$ -CATE	What is the average contrast under $a_{S}$ versus $a_{0}$ among histories $H_{b}=h$ ?	Population Effect; conditional population/subgroup.
$\mathsf{CC}$ -ITE	What is the realized five-year difference for patient $i$ under $a_{S}$ versus $a_{0}$ (not the CC-CATE expected difference for histories like $i$ ’s)?	Individual Counterfactual; personalized.
$\mathsf{CC}$ -STR	How much of the fixed-intervention contrast is attributed to $A\rightarrow M^{\mathrm{CV}}\rightarrow Y$ ?	Structural and Attributional; structural.

Appendix

Table S3: Toy dementia–statin questions illustrating the nine formal claim classes.

Mandatory check	P	F	I (primary code)
Factual	$D_{F}\leq q_{F,c}$ and $U_{F,c}\leq\varepsilon_{F,c}$	$D_{F}>q_{F,c}$ for registered target	Weak precision or material unregistered mismatch (I-E)
Observed Reality	Compatible with informative, qualified causal constraint and defensible assumptions	Qualified, registered causal contradiction	Uninformative (I-E), A2 uncertain (I-A), disagreement (I-D), no support (I-S)
Knowledge	Qualified adjudication finds no registered clinical violation	Confirmed registered admissibility violation	Source gap (I-E), unresolved clinical dispute (I-D)
Virtual (when required)	Qualified simulator recovery under registered scenarios/couplings	Confirmed registered failure to recover simulator benchmark	Poor qualification/coverage (I-E), coupling instability (I-U)
Falsification	Adequately powered registered tests find no contradiction	Confirmed registered falsifier detects material contradiction	Power below $1-\beta_{c}$ or missing test (I-E)
Provenance audit	Dependence and sensitivity audited; required status is stable	Confirmed registered mandatory violation	Empty independent lineage (I-E), unresolved source conflict (I-D), instability (I-U)

Appendix

Table S4: Claim-specific grading of mandatory checks. P means a check is completed at the registered evidential standard, not clinical counterfactual truth; I codes are non-exclusive. Virtual evidence is a verification-type necessary condition; Knowledge is rejection-only. Only qualified Observed-Reality evidence directly supports a clinical causal contrast.

Standard	Required evidence	Failure consequence	Core	High-risk
Target-population alignment	Eligibility, setting, calendar period and subgroup coverage match the target population	Restrict claim or perform transport analysis; record unsupported subgroups.	Y	+
Well-defined intervention	Treatment version, dose, timing, duration, adherence and switching are encoded	Consistency is not credible.	Y	–
Temporal validity	Confounders precede treatment; branching time and visit order are auditable; no future leakage	Factual and causal checks are invalid.	Y	+
Observation-process validity	Censoring, dropout, visit timing, competing events and missing modalities are described and handled relative to the target estimand	Factual visibility may be selectively biased.	Y	+
Confounder coverage	Clinically prespecified common causes are measured with adequate quality	Exchangeability remains uncertain; a broader sensitivity analysis is required.	Y	+
Sequential support	Required actions have adequate probability over evolving histories and horizons	Unsupported regions are excluded, bounded or Indeterminate.	Y	+

Appendix

Table S5: Observed-Reality qualification; core standards apply to registered use, with enhanced evidence (+) for higher-risk claims.

Standard	Required evidence	Failure consequence	Core	High-risk
Structural transparency	State, treatment, outcome, mediator, latent and stochastic mechanisms are documented and versioned	Counterfactual quantities cannot be interpreted.	Y	–
Modular intervention	Replacing the intervention changes only the declared mechanism unless explicitly targeted otherwise	Contrasts are not causally attributable.	Y	–
Paired virtual patients	Baseline and declared latent characteristics are preserved across interventions; noise coupling is specified	Individual/pathwise contrasts are ill-defined.	Y	–
Known counterfactual benchmark	Paired potential outcomes, joint distributions, individual contrasts or pathways required by the claim are exposed	No direct recovery test exists for that claim.	Y	–
Assumption alignment	Confounding, support, coupling and pathway assumptions are explicit and perturbable	Recovery cannot be interpreted against the claim.	Y	+
Stochastic replication	Heterogeneity, process noise and repeated patient-intervention trajectories are represented	Uncertainty is artificially narrow.	Y	+

Appendix

Table S6: Qualification standards for Virtual-Reality paired-simulator evidence, when required by the claim. These concern simulator suitability as benchmark evidence, not clinical truth.

Standard	Required evidence	Failure consequence	Core	High-risk
Domain authority	Relevant clinician expertise and/or curated, revisable evidence define admissible knowledge	No defensible reference standard.	Y	+
Evidence grounding	Authoritative, versioned sources; when an LLM is used, retrieval-augmented generation over a frozen, cited medical corpus and, where useful, a versioned ontology/knowledge graph, with prespecified retrieval configuration	judgment may rely on unsupported model memory or changing evidence.	Y	–
Task limitation	The clinician-calibrated medical large language model judges clinical, temporal, pharmacological and pathway plausibility; never the missing potential outcome and never as an independent Reality	Epistemic misuse or false evidential independence.	Y	–
Clinician-defined rubric	Hard violations, graded criteria, abstention conditions and escalation rules are prespecified	Subjective/non-reproducible evaluation.	Y	–
Human calibration	Error characteristics are measured on held-out clinician-labeled cases using sensitivity, specificity, abstention, clinically weighted loss and a prespecified false-concordance operating target	Judge reliability and unsafe endorsement risk are unknown.	Y	+
Blind evaluation	Generator identity is hidden and presentation order randomized	Preference bias may contaminate results.	Y	+

Appendix

Table S7: Qualification standards for Knowledge-Reality evidence and optional clinician-calibrated medical LLM adjudication (LLM-specific checks apply only when used).

Group	Registration field	Toy specification
Claim	Claim class	$\mathsf{CC}$ -ATE: Population Effect; population; non-structural; single-point; non-policy.
	Target population $\mathcal{P}_{c}$	Registered dementia–statin target population and relevant subgroups.
	Intervention contrast $\mathcal{G}_{c}$	Fixed statin intervention $a_{S}$ versus comparator $a_{0}$ .
	Time horizon $\tau_{c}$	Five years.
	Causal quantity $\psi_{c}$	$\mathbb{E}(Y^{a_{S}}-Y^{a_{0}})$ .
	Intended region $\Omega_{c}$	Registered clinical histories for which both interventions are to be assessed; later support restrictions require re-registration.

Appendix

Table S8: Filled toy registration for the dementia–statin

\mathsf{CC}

-ATE example. The values illustrate CPC registration fields and do not assert a clinical statin effect on dementia.

Counterfactual Evaluation

Counterfactual evaluation tests how models, explanations, and policies behave under hypothetical changes to inputs, evidence, actions, or environments.

See all 94 papers

The Digital Twin Counterfactual Framework: A Validation Architecture for Simulated Potential Outcomes
Apr 1, 2026Olav LaudyCounterfactual Evaluation Causal Effect Estimation
Equilibrium Causal Digital Twins: Validation, Transport, and Identification Limits
Jul 23, 2026Faraz Dadgostari, Neda NazemiCounterfactual Evaluation Structural Causal Models
Department of Mechanical & Industrial Engineering, Montana State University · Gianforte School of Computing, Montana State University
When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability
Sep 7, 2026Steven Wang, Kyle Hunt, Shaojie Tang +1Human Behavior Simulation Digital Twins
University at Buffalo

Apr 1, 2026J/K move · Enter open · S save

The Digital Twin Counterfactual Framework: A Validation Architecture for Simulated Potential Outcomes

Olav Laudy

Counterfactual Evaluation Causal Effect Estimation Causal Reasoning Causal Identifiability Digital Twins

Open paper AskPDF arXiv

The fundamental problem of causal inference - that the counterfactual outcome for any individual is never observed - has shaped the entire methodology of the field. Every existing approach substitutes assumptions for missing data: ignorability, parallel trends, exclusion restrictions. None produces the counterfactual itself. This paper proposes the Digital Twin Counterfactual Framework (DTCF): rather than estimating the counterfactual statistically, we simulate it using a digital twin and subject the simulation to a hierarchical validation regime. We formalize the digital twin simulator as a stochastic mapping within the potential outcomes framework and introduce a hierarchy of twin fidelity assumptions - from marginal fidelity through joint fidelity to structural fidelity - each unlocking a progressively richer class of estimands. The central contribution is threefold. First, a five-level validation architecture converts the unfalsifiable claim that the simulator produces correct counterfactuals into falsifiable tests against observable data. Second, a formal decomposition separates causal quantities into those that are marginally validated (ATE, CATE, QTE - testable through observable-arm comparison) and those that are copula-dependent (the ITE distribution, probability of benefit/harm, variance of treatment effects - permanently reliant on the unobservable within-individual dependence structure). Third, bounding, sensitivity, and uncertainty quantification tools make the copula dependence explicit. The DTCF does not resolve the fundamental problem of causal inference. What it provides is a framework in which marginal causal claims become increasingly testable, joint causal claims become explicitly assumption-indexed, and the gap between the two is formally characterized.

Abstract

Counterfactual Evaluation

Explore similar work

The Digital Twin Counterfactual Framework: A Validation Architecture for Simulated Potential Outcomes

Equilibrium Causal Digital Twins: Validation, Transport, and Identification Limits

When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability