cs.LGMar 6, 2026
SaveFrom Statistical Fidelity to Clinical Consistency: Scalable Generation and Auditing of Synthetic Patient Trajectories
Organizations: The University of Queensland, Brisbane, QLD, Australia · The University of New South Wales, Sydney, NSW, Australia · Shahid Sadoughi University of Medical Sciences and Health Services, Yazd, Iran · The University of Auckland, Auckland, New Zealand
Abstract
Access to electronic health records (EHRs) for digital health research is often limited by privacy regulations and institutional barriers. Synthetic EHRs have been proposed as a way to enable safe and sovereign data sharing; however, existing methods may produce records that capture overall statistical properties of real data but present inconsistencies across clinical processes and observations. We developed an integrated pipeline to make synthetic patient trajectories clinically consistent through two synergistic steps: high-fidelity generation and scalable auditing. Using the MIMIC-IV database, we trained a knowledge-grounded generative model that represents nearly 32,000 distinct clinical events, including demographics, laboratory measurements, medications, procedures, and diagnoses, while enforcing structural integrity. To support clinical consistency at scale, we incorporated an automated auditing module leveraging large language models to filter out clinical inconsistencies (e.g., contraindicated medications) that escape probabilistic generation. We generated 18,071 synthetic patient records derived from a source cohort of 180,712 real patients. While synthetic clinical event probabilities demonstrated robust agreement (mean bias effectively 0.00) and high correlation (R2=0.99) with the real counterparts, review of a random sample of synthetic records (N=20) by three clinicians identified inconsistencies in 45-60% of them. Automated auditing reduced the difference between real and synthetic data (Cohen's effect size d between 0.59 and 1.60 before auditing, and between 0.18 and 0.67 after auditing). Downstream models trained on audited data matched or even exceeded real-data performance. We found no evidence of privacy risks, with membership inference performance indistinguishable from random guessing (F1-score=0.51).
Figures & tables
Figure 1: Fidelity and structural evaluation of synthetic EHR data. a Vocabulary coverage across 31,901 concepts. While standard benchmarks (blue) capture high-frequency diagnoses, Coogee models the “long tail” of clinical care (red), including specific medications (e.g., Semaglutide) and laboratory tests (e.g., HbA1c) required for high-fidelity disease modeling. b-c , Structural characteristics comparing real (blue) and synthetic (red) cohorts. The synthetic data mirrors real-world distributions for visits per patient and codes per visit, with minor truncation in the extreme upper tail due to model context window limits. d-f , Probabilistic agreement assessed via density-scaled Bland-Altman plots. Minimal mean bias (0·00) and narrow 95% limits of agreement across unigrams (marginal code probabilities), same-visit co-occurrences, and sequential-visit dependencies confirm that the model reproduces phenotype prevalence and care pathways without systematic bias. g-i , Temporal fidelity evaluation using empirical cumulative distribution functions (ECDFs). Distributional alignment (KS-stats 0 06) across event gaps, length of stay, and inter-visit intervals confirms the model’s ability to capture the irregular cadence of healthcare.
Figure 2: Preservation of complex clinical phenotypes and syndromic co-occurrences. Co-occurrence heatmaps of the top 150 diagnosis codes in Real (left) and Synthetic (right) cohorts, with axes sorted via hierarchical clustering. High Pearson ( ) and Spearman ( ) correlations demonstrate that Coogee generates coherent clinical syndromes rather than isolated codes. Annotated regions highlight the faithful reproduction of higher-order disease mechanisms, including the Cardio-Renal Syndrome and metabolic disease progression.
| Round 1 (Unaudited) | Round 2 (Audited) | ||||||||
| Evaluator | Review Strategy | Real | Syn. | -val | Effect Size ( ) | Real | Syn. | -val | Effect Size ( ) |
| Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | ||||||
| Human review | |||||||||
| Reviewer 1 | Narrative | 7 45 (1 83) | 6 15 (2 41) | 0 07 | 0 59 | 6 45 (2 18) | 6 00 (2 59) | 0 57 | 0 18 |
| Reviewer 2 | Narrative | 6 30 (2 65) | 4 80 (2 69) | 0 09 | 0 55 | 5 50 (2 46) | 4 05 (2 33) | 0 07 | 0 59 |
| Reviewer 3 | Rule Check | 8 10 (0 99) | 5 60 (1 91) | 0 001 | 1 60 | 6 90 (1 26) | 5 80 (1 89) | 0 04 | 0 67 |
Table 1: Impact of automated auditing on clinical consistency and realism. Comparison of realism scores assigned by clinicians and LLMs before (Round 1) and after (Round 2) automated auditing. The Effect Size (Cohen’s ) quantifies the statistical separation between real and synthetic records (lower indicates higher indistinguishability). In Round 1, Rule-checking evaluators easily distinguished synthetic records ( , ). In Round 2, discrimination ability decreased significantly.
| PANEL A: downstream predictive utility | ||||||
| AUROC (95% CI) | F1-Score | Recall (Sensitivity) | ||||
| Clinical Task | Real | Synthetic | Real | Synthetic | Real | Synthetic |
| 1. Phenotyping (25-label) | 0 96 (0 95–0 96) | 0 91 (0 89–0 94) | 0 93 | 0 90 | 0 95 | 0 89 |
| 2. Mortality (Binary) | 0 76 (0 73–0 78) | 0 69 (0 65–0 72) | 0 72 | 0 70 | 0 75 | 0 98 |
| 3. Length-of-Stay (10-class) | 0 77 (0 76–0 79) | 0 74 (0 73–0 76) | 0 32 | 0 21 | 0 34 | 0 25 |
| 4. Readmission (Binary) | 0 61 (0 57–0 64) | 0 62 (0 58–0 65) | 0 51 | 0 62 | 0 44 | 0 66 |
Table 2: Downstream utility and privacy preservation. Panel A: compares the predictive performance of models trained on Real vs. Synthetic data across four clinical tasks. Panel B: quantifies privacy risks.
| Train | Validation | Test | Total | ||
| Sample size, n | Patients | 146,377 (81%) | 16,264 (9%) | 18,071 (10%) | 180,712 |
| Hospital admissions | 348,900 | 39,432 | 42,743 | 431,075 | |
| Tokens | 121,424,363 | 13,665,709 | 14,467,629 | 149,557,701 | |
| Age in years, mean (std) | - | 56 20 (20 19) | 56 21 (20 08) | 56 36 (20 27) | 56 22 (20 19) |
| Gender, n (%) | Female | 77,344 (52 8%) | 8,719 (53 6%) | 9,656 (53 4%) | 95,719 (53 0%) |
| Male | 46,202 (45 4%) | 7,545 (46 4%) | 8,415 (46 6%) | 84,993 (47 0%) |
Table S1 : Patient characteristics in the MIMIC-IV dataset used in our experiments. The data distribution is consistent across splits, ensuring comparability of training and evaluation cohorts.
Figure S1 : Generating high-fidelity, full-spectrum patient trajectories with Coogee. i , Schematic patient trajectory representation integrating multimodal information: demographics (age, sex, race, marital status, calender year), laboratory tests, diagnoses (ICD-10-CM), procedures (ICD-10-PCS), medications (ATC codes), and structural tokens that encode time gaps and delimiters for records and visits). ii , Knowledge-aware representation of clinical events: each code is linked to its ontology hierarchy (e.g., ICD structure) and mapped to textual descriptions. iii , Transformer-based generative architecture: tokenized medical events are embedded, enriched with knowledge-aware representations, and processed to model longitudinal and multimodal patient trajectories. iv , Example of a synthetic trajectory generated by Coogee: starting from demographics, the model produces diabetes-related codes including labs (glucose, creatinine), diagnoses (type 2 diabetes with CKD), medications (insulin, metformin), and associated procedures, with realistic temporal gaps between visits.
| Category | # Unique Tokens | Description / Notes |
| Patient Attributes (Static) | ||
| Age | 17 | Discretized 5-year intervals (e.g., AGE_15_20_years , …, AGE_95_100_years ) |
| Sex | 2 | Biological sex ( SEX_M and SEX_F ) |
| Race/Ethnicity | 6 | RACE_ { ASIAN, BLACK, HISPANIC, OTHER, UNKNOWN, WHITE } |
| Marital Status | 5 | MARITAL_STATUS_ { DIVORCED, MARRIED, SINGLE, UNKNOWN, WIDOWED } |
| Calendar Year | 16 | Year of admission start (from YEAR_2005 to YEAR_2020 ) |
Table S2 : Atomic tokenization and vocabulary composition. The Coogee vocabulary maps 31,901 distinct clinical concepts to single tokens to prevent code fragmentation. The schema integrates static demographics with longitudinal events, utilizing quantile bins to capture continuous laboratory values and discrete interval tokens to preserve temporal resolution.
Figure S2 : Scaling law investigation of model-data trade-offs in Coogee. (a) IsoFLOP curves showing validation loss as a function of model size (parameters) for different computational budgets. Each curve corresponds to a constant training FLOPs constraint, with dashed lines denoting quadratic fits in log–parameter space. (b) Optimal model size (minimizing validation loss per compute budget) follows a power-law relationship with total training FLOPs, consistent with Chinchilla-style scaling trends. Hoffmann et al., 2022 (c) Corresponding optimal token counts also obey a power-law with compute. Together, these analyses quantify the balance between model capacity and dataset size for efficient training. Based on the identified isoFLOP frontier and available computational resources, we selected the 50M-parameter (13M trainable) configuration for our main experiments.
| Hyperparameter | Value | Params | Trainable | Frozen |
| Vocab size | 31,901 | – | – | – |
| Context length | 2,048 | – | – | – |
| Embedding dim | 384 | – | – | – |
| Hidden dim | 1,024 | – | – | – |
| Decoder layers | 6 | – | – | – |
| Attention heads | 6 | – | – | – |
Table S3 : Architecture and parameter breakdown of the Coogee-50M model. The table details the hyperparameters, embedding dimensions, and per-component parameter counts. Of the total 50M parameters, 13.2M are trainable and 36.7M are frozen. Knowledge-aware embeddings, including structural (hierarchy and semantic) embeddings, are frozen to preserve ontology-derived structure, while trainable parameters lie primarily in the token embedding layer and decoder blocks.
| Metric | Real (Truncated) | Synthetic |
| Trajectory Structure | ||
| Visits per Patient | Median: 1 0; IQI: 1 0-2 0 | Median:1 0; IQI: 1 0-3 0 |
| Codes per Visit | Median:131 0; IQI: 46 0-316 0 | Median:124 0; IQI: 45 0-289 0 |
| Probabilistic Alignment ( ) | ||
| Unigram Probability | 0 997 | |
| Same-Visit Co-occurrence | 0 988 | |
Table S4 : Comparison of structural metrics controlled for context window limits. When the real baseline is restricted to trajectories that fit within the model’s context window ( 2,048 tokens), the synthetic data mirrors the real distributions with high precision.
Figure S3 : Preservation of population-level statistics: correlations of unigram (marginal), same-visit bigram (co-occurrence), and sequential-visit bigram (temporal) probabilities.
Figure S4 : Clinician and LLM-based realism scores in Round 1. (a) Distribution of realism scores. Narrative-focused clinicians (Reviewers 1 and 2) show significant distributional overlap between Real (blue) and Synthetic (red) records. In contrast, the rule-checking clinician (Reviewer 3) and LLMs show a clear separation, identifying logical inconsistencies. (b) Inter-rater reliability measured through Intraclass Correlation Coefficients (ICCs). The strong correlation between Reviewer 3 and LLMs (ICCs of 0 34, 0 56 and 0 61) suggests that automated auditors may focus on the validity of clinical rules within the records.
Figure S5 : Validation loss curves of Coogee with and without knowledge-informed embeddings. The plot compares the validation loss trajectories during training for two model variants: Coogee with knowledge_embd (red) and Coogee without knowledge_embd (blue). Both models share identical hyperparameters and training configurations, differing only in the inclusion of auxiliary biomedical knowledge embeddings (hierarchical and semantic). The model with knowledge integration exhibits consistently lower validation loss across training steps.
| time | code | numerical_value | code_label |
|---|---|---|---|
| 11-29 22:01 | START_RECORD | ||
| 11-29 22:01 | AGE_75_80_years | ||
| 11-29 22:01 | SEX_F | ||
| 11-29 22:01 | RACE_UNKNOWN | ||
| 11-29 22:01 | MARITAL_STATUS_MARRIED | ||
| 11-29 22:01 | YEAR_2014 |
Table S5 : A synthetic patient trajectory achieving a mean realism score of 7.7/10 in combined clinician and automated evaluation (§ 3.2 ). This synthetic record represents an elderly female undergoing Coronary Artery Bypass Grafting (CABG). Evaluators selected this trajectory for its high degree of pathophysiological coherence. Specifically, the human reviewer noted that the severe drop in hemoglobin and hematocrit was physiologically consistent with post-cardiac surgery and logically justified the subsequent diagnosis of acute posthemorrhagic anemia (D62). LLM auditors (Gemini, Qwen) similarly highlighted the highly realistic temporal trends in coagulation monitoring and the use of guideline-concordant discharge medications (e.g., aspirin, statins, beta-blockers) as key factors driving the high realism score.