Privacy-Preserving EHR Data Transformation via Geometric Operators: A Human-AI Co-Design Technical Report
Authors: Maolin Wang, Beining Bao, Gan Yuan, Hongyu Chen, Bingkun Zhao, Baoshuo Kan, Jiming Xu, Qi Shi, Yinggong Zhao, Yao Wang, Wei Ying Ma, Jun Yan
Organizations: Hong Kong Institute of AI for Science (HKAI-Sci), City University of Hong Kong · Department of Biostatistics, City University of Hong Kong · YIDU TECH co. Ltd. · Institute of Digital Medicine, City University of Hong Kong
Electronic health records (EHRs) and other real-world clinical data are essential for clinical research, medical artificial intelligence, and life science, but their sharing is severely limited by privacy, governance, and interoperability constraints. These barriers create persistent data silos that hinder multi-center studies, large-scale model development, and broader biomedical discovery. Existing privacy-preserving approaches, including multi-party computation and related cryptographic techniques, provide strong protection but often introduce substantial computational overhead, reducing the efficiency of large-scale machine learning and foundation-model training. In addition, many such methods make data usable for restricted computation while leaving them effectively invisible to clinicians and researchers, limiting their value in workflows that still require direct inspection, exploratory analysis, and human interpretation. We propose a real-world-data transformation framework for privacy-preserving sharing of structured clinical records. Instead of converting data into opaque representations, our approach constructs transformed numeric views that preserve medical semantics and major statistical properties while, under a clearly specified threat model, provably breaking direct linkage between those views and protected patient-level attributes. Through collaboration between computer scientists and the AI agent SciencePal, acting as a constrained tool inventor under human guidance, we design three transformation operators that are non-reversible within this threat model, together with an additional mixing strategy for high-risk scenarios, supported by theoretical analysis and empirical evaluation under reconstruction, record linkage, membership inference, and attribute inference attacks.
Figures & tables
Figure 1: SciencePal co-design protocol for geometric operator families. Step 1 : Humans specify operator clauses C1–C4 and threat model C5. Step 2 : SciencePal searches PPDP/PPDM and related methods, finding no operator that satisfies all clauses. Step 3 : SciencePal proposes candidate operators T1, T2, and T3. Step 4 : Humans prove properties and run attack-based evaluation, keeping T1/T2 and rejecting T3. Step 5 : Observing residual high reconstruction risk for some variables, humans and SciencePal co-design per-stay Q-mix extensions, yielding the final operator family.
Figure 2: Overview of the column-wise geometric operators studied in this work. T1 applies local triplet rotations on M(0,1) , preserving mean/variance and short-range autocorrelation with moderate privacy. T2 adds bounded z-score noise with global ℓ∞ limit α , then recenters to M(0,1) , giving stronger privacy while largely preserving marginals and correlations. T3 is a global Householder reflection that preserves low-order statistics but is highly invertible ( R2≈1 ), and serves as a negative control. Q-mix wraps T1/T2 with per-stay orthogonal mixing on selected high-risk variables, keeping fixed while sharply reducing reconstruction risk and retaining acceptable utility.
Setting
Operator
Δμmax
Δσmax
∣δ∣max
∣δ∣max,max
Unchanged max
n
phys
T1_uniform
5.7×10−14
9.1×10−13
0.999975
0.9999993
0.0000
30
phys
T1_weighted
5.7×10−14
9.1×10−13
0.999974
0.9999995
0.0098
30
phys
T2
2.8×10−14
9.1×10−13
0.91211
0.99322
0.0000
10
phys
T3
4.3×10−14
4.6×10−13
0.25057
0.72993
0.0000
10
z
T1_uniform
1.8×10−17
5.6×10−16
0.74980
0.9999991
0.0000
60
z
T1_weighted
2.0×10−17
8.9×10−16
0.74956
0.9999997
0.0079
60
Table 1: Summary of geometric sanity statistics by operator and evaluation setting. Columns Δμmax and Δσmax report the maximum absolute differences in column-wise mean and standard deviation across variables. The remaining columns summarize aggregate ℓ∞ behavior and the maximum fraction of unchanged points. All values are computed over the training cohort and all α values used in the geometric experiments.
Figure 3: Geometric sanity heatmaps. Left: log 10 of the maximum absolute mean deviation Δμmax by setting (phys vs z) and operator. Right: max_abs_delta_mean by setting and operator. All geometric operators keep mean deviations at or below machine precision.
Figure 4: Boxplots of ∣Δμv∣ and ∣Δσv∣ grouped by evaluation setting. Deviations in standardized space are essentially at numerical precision; in physical space they remain on the order of 10−13 – 10−14 .
Figure 5: Average z-score ℓ∞ perturbation ∥δs,v∥∞ by operator for two target bounds α=0.5 and α=1.0 . For all T1/T2/T3 operators the empirical maximum ∥δs,v∥∞ never exceeds the configured α ; the average perturbation typically lies in the 0.7 – 0.9α range.
Figure 6: Mean and standard deviation of KSv across variables for different operators at α=0.8 (lower is better distributional fidelity). All three operators keep KS distances small; T3 is the most conservative, followed by T2 and T1.
Figure 7: Mean KSv by variable and operator at α=0.8 . For most vitals and labs, T1/T2 keep KS distances below 0.1 ; heavy-tailed labs experience slightly larger shifts but remain within approximately 0.15 . T3 is uniformly closest to the original distribution.
Figure 8: Marginal distributions of SBP, HR, and creatinine in standardized space. For each variable we show the histogram of z and z′ and the Q–Q plot under T1/T2/T3 at α=0.8 . All three operators largely preserve the non-Gaussian features of the marginals.
Figure 9: Out-of-range rates OORv (%) for HR and glucose at α=1.0 with Q-mix enabled. For HR the OOR remains below 0.7% across operators; for glucose it is around 3% , comparable to the original heavy-tailed distribution.
Figure 10: Scatter plots of key variable pairs in standardized space. Top: HR vs SBP (original, T1, T2, T3). Bottom: SBP vs DBP (original, T1, T2, T3). All operators preserve the overall correlation pattern; T3 is almost visually indistinguishable from the original.
Figure 11: ACF and PACF of HR and SBP in standardized space for the original data and after applying T1/T2/T3 at α=1.0 . T1/T2 introduce mild attenuation in higher lags but retain the overall temporal dependence shape; T3 leaves ACF/PACF almost unchanged.
Figure 12: Ljung–Box p-values for HR (left) and SBP (right) at lags K∈{5,10,20} in standardized space (log scale). All operators preserve highly significant autocorrelation patterns, confirming that time-series structure is retained.
Figure 13: Welch power spectral density estimates for HR (left) and SBP (right) in standardized space. T1/T2 slightly reshape spectral energy at some frequencies, while T3 is nearly identical to the original.
Figure 14: Attack A reconstruction R2 vs privacy strength α for HR in standardized space under L2 (20% pairs, linear attacker). Larger α leads to lower R2 for T1/T2, while T3 remains almost perfectly invertible.
operator
α=1.0
α=2.0
α=3.0
α=5.0
T1_uniform
0.8192
0.7560
0.7555
0.7570
T2
0.9738
0.9026
0.8655
0.7841
T3
0.999997
0.999997
0.999997
0.999997
Table 2: Attack A reconstruction R2 for HR in standardized space under L2 (20% pairs, linear attacker) as a function of α .
operator
α=1.0
α=2.0
α=3.0
T1_uniform
0.8598
0.7943
0.7757
T2
0.9601
0.9053
0.8699
T3
0.999984
0.999984
0.999984
Table 3: Attack A reconstruction R2 for glucose in standardized space under L2 (20% pairs, linear attacker).
Figure 15: Attack A (L2, linear) reconstruction R2 for HR (left) and glucose (right) at α=1.0 , with and without Q-mix. Q-mix drives R2 from approximately 0.8 – 0.98 down to approximately 0 .
variable
operator
R2
MAEz
No Q-mix
HR
T1_uniform
0.8308
0.3267
HR
T1_weighted
0.8288
0.3219
HR
T2
0.9734
0.1278
Glucose
T1_uniform
0.9592
0.1437
Glucose
T1_weighted
0.8769
0.2688
Table 4: Attack A reconstruction R2 and z-score MAE MAEz at α=1.0 for HR and glucose under L2 (20% pairs, linear attacker), with and without Q-mix.
Figure 16: Attack B (record linkage) Re-id@1 at α=1.0 , candidate set size m=10 , for HR and glucose with Q-mix. All values are close to the random baseline 1/m=0.1 .
Figure 17: Attack C (membership inference) ROC AUC at α=1.0 for HR, with and without Q-mix. Values remain close to 0.5 , implying negligible advantage.
Figure 18: Attack D (attribute inference) R2 for HR at α=1.0 , with and without Q-mix. Q-mix dramatically reduces attribute leakage, especially for max and above_p90.
Figure 19: Attack D (distinguishability) IND advantage ∣acc−0.5∣ for HR at α=1.0 , with and without Q-mix. Values remain modest, and accuracies stay close to 0.5 .
Figure 20: Shape utility scores utility_shape=1−KS for HR and glucose at α=1.0 with Q-mix. Scores remain high, indicating that marginal distributions are preserved reasonably well.
Figure 21: Utility for LOS prediction using HR or glucose at α=1.0 , with and without Q-mix. Bars show differences in AUROC and AUPRC between perturbed and raw data. For HR, Q-mix + T1/T2 slightly improves AUROC and AUPRC.
Figure 22: Privacy–utility trade-off for HR at α=1.0 . The x-axis shows privacy score 1−R2 , and the y-axis shows change in AUROC for LOS prediction. Q-mix + T1/T2 achieve both high privacy and non-degraded utility.
Figure 23: System-level overview of the in-hospital nightly privacy pipeline and geometric privacy engine. Each night, raw EHR tables (ICU vitals, labs, medications in our running example) are exported from the production warehouse into a harmonized, hourly-gridded view. Given a threat model and a chosen privacy skill configuration, the geometric privacy engine applies column-wise operators (T1/T2/T3 and per-stay Q-mix) within a CPU-only pipeline to produce a secure numeric cache Dpriv . Privacy skills and usage profiles distinguish low-privacy, high-utility views for in-hospital QA and research from strong-privacy export views with Q-mix enabled on selected variables. An LLM-based EHR-Privacy-Agent and the SciencePal assessment loop sit on top of these views to support interactive analysis, monitoring of privacy and shape metrics, and collaborative tuning of skills, while all data remain on-premise and never leave the hospital intranet.
Figure 24: Example YAML skill definitions for two deployment scenarios. (a) A typical in-hospital research profile with moderate privacy and no Q-mix. (b) A conservative export profile with higher α and Q-mix enabled for high-risk variables.
Figure 25: Left: geometric numeric operators T1–T3 applied to one standardized vital - sign column, with toy before/after values showing mean–variance–preserving local rotation (T1), stronger scrambling via bounded noise plus manifold projection (T2), and an almost invertible Householder reflection as a negative privacy case (T3). Right: auxiliary operators for time/indexing, IDs and demographics, counts and tabular synthesis, categories/knowledge graph, and clinical text/PHI, each illustrated with minimal before/after snippets to show how identifiers are removed or blurred while keeping data usable for analysis.
Figure 26: Example ICU prediction performance (24h features) on Raw vs a de-identified view. Both AUROC and AUPRC remain very close, illustrating the near-lossless utility of medium- α geometric transforms in in-hospital settings.
Figure 27: ICU LOS 3-class prediction. Macro-AUROC and macro-AUPRC for Raw, T1+T2@α=0.5 .
View
Macro-AUROC
Macro-AUPRC
Macro-F1
Raw
≈0.70
≈0.44
≈0.40
T1+T2@ α=0.5
≈0.69 ( −0.01 )
≈0.44 ( ±0.00 )
≈0.39 ( ±0.01 )
Gaussian noise
≈0.65 – 0.67
≈0.40 – 0.41
lower F1, especially for long LOS
CTGAN synthetic
≈0.63 – 0.65
≈0.37 – 0.39
notable degradation on long LOS recall
Table 5: LOS prediction (3-class, 48h window, GRU model). Approximate macro metrics on the test set.
Figure 28: Numeric theory constraints in a strong numeric pipeline: absolute mean/variance deviations, maximum absolute perturbation, and unchanged ratio. Deviations remain at machine precision, confirming C1–C2 in large-scale runs.
Figure 29: ID/time validation across key identifiers and timestamps in the pipeline. All checks pass, indicating that the numeric pipeline preserves primary/foreign-key and temporal consistency.
Figure 30: Numeric operator scaling with column length N on log–log axes. All core operators scale approximately linearly in N ; the end-to-end pipeline follows the same trend.
Figure 31: Runtime comparison: geometric pipeline vs CTGAN. Bars show total time (log scale) for CPU-only geometric pipelines at different privacy levels versus CTGAN training and sampling on GPU/CPU.
Feature engineering for Electronic Health Records (EHR) is complicated by irregular observation intervals, variable measurement frequencies, and structural sparsity inherent to clinical time series. Existing automated methods either lack clinical domain awareness or assume clean, regularly sampled inputs, limiting their applicability to real-world EHR data. We present FeatEHR-LLM, a framework that leverages Large Language Models (LLMs) to generate clinically meaningful tabular features from irregularly sampled EHR time series. To limit patient privacy exposure, the LLM operates exclusively on dataset schemas and task descriptions rather than raw patient records. A tool-augmented generation mechanism equips the LLM with specialized routines for querying irregular temporal data, enabling it to produce executable feature-extraction code that explicitly handles uneven observation patterns and informative sparsity. FeatEHR-LLM supports both univariate and multivariate feature generation through an iterative, validation-in-the-loop pipeline. Evaluated on eight clinical prediction tasks across four ICU datasets, our framework achieves the highest mean AUROC on 7 out of 8 tasks, with improvements of up to 6 percentage points over strong baselines. Code is available at github.com/hojjatkarami/FeatEHR-LLM.
Hojjat Karami, David Atienza, Jean-Philippe Thiran +1
1École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
The development of medical AI is constrained by limited access to high-quality clinical data due to institutional silos and strict privacy regulations such as HIPAA and GDPR. Synthetic data generation offers a potential solution, but existing methods lack principled mechanisms to explicitly manage the privacy-utility trade-off, often degrading clinically meaningful patterns or risking patient re-identification. We present PSyGenTAB, a privacy-preserving generative framework that formulates synthetic healthcare data generation as a constrained optimization problem solved using the Augmented Lagrangian Method. By embedding configurable privacy constraints directly into model training, PSyGenTAB enforces minimum privacy thresholds while maximizing clinical data utility. Across multiple clinically motivated benchmarks, PSyGenTAB preserves inter-feature clinical relationships and minority-class diagnostic patterns essential for reliable health AI. Downstream evaluation using Train-on-Synthetic, Test-on-Real and Train-on-Real, Test-on-Synthetic protocols shows that models trained on synthetic data achieve performance comparable to those trained on real patient records. Privacy auditing further demonstrates reduced exact record reproduction and strong resilience to membership inference attacks. These results establish PSyGenTAB as a principled framework for balancing privacy protection and clinical utility in synthetic healthcare data, supporting secure cross-institutional AI development.
Arshia Ilaty, Hossein Shirazi, Manasi Chitale +5
San Diego State University, San Diego, CA 92182 USA. · University of California, Irvine, CA 92697 USA.
Reasoning over privacy-constrained records requires combining structured attributes with evidence from free-text narratives. We introduce CIPHER (Cross-record Inference over Privacy-Hardened Evidence Records), a benchmark of expert-validated questions from consumer-finance, clinical, and law-enforcement records. The questions cover common tabular operations and include executable SQL supervision. We evaluate retrieval, prompting, table-specialist, and hybrid symbolic-neural systems under native redaction and surrogate-based evidence restoration. All system families exhibit substantial failures even when supporting records are provided. Most errors arise from incorrect record selection and predicate interpretation rather than arithmetic execution. Privacy transformations have non-uniform effects, sometimes obscuring necessary evidence and sometimes reducing distraction. CIPHER provides a reproducible testbed for diagnosing these failures and assessing how transformations of sensitive text affect reasoning over hybrid records.
Suparno Roy Chowdhury, Manan Roy Choudhury, Dhruv Madhwal +1