Knowledge-Guided Retrieval-Augmented Generation for Zero-Shot Psychiatric Data: Privacy Preserving Synthetic Data Generation

Adam Jakobsen; Sushant Gautam; Hugo Lewi Hammer; Susanne Olofsdotter; Miriam S Johanson; Pål Halvorsen; Vajira Thambawita

cs.LGMar 26, 2026

Knowledge-Guided Retrieval-Augmented Generation for Zero-Shot Psychiatric Data: Privacy Preserving Synthetic Data Generation

Authors: Adam Jakobsen, Sushant Gautam, Hugo Lewi Hammer, Susanne Olofsdotter, Miriam S Johanson, Pål Halvorsen, Vajira Thambawita

Organizations: ∗SimulaMet, Norway · †Oslo Metropolitan University, Norway · ‡Uppsala University, Sweden

Abstract

AI systems in healthcare research have shown potential to increase patient throughput and assist clinicians, yet progress is constrained by limited access to real patient data. To address this issue, we present a zero-shot, knowledge-guided framework for psychiatric tabular data in which large language models (LLMs) are steered via Retrieval-Augmented Generation using the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and the International Classification of Diseases (ICD-10). We conducted experiments using different combinations of knowledge bases to generate privacy-preserving synthetic data. The resulting models were benchmarked against two state-of-the-art deep learning models for synthetic tabular data generation, namely CTGAN and TVAE, both of which rely on real data and therefore entail potential privacy risks. Evaluation was performed on six anxiety-related disorders: specific phobia, social anxiety disorder, agoraphobia, generalized anxiety disorder, separation anxiety disorder, and panic disorder. CTGAN typically achieves the best marginals and multivariate structure, while the knowledge-augmented LLM is competitive on pairwise structure and attains the lowest pairwise error in separation anxiety and social anxiety. An ablation study shows that clinical retrieval reliably improves univariate and pairwise fidelity over a no-retrieval LLM. Privacy analyses indicate that the real data-free LLM yields modest overlaps and a low average linkage risk comparable to CTGAN, whereas TVAE exhibits extensive duplication despite a low k-map score. Overall, grounding an LLM in clinical knowledge enables high-quality, privacy-preserving synthetic psychiatric data when real datasets are unavailable or cannot be shared.

Figures & tables

Fig. 1: Pipeline of the proposed framework. Synthetic patients are generated by conditioning a Large Language Model (LLM) on sampled personas and clinical knowledge retrieved via Retrieval-Augmented Generation (RAG). Outputs are tabular datasets evaluated for fidelity and privacy against baselines. Abbreviations: LLM, Large Language Model; RAG, Retrieval-Augmented Generation.

Disorder	Method	JSD $\downarrow$	$\mathbf{MAE}_{V}$ $\downarrow$	ED 2 $\downarrow$
Agoraphobia	CTGAN	0.033 [0.025, 0.042]	0.249 [0.218, 0.279]	0.033 [0.024, 0.043]
	TVAE	0.070 [0.060, 0.081]	0.159 [0.129, 0.192]	0.061 [0.049, 0.075]
	DSM-V	0.082 [0.072, 0.094] *	0.200 [0.132, 0.246] *	0.097 [0.084, 0.112] *
	Dual-KB	0.083 [0.072, 0.095]	0.207 [0.169, 0.246]	0.097 [0.084, 0.113] *
	ICD-10	0.094 [0.083, 0.106]	0.212 [0.124, 0.269]	0.101 [0.089, 0.116]
	No-KB	0.101 [0.087, 0.117]	0.250 [0.139, 0.300]	0.110 [0.094, 0.129]

TABLE I: Fidelity with 95% bootstrap confidence intervals. Best results in bold .

Disorder	Method	$\Delta$ $\uparrow$	$\Delta$ MAE V $\uparrow$	$\Delta$ ED 2 $\uparrow$
Agoraphobia	ICD-10	0.008 [-0.008, 0.024]	0.038 [-0.090, 0.141]	0.009 [-0.009, 0.029]
	DSM-V	0.020 [0.005, 0.035]	0.050 [-0.067, 0.131]	0.014 [-0.004, 0.032]
	Dual-KB	0.019 [0.004, 0.034]	0.043 [-0.064, 0.093]	0.013 [-0.005, 0.031]
Generalized Anxiety	ICD-10	0.035 [0.015, 0.056]	0.032 [-0.009, 0.097]	0.031 [0.004, 0.059]
	DSM-V	0.029 [0.009, 0.050]	0.028 [-0.003, 0.061]	0.022 [-0.005, 0.049]
	Dual-KB	0.028 [0.008, 0.048]	0.071 [0.030, 0.109]	0.022 [-0.004, 0.048]

TABLE II: Improvements over No Knowledge Base ( No-KB ) with paired 95% Confidence Intervals (CIs). Positive values indicate gains from DSM-V, ICD-10, or Dual-KB retrieval. Best per metric in bold .

Disorder	Method	Exact Overlap $\downarrow$	$q_{0.05}\{d_{\mathrm{NN}}\}$ $\uparrow$	Share( $d_{\mathrm{norm}}\!\leq\!1/p$ ) $\downarrow$	$\mathrm{Risk}_{\text{k-map}}$ $\downarrow$
Agoraphobia	Random	0.000	0.333	0.000	0.022
	CTGAN	0.004	0.083	0.078	0.026
	ICD-10	0.296	0.000	0.599	0.025
	Dual-KB	0.218	0.000	0.493	0.020
	DSM-V	0.206	0.000	0.498	0.032
	No-KB	0.245	0.000	0.502	0.022

TABLE III: Privacy metrics. Best per metric within each disorder is bold .

Explore similar work

Apr 29, 2026

Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation

The scarcity of high-quality annotated medical data, particularly in mental health, poses a significant bottleneck for training robust machine learning models. Privacy regulations restrict data sharing, making synthetic data generation a promising alternative. The use of Large Language Models (LLMs) in a data augmentation pipeline could be leveraged as an alternative in this field. In the proposed methodology, DeepSeek-R1, OpenBioLLM-Llama3 and Qwen 3.5 are used to generate synthetic mental health evaluation reports conditioned on specific International Classification of Diseases, Tenth Revision (ICD-10) codes. Because naive text generation can lead to mode collapse or privacy breaches (memorization), a comprehensive evaluation framework is introduced. The generated diagnostic texts are assessed across three dimensions: semantic fidelity, lexical diversity, and privacy/plagiarism. The results demonstrate that all models can generate clinically coherent, diverse, and privacy-safe synthetic reports, significantly expanding the available training data for clinical natural language processing tasks without compromising patient confidentiality.

Guillermo Iglesias, Gema Bello-Orgaz, María Navas-Loro +3

Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain · Department of Psychiatry, University Hospital Rey Juan Carlos, Mostoles, Spain · Department of Psychiatry, University Hospital Jimenez Diaz Foundation, Madrid, Spain +5

Synthetic Data Augmentation Synthetic Data Generation Privacy Leakage in Language Models Medical LLMs Synthetic Data Evaluation

Ask PDF

Jun 16, 2026

PSyGenTAB: A Privacy-Preserving Framework for Synthetic Clinical Tabular Data Generation via Constrained Optimization

The development of medical AI is constrained by limited access to high-quality clinical data due to institutional silos and strict privacy regulations such as HIPAA and GDPR. Synthetic data generation offers a potential solution, but existing methods lack principled mechanisms to explicitly manage the privacy-utility trade-off, often degrading clinically meaningful patterns or risking patient re-identification. We present PSyGenTAB, a privacy-preserving generative framework that formulates synthetic healthcare data generation as a constrained optimization problem solved using the Augmented Lagrangian Method. By embedding configurable privacy constraints directly into model training, PSyGenTAB enforces minimum privacy thresholds while maximizing clinical data utility. Across multiple clinically motivated benchmarks, PSyGenTAB preserves inter-feature clinical relationships and minority-class diagnostic patterns essential for reliable health AI. Downstream evaluation using Train-on-Synthetic, Test-on-Real and Train-on-Real, Test-on-Synthetic protocols shows that models trained on synthetic data achieve performance comparable to those trained on real patient records. Privacy auditing further demonstrates reduced exact record reproduction and strong resilience to membership inference attacks. These results establish PSyGenTAB as a principled framework for balancing privacy protection and clinical utility in synthetic healthcare data, supporting secure cross-institutional AI development.

Arshia Ilaty, Hossein Shirazi, Manasi Chitale +5

San Diego State University, San Diego, CA 92182 USA. · University of California, Irvine, CA 92697 USA.

Synthetic Tabular Data Generation Tabular Data Synthesis Privacy-Utility Tradeoff

Ask PDF

May 18, 2026

Memisis: Orchestrating and Evaluating Synthetic Data for Tabular Health Datasets

Synthetic data is widely used in healthcare to create datasets that preserve statistical properties of real data without exposing sensitive patient information. Generating and evaluating synthetic data across privacy, utility, and fairness dimensions is crucial for enabling high-quality data availability in downstream prediction tasks and clinical decision making. We present Memisis, a tool that orchestrates and evaluates synthetic data by leveraging existing synthesis libraries, large language models (LLMs), and state-of-the-art evaluation metrics. Our tool creates a unified workflow for data generation, validation, and evaluation. Users can control training size, training epochs, and the number of synthetic rows to sample. Beyond manual configuration, an interactive agent mode allows users to specify data generation goals in natural language, and the tool orchestrates the full pipeline by invoking existing synthesizers while performing the requisite evaluation. For the demo, we use an open-source schizophrenia dataset with protected attributes related to race and gender, evaluate six synthesizers spanning GANs, VAEs, diffusion models, and normalizing flows, and use a local LLM to orchestrate the workflow. The system affords users flexibility and control over the data generation and evaluation process.

Nitish Nagesh, Pengbao Zhou, Atchuth Naveen Chilaparasetti +8

University of California, Irvine, CA, USA · PathAI, USA

Healthcare Synthetic Tabular Data Generation Privacy-Preserving ML Synthetic Data Evaluation

Ask PDF