Dissecting Neuro-Symbolic Quality Assurance for Synthetic Oncology Data Generation

Laxmigayathri Challa; Yuhan Zhou; Ana Cleveland; Haihua Chen

cs.AIAug 22, 2026

Dissecting Neuro-Symbolic Quality Assurance for Synthetic Oncology Data Generation

Authors: Laxmigayathri Challa, Yuhan Zhou, Ana Cleveland, Haihua Chen

Organizations: Department of Information Science, University of North Texas, Denton, TX 76203 USA · The Anuradha and Vikas Sinha Department of Data Science, University of North Texas, Denton, TX 76203 USA

Abstract

Synthetic clinical data generation with large language models addresses the scarcity that limits cancer staging research, but oncology hallucinations are categorically harmful: one clinically impossible staging assignment contaminates every downstream model trained on it. Neuro-symbolic pipelines validate during generation, yet the contribution of individual quality-assurance components remains unclear. We report three controlled studies isolating gate necessity, constraint attribution, and retrieval conditionality, holding generation protocol, diversity thresholds, and fine-tuning hyperparameters constant across adapter conditions. The symbolic gate enforces schema completeness, ontology coverage against the Systematized Nomenclature of Medicine, and staging-logic consistency under American Joint Committee on Cancer eighth-edition rules. Ungated, 29.9% of records contain schema failures and 20.1% contain clinically invalid staging. Schema validation is the load-bearing filter: within the fully gated corpus it rejects 148 of 512 records, ontology grounding a further 24, and staging-logic validation none---the only generator producing logic violations is already excluded on schema, making clinical-logic validation a generator-conditional safeguard rather than the dominant filter. Retrieval augmentation is strongly model-dependent: it improves gate compliance for one generator by 12.5 percentage points, has no measurable effect for a second, and collapses output in a third. Across gated configurations ontology density is largely unchanged, indicating that symbolic validation improves clinical validity rather than vocabulary richness. Symbolic gating therefore buys corpus validity but no commensurate gain on real lung-cancer notes in this study; retrieval should be evaluated per model, and ontology density should not be reported as a proxy for corpus quality.

Figures & tables

Fig. 1 : Demonstration of different synthetic data generation methods.

Fig. 2 : Neuro-symbolic generation pipeline and its component dissection. Top (target pipeline). The full framework couples LLM generation with a neuro-symbolic gate $G(x)=S\wedge O\wedge C$ —schema validity ( $S$ ), SNOMED CT ontology coverage ( $O$ ), and AJCC clinical-logic consistency ( $C$ )—and, in the retrieval condition, prepends MedCPT-retrieved PubMed context to the generation prompt as grounding before the gate is applied. Records that pass the gate enter the synthetic corpus as structured rigid.v3 fields plus a free-text clinical narrative. Bottom (dissection of components). Five matched conditions (Table II ) remove or add one component at a time: ungated baseline (Adapter A); schema only (Adapter B); schema $+$ ontology (Adapter C); full gate, no retrieval (Adapter D); and full gate $+$ RAG (Adapter E). The three within-gate conditions (B, C, D) isolate the marginal contribution of each constraint (RQ 2); A versus D isolates gate necessity (RQ 1); and D versus E isolates retrieval conditionality (RQ 3). The three generators (GPT-4o, Llama-3.3-70B, ClinicalCamel-70B), the 32-cell TNM seeding grid, the entropy floors, the QLoRA hyperparameters, and the TSTR evaluation protocol are held fixed across every condition (Sections IV-B – IV-D ), so that any measured difference is attributable to the manipulated component alone.

Adapter	Gate condition	RAG	Ablation
A	None (ungated)	No	1
B	$S$ only	No	2
C	$S\wedge O$	No	2
D	$S\wedge O\wedge C$	No	1, 2, 3
E	$S\wedge O\wedge C$	Yes	3

TABLE II : Adapter conditions and ablation mapping. Adapter D is the shared full-gate, no-RAG baseline across all three ablations. Per-model run counts differ by construction: GPT-4o contributes 128 generations in every condition, 64 API calls having returned no output; Llama-3.3-70B and ClinicalCamel-70B contribute 192 each (no-RAG) and 160 each (RAG). Totals are 512 per no-RAG condition and 448 for the RAG condition. All adapters use identical QLoRA configuration (Appendix C ).

Adapter	Condition	Gen.	Yield	Schema	Onto.	Logic	SNOMED/100w	Unique
A	Ungated	512	100.0%	70.1%	100.0%	79.9%	28.39	213
B	Schema only	512	70.5%	70.5%	100.0%	78.1%	29.71	192
C	Schema + Onto.	512	70.3%	70.3%	100.0%	77.0%	29.86	201
D	Full $G(x)$ , No-RAG	512	66.4%	71.1%	95.3%	77.1%	29.83	198
E	Full $G(x)$ + RAG	448	69.2%	70.3%	83.3%	71.0%	29.52	191

TABLE III : Corpus quality summary by ablation condition. Yield = admitted / generated. Schema, Onto., and Logic pass rates are computed over all generated records. All entropy floors (

H_{T},H_{N}\geq 1.109

;

H_{M}\geq 0.554

) pass in every condition. SNOMED density is length-normalized (terms per 100 words) over admitted records. Unique = distinct SNOMED concept IDs across all admitted records.

Fig. 3 : Gate vs. No-Gate. (a) Stacked gate failure attribution across all five conditions. Column A (Ungated) contains 29.9% schema failures and 20.1% logic failures that would silently enter fine-tuning without the gate. Column D (Full

G(x)

) shows the residual failure structure after all three constraints are applied; admitted records pass all three simultaneously. (b) Individual gate component pass rates. Ontology coverage reaches 100% for conditions A, B, and C by design (the schema mandates histology codes); the full gate (D, E) applies all three constraints independently, and the residual ontology failures at D are the 24 Llama records identified in Fig. 4 .

Model	Condition	Schema	Onto.	Logic	Yield
GPT-4o	Ungated	100.0%	100.0%	100.0%	100.0%
GPT-4o	Full gate	100.0%	100.0%	100.0%	100.0%
Llama-3.3-70B	Ungated	100.0%	100.0%	100.0%	100.0%
Llama-3.3-70B	Full gate	100.0%	87.5%	100.0%	87.5%
ClinicalCamel	Ungated	20.3%	100.0%	46.4%	100.0%
ClinicalCamel	Full gate	22.9%	100.0%	39.1%	22.9%

TABLE IV : Ablation 1 gate metrics by model. Schema, Onto., and Logic are pass rates over all generated records. Yield for the ungated condition is 100% by construction (no gate applied); for the full-gate condition it equals the fraction of records satisfying all three constraints simultaneously.

Fig. 4 : Gate decomposition (within-corpus). (a) Gate funnel within the full-gate corpus: of 512 generated records, schema admits 364; ontology grounding then removes 24, leaving 340; and AJCC clinical-logic validation removes none. The marginal rejections—schema 148 (all ClinicalCamel), ontology 24 (all Llama), logic 0—identify schema as the load-bearing filter. (b) SNOMED density by model across gate levels; flat for GPT-4o and Llama (

\approx

30 terms/100w), lower for ClinicalCamel (

\approx

25), confirming the gate filters validity, not vocabulary.

Model	Gate level	Schema	Onto.	Logic	Yield
GPT-4o	$S$	100.0%	100.0%	100.0%	100.0%
GPT-4o	$S\wedge O$	100.0%	100.0%	100.0%	100.0%
GPT-4o	$S\wedge O\wedge C$	100.0%	100.0%	100.0%	100.0%
Llama-3.3-70B	$S$	100.0%	100.0%	100.0%	100.0%
Llama-3.3-70B	$S\wedge O$	100.0%	100.0%	100.0%	100.0%
Llama-3.3-70B	$S\wedge O\wedge C$	100.0%	87.5%	100.0%	87.5%

TABLE V : Ablation 2 gate metrics by model across all three gate levels. GPT-4o and Llama-3.3-70B maintain near-perfect compliance at every level; ClinicalCamel-70B’s compliance (21–23%) is unchanged by which constraints are active; its logic violations co-occur with schema failures, so they are removed by the schema constraint rather than contributing marginal logic-gate rejections.

Fig. 5 : RAG vs. No-RAG. (a) Per-run SNOMED density scatter. GPT-4o and Llama points cluster symmetrically around the diagonal; ClinicalCamel triangles collapse to the

y{=}0

and

y{=}15.75

floor under RAG. (b) Distribution of per-run density delta (RAG

-

No-RAG) by model. GPT-4o centered near zero (two-sided

p=0.498

, ns); Llama a small but significant negative shift (

p=0.002

); ClinicalCamel strongly negative (

p<0.001

), reflecting retrieval collapse. (c) Gate pass rate by model. RAG lifts Llama gate compliance from 87.5% to 100%; ClinicalCamel falls from 22.9% to 13.8%.

Fig. 6 : ClinicalCamel-70B RAG Collapse. (a) RAG SNOMED density per run colored by gate outcome (green = PASS, red = FAIL). Runs 0–84 show mixed outcomes with densities around and below 15.75; from run 85 onward, density collapses to exactly 0.0 in nearly all runs, corresponding to complete output failure. (b) No-RAG (circles) versus RAG (crosses) density paired by run. No-RAG density is stable throughout (20–33 terms/100w); RAG density disintegrates in the second half of the run sequence. 75/160 runs produce zero SNOMED output and 113/160 produce density

\leq 15.75

under RAG.

Fig. 7 : Downstream TSTR by axis and cohort. Cell color is accuracy minus the majority-class baseline (blue clears the baseline, red falls below); the printed value is raw accuracy, and the bottom row gives the baseline per column. In-distribution (Synthetic) every adapter clears the baseline on all three axes; on real TCGA accuracy collapses toward the majority class, especially lung N-stage. On Cross-Tumor, ontology-aware adapters C and D most decisively clear the N-stage baseline. Per-axis macro-F1 is reported in Table VII .

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Adapter	$H_{T}$	$H_{N}$	$H_{M}$
A–E (all)	1.39	1.39	0.69
Floor (0.8 $\times$ max)	1.109	1.109	0.554

Appendix

TABLE VI : Label diversity (Shannon entropy, nats) of the admitted corpus per TNM axis. All conditions sit at the theoretical maxima (

\ln 4=1.386

for T,N;

\ln 2=0.693

for M), well above the 80% floors, because the 32-cell seed grid balances labels by construction.

	Synthetic			TCGA Lung			Cross-Tumor
Adapter	T	N	M	T	N	M	T	N	M
A	0.56	0.60	0.97	0.41	0.33	0.53	0.56	0.53	0.67
B	0.50	0.61	0.97	0.40	0.33	0.59	0.58	0.56	0.66
C	0.48	0.59	0.97	0.41	0.41	0.59	0.59	0.64	0.67
D	0.50	0.62	0.97	0.42	0.39	0.58	0.59	0.60	0.66
E	0.53	0.61	0.97	0.44	0.33	0.58	0.57	0.55	0.69

Appendix

TABLE VII : Per-axis macro-F1 for all five adapters under TSTR, across the three evaluation cohorts. Macro-F1 weights each class equally, making it sensitive to minority-class recovery in a way aggregate accuracy is not—particularly for M-stage, where the M0 majority inflates accuracy while macro-F1 stays low.

Setting	Value
Base model and Quantization
Base model	meta-llama/Meta-Llama-3-8B-Instruct
quantization	4-bit NF4, double quantization
Compute dtype	bfloat16
LoRA
Rank ( $r$ )	16

Appendix

TABLE VIII : QLoRA fine-tuning configuration, identical across all five adapters (A–E).

Synthetic Clinical Data Generation

See all 22 papers

Jun 24, 2026

OncoSynth: Synthetic data generation for treatment effect estimation in oncology

In oncology, access to patient-level data is often restricted. Synthetic data provides an alternative for analyzing treatment effectiveness, but existing methods for synthetic data generation fail to preserve the causal relationships between covariates, treatments, and outcomes, thereby leading to biased estimates of treatment effects. Here, we introduce OncoSynth, a generative, causally-aware machine learning framework designed to produce synthetic cohorts that enable accurate estimation of population- and patient-level treatment effects. OncoSynth uses a diffusion-based sequential approach to model how covariates influence treatment assignment and how treatment affects survival. We evaluate OncoSynth using large lung (N = 37,128) and breast cancer (N = 17,046) cohorts. Our results show that OncoSynth generates high-fidelity synthetic patient cohorts that preserve real-world patient, treatment, and outcome distributions. Notably, OncoSynth improves treatment effect estimation over existing approaches, by reducing population-level treatment effect error by up to 66%, and patient-level treatment effect error by up to 58%. Thereby, OncoSynth supports reliable evidence generation for precision oncology in settings where data sharing is restricted.

Octavia-Andreea Ciora, Julian Welzel, Dennis Frauen +6

1LMU Munich, Munich, Germany · 2Munich Center for Machine Learning (MCML), Munich, Germany · Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom +3

Causal Effect Estimation Synthetic Data Generation Synthetic Tabular Data Generation Oncology Individualized Treatment Effect Estimation

Ask PDF

Jun 8, 2026

Synthetic but Not Realistic: The Evaluation Challenge in Generative Modelling for Structured Electronic Medical Records

Synthetic healthcare data are widely proposed as privacy-preserving substitutes for real patient data, yet their evaluation remains dominated by statistical similarity and predictive performance that do not reflect clinical validity. We introduce a multi-dimensional evaluation framework grounded in epidemiology, assessing descriptive fidelity, clinical utility, and structural validity, corresponding to descriptive, predictive, and causal questions. We evaluate four representative generative paradigms - GAN-based, VAE-boosted, diffusion-based, and masked modelling - using PRIME-CVD, a 50,000-person cohort with known ground-truth structure. While all models reproduce marginal distributions, none simultaneously preserve subgroup structure, effect estimates, and dependency structure. Notably, models with strong distributional fidelity can exhibit poor calibration and distorted relationships, leading to unreliable inference. These results show that current evaluation practices can overestimate synthetic data quality and motivate domain-informed assessment based on the ability to support valid clinical and scientific conclusions.

Nicholas I-Hsien Kuo, Blanca Gallego, Louisa Jorm

Centre for Big Data Research in Health, the University of New South Wales, Sydney, Australia

Synthetic Tabular Data Generation Electronic Health Records Synthetic Data Evaluation Synthetic Clinical Data Generation

Ask PDF

Apr 11, 2026

CircuitSynth: Reliable Synthetic Data Generation

The generation of high-fidelity synthetic data is a cornerstone of modern machine learning, yet Large Language Models (LLMs) frequently suffer from hallucinations, logical inconsistencies, and mode collapse when tasked with structured generation. Existing approaches, such as prompting or retrieval-augmented generation, lack the mechanisms to balance linguistic expressivity with formal guarantees regarding validity and coverage. To address this, we propose CircuitSynth, a novel neuro-symbolic framework that decouples semantic reasoning from surface realization. By distilling the reasoning capabilities of a Teacher LLM into a Probabilistic Sentential Decision Diagram (PSDD), CircuitSynth creates a tractable semantic prior that structurally enforces hard logical constraints. Furthermore, we introduce a convex optimization mechanism to rigorously satisfy soft distributional goals. Empirical evaluations across diverse benchmarks demonstrate that CircuitSynth achieves 100% Schema Validity even in complex logic puzzles where unconstrained baselines fail (12.4%) while significantly outperforming state-of-the-art methods in rare-combination coverage.

Zehua Cheng, Wei Dai, Jiahao Sun +1

Department of Computer Science, University of Oxford · Institute of Logic and Computation, TU Wien

Synthetic Data Generation Synthetic Data Generation for Language Models Constrained Generative Modeling Neuro-Symbolic Reasoning

Ask PDF

Abstract

Synthetic Clinical Data Generation

Explore similar work

OncoSynth: Synthetic data generation for treatment effect estimation in oncology

Synthetic but Not Realistic: The Evaluation Challenge in Generative Modelling for Structured Electronic Medical Records

CircuitSynth: Reliable Synthetic Data Generation