From Augmentation to Synthetic Evidence: Evaluating Generated Chest Radiographs at Dataset Intake

Shaikh Akib Shahriyar; Zinedine Sehili; Adam Arthur; Christopher Schwartz

cs.CVOct 21, 2026

From Augmentation to Synthetic Evidence: Evaluating Generated Chest Radiographs at Dataset Intake

Authors: Shaikh Akib Shahriyar, Zinedine Sehili, Adam Arthur, Christopher Schwartz

Organizations: Rochester Institute of Technology Rochester, NY, USA

Abstract

A generated chest radiograph can be useful as declared augmentation, but the same image becomes unsupported evidence when submitted as the record of a real examination. We study this problem by separating three questions: whether an image passes an automated quality check, whether an origin classifier flags it as generated, and whether a trusted record supports the claimed acquisition. In our experiments, Qwen selects up to 20 images from pools of 1,000 existing Cheff-identified or SynthCheX candidates. A receiver evaluates quality with MedGemma and predicted origin with a classifier on frozen DINOv3 features. Quality-based selection raises quality passes from 12 to 19 of 20 for Cheff-identified images and from 8 to 17 for SynthCheX. We find no evidence that selection improves passage through both checks: no selected Cheff-identified image passes both, while uniform, quality, and quality-constrained surrogate selection produce 2, 1, and 2 SynthCheX passes. On unprocessed test images, the origin classifier flags 4,677 of 5,000 SynthCheX images, but it also misclassify 110 of 2,297 NIH (real) images and 568 of 957 genuine images from another collection as fake. Our results further show that JPEG compression reduces SynthCheX-versus-NIH AUROC from 0.987 to 0.841. Even without processing, 152 SynthCheX images pass both pixel checks. These checks can filter submissions, but they cannot verify an acquisition claim. We strongly advocate that source and intended use must be recorded and checked separately for admission of chest radiographs in medical datasets.

Figures & tables

Panel of Figure 1 — Figure 1: One example for each outcome of the two checks, chosen after the analysis by a fixed hash rule. Images (a) and (d) are generated (SynthCheX), and (b) and (c) are genuine (NIH ChestX-ray14). Image (a) passes both checks, but its pixels do not support a claim that it records a clinical acquisition.

Figure 2: Workflow of the intake test. The submitter selects at most

K

B

generated candidates using only Qwen judgments and has no access to receiver results. Two models with fixed thresholds answer the quality and predicted-origin questions, and a registry answers the claim-support question.

Collection	Fit	Dev.	Cal.	Test
NIH genuine	1,500	1,000	3,000	2,297
Cheff-identified	1,500	500	0	1,000
SynthCheX	1,000	500	0	5,000
Total	4,000	2,000	3,000	8,297

Table 1: Images in each split. The calibration split holds genuine images only. The raw CLS head is fitted on NIH and Cheff-identified images, and the SynthCheX fit and development images serve secondary heads.

Origin classifier	NIH flagged	Cheff rejected	SynthCheX rejected	Unavailable
DINOv3 CLS head, raw (primary)	110 (4.0–5.7)	1,000 (99.6–100.0)	4,677 (92.8–94.2)	0/0/0
Medical head, raw	107 (3.9–5.6)	1,000 (99.6–100.0)	4,673 (92.7–94.1)	0/0/0
DINOv3 CLS head, augmented	97 (3.5–5.1)	1,000 (99.6–100.0)	4,644 (92.1–93.6)	0/0/0
Rubric-field probe	22 (0.6–1.4)	13 (0.8–2.2)	99 (1.6–2.4)	78/12/80
Pixel-statistics classifier	50 (1.7–2.9)	603 (57.2–63.3)	2,782 (54.3–57.0)	0/0/0

Table 2: Origin check on canonical test images at the nominal 5% false-alarm rate. Cells give counts out of all attempted images (2,297 NIH, 1,000 Cheff-identified, 5,000 SynthCheX) with Wilson 95% intervals in percent. Thresholds use 3,000 genuine calibration images (2,930 valid replies for the rubric-field probe). The last column counts unavailable outputs (NIH/Cheff/SynthCheX), which are neither flags nor passes.

Figure 3: SynthCheX images that pass both checks, with $K=20$ : (a) per submitted image and (b) per candidate in the pool of size $B$ . The vertical axis calls them “quality-qualified false passes”. Solid bars are Wilson 95% intervals for the fixed lists. Dashed bars come from 2,000 resamples that repeat the selection ( $B=1{,}000$ and $5{,}000$ only).

Figure 4: Percent of the 1,000 test images per collection (“bases”) rejected (a, b) or flagged (c) under 13 conditions, at the nominal 5% level. “Cut” means decision threshold. Raw/canonical: canonical threshold. Raw/matched: threshold recalibrated per condition. Augmented/matched: augmented head, matched threshold.

Checkpoint	Revision	Processor input
Qwen3.8-27B	1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0	maximum 262,144 pixels
MedGemma 1.5 4B IT	91850547d9f0b2fdd21aa7c5f4f3d1a8a52c243b	$896\times 896$
DINOv3 ViT-7B/16	b80367753773648a6793235ab9c65cdbb029506f	$224\times 224$

Table S1: Pinned checkpoints (IT means instruction-tuned). Processor input is the processor setting, which differs from the stored dimensions of the source images.

Figure S1: Development quality controls. Each collection contributes 100 images and 900 pairs of an undegraded image and a degraded copy, and the repeated degraded versions are not independent images. Invalid-input passes count 60 distinct patterns, each counted as a pass if any repeated exposure passes. Failures and unavailable pairs are shown explicitly. The alternative rubric improves pooled strict ordering but does worse on SynthCheX. Panel A shows the ordering outcomes, panel B the invalid-pattern passes without the flat-image guard, and panel C (“burden”) the clean NIH images without an assessable judgment. In the notes, “native-final failures” are failed final replies, and “cap” is a generation that hit the token cap.

Evaluator	Collection	Attempted N	Valid of attempted	Unavailable of attempted	Unassessable of valid	Guarded pass of attempted	Top score of valid
MedGemma alt.	NIH	2297	2219/2297	78/2297	2/2219	1717/2297	1229/2219
MedGemma alt.	Cheff	1000	988/1000	12/1000	6/988	668/1000	305/988
MedGemma alt.	SynthCheX	5000	4920/5000	80/5000	12/4920	2656/5000	856/4920
Qwen original	NIH	2297	2297/2297	0/2297	0/2297	2059/2297	1558/2297
Qwen original	Cheff	1000	1000/1000	0/1000	1/1000	695/1000	416/1000
Qwen original	SynthCheX	5000	5000/5000	0/5000	0/5000	3056/5000	1238/5000

Table S2: All quality evaluators on each test collection. Valid and unavailable use attempted images as the denominator. Unassessable and top-score counts use valid replies. Guarded passes use all attempted images with the flat-image guard. Original scores keep their 0–100 scale (top score 100). The alternative rubric keeps its ordinal fields (top score: visibility 4 and artifact severity 0) and is never converted to 100.

Evaluator	Collection	Strategy	Pass k/n [95% CI]	P(genuine below) [95% CI]	Ordinal n genuine / selected	Unavailable genuine / selected
MedGemma alt.	Cheff	Uniform	12/20 [0.387, 0.781]	0.242 [0.164, 0.317]	2155 / 18	78 / 0
MedGemma alt.	Cheff	Quality	19/20 [0.764, 0.991]	0.337 [0.276, 0.395]	2155 / 20	78 / 0
MedGemma alt.	Cheff	Surrogate	19/20 [0.764, 0.991]	0.333 [0.269, 0.391]	2155 / 20	78 / 0
MedGemma alt.	SynthCheX	Uniform	8/20 [0.219, 0.613]	0.179 [0.098, 0.267]	2155 / 18	78 / 1
MedGemma alt.	SynthCheX	Quality	17/20 [0.640, 0.948]	0.344 [0.271, 0.410]	2155 / 20	78 / 0
MedGemma alt.	SynthCheX	Surrogate	14/20 [0.481, 0.855]	0.258 [0.183, 0.333]	2155 / 17	78 / 2

Table S3: MedGemma’s quality judgments of the 12 fixed submitted lists (

B=1{,}000

K=20

): two generated collections, three strategies, and both rubrics. Pass uses all 20 submissions, with Wilson 95% intervals as fractions. P(genuine below) is the share of genuine images rated strictly below a selected image, with an interval that conditions on the fixed list. Ordinal and unavailable counts are given as genuine/selected. A valid reply with a null alternative artifact field can lack an ordinal rank.

Evaluator	Collection	Rating field	Mean difference [95% CI]	Median difference [95% CI]	Finite n genuine / generated
MedGemma alt.	Cheff	Visibility	-0.252 [-0.293, -0.213]	-1.000 [-1.000, -1.000]	2219 / 988
MedGemma alt.	Cheff	Artifacts	0.378 [0.313, 0.443]	1.000 [1.000, 1.000]	2155 / 961
MedGemma alt.	SynthCheX	Visibility	-0.397 [-0.422, -0.372]	-1.000 [-1.000, -1.000]	2219 / 4920
MedGemma alt.	SynthCheX	Artifacts	0.693 [0.649, 0.734]	1.000 [1.000, 1.000]	2155 / 4817
Qwen orig.	Cheff	Technical	-0.484 [-0.546, -0.423]	-1.000 [-1.000, -1.000]	2297 / 1000
Qwen orig.	Cheff	Anatomy	-0.217 [-0.254, -0.182]	0.000 [0.000, 0.000]	2297 / 1000

Table S4: Generated-minus-genuine differences in rating fields, part 1 of 2. Higher technical, anatomy, visibility, and original-score values are better, and higher artifact severity is worse. Means, medians, and bootstrap intervals use valid replies with a non-null field. The scales differ and are not pooled or remapped.

Evaluator	Collection	Rating field	Mean difference [95% CI]	Median difference [95% CI]	Finite n genuine / generated
Qwen orig.	SynthCheX	Artifacts	0.328 [0.306, 0.349]	0.000 [0.000, 1.000]	2297 / 5000
Qwen orig.	SynthCheX	Score	-12.060 [-12.677, -11.443]	-15.000 [-15.000, -10.000]	2297 / 5000
MedGemma orig.	Cheff	Technical	-0.241 [-0.300, -0.183]	0.000 [0.000, 0.000]	2296 / 999
MedGemma orig.	Cheff	Anatomy	-0.101 [-0.129, -0.074]	0.000 [0.000, 0.000]	2296 / 999
MedGemma orig.	Cheff	Artifacts	0.255 [0.203, 0.308]	0.000 [0.000, 0.000]	2296 / 999
MedGemma orig.	Cheff	Score	-4.697 [-5.722, -3.688]	0.000 [0.000, 0.000]	2296 / 999

Table S5: Continuation of Table S4 , part 2 of 2.

Figure S2: Distributions of canonical quality components, Cheff-identified images against NIH images. Higher is better for the original technical and anatomical fields and for alternative visibility, and higher is worse for artifact severity. The components are on different scales, and unavailable and null-field replies are not plotted. In the legends, “raw” means all Cheff-identified test images, “quality K20” and “surrogate K

\leq

20” mean the images chosen by those strategies at

B=1{,}000

, and the fractions count the images with a plotted score. “Native score” is the score on the rubric’s own scale, and “Med” means MedGemma. Overlap between the curves does not show that the images are indistinguishable in origin.

Figure S3: Distributions of canonical quality components, SynthCheX images against NIH images, with the conventions of Figure S2 . SynthCheX was filtered before release and may share conditioning, which limits what the distributions show.

Figure S4: Quality and origin are different questions. Each point is one test image, placed by its original-rubric MedGemma score (0–100 scale, called “Original Med quality proxy” in the axis label) and by the raw CLS head’s origin score (higher means more likely generated; the figure calls it “suspicion”). Legends give the numbers of images with a finite score. The remaining images (1 NIH, 1 Cheff-identified, and 6 SynthCheX) have an unavailable original-rubric reply. Circles mark the images selected at $B=1{,}000$ and $K=20$ , and the legend separates quality-selected from surrogate-selected images. The dotted line is the raw head’s 5% threshold, calibrated on genuine NIH images, and “DINO” in the figure means DINOv3. The primary quality check uses the alternative rubric’s ordinal fields, not this score.

Rubric	Collection	Strict	Tied	Reversed	Unavailable
Original	NIH	179	370	49	2
Original	Cheff-identified	194	323	83	0
Original	SynthCheX	209	295	96	0
Alternative	NIH	274	207	95	24
Alternative	Cheff-identified	234	224	134	8
Alternative	SynthCheX	142	252	197	9

Table S6: Held-out ordering outcomes for the 600 expected pairs per collection and rubric. A pair is strict when the undegraded image is rated above its degraded copy. The six degraded copies of one image stay paired, and missing replies stay in the denominator.

Rubric	Invalid pass No guard (CI)	Invalid pass Guard (CI)	NIH images without an assessable judgment	NIH unavailable
Original	5/60 [3.61, 18.07]	0/60 [0.00, 6.02]	0/100	0/100
Alternative	0/60 [0.00, 6.02]	0/60 [0.00, 6.02]	3/100	3/100

Table S7: Held-out invalid-pattern passes (out of 60 distinct patterns, with Wilson 95% intervals in percent) and clean genuine images (out of 100 NIH images). The column “without an assessable judgment” counts unavailable, unassessable, and non-chest replies once per image. Its unavailable part is shown separately.

Rubric	Collection	Canonical	JPEG75	Resize128	Native512
Original	NIH	278 (0)	282 (0)	278 (0)	286 (0)
Original	Cheff-identified	241 (1)	243 (0)	226 (0)	248 (0)
Original	SynthCheX	235 (0)	237 (0)	232 (0)	241 (1)
Alternative	NIH	223 (11)	224 (5)	235 (2)	233 (10)
Alternative	Cheff-identified	189 (3)	196 (4)	207 (3)	188 (3)
Alternative	SynthCheX	182 (5)	169 (8)	188 (5)	170 (7)

Table S8: Quality passes with the flat-image guard among 300 attempted images per collection and condition, with unavailable replies in parentheses. Each row repeats the same images. The three paired intervals for the primary NIH pass shifts include zero, which does not establish equivalence. Across all collections and conditions, the original and alternative rubrics have 2 and 66 unavailable replies out of 3,600.

Rubric	Collection	Family	Expected chains	Valid	Non- increasing	Strict	Reversed	Tie	Unavailable
Original	NIH	Blur	100	100	85	10	15	77	0
Original	NIH	Resample	100	100	83	2	17	86	0
Original	NIH	Noise	100	98	84	17	14	70	2
Original	Cheff-identified	Blur	100	100	72	15	28	63	0
Original	Cheff-identified	Resample	100	100	73	5	27	78	0
Original	Cheff-identified	Noise	100	100	71	10	29	65	0

Table S9: Held-out results for 18 combinations of rubric, collection, and degradation family. Each row covers 100 expected chains of one image under increasing blur, resampling, or noise, starting from the canonical view. A chain is valid when every rating in it is available. Non-increasing allows ties, and strict requires every step to decrease. The reversal and tie columns count chains with at least one such step, so they can overlap. Unavailable chains are the expected chains that are not valid.

Rubric	Collection	Valid / attempted	Replies at top score	Of these, eligible	Top score / all valid
Original	NIH	698/700	445	445	445/698
Original	Cheff-identified	700/700	376	376	376/700
Original	SynthCheX	700/700	347	347	347/700
Original	Non-image	67/70	0	0	0/67
Alternative	NIH	691/700	155	155	155/691
Alternative	Cheff-identified	692/700	105	105	105/692

Table S10: Replies at the top of each rubric’s own scale in the held-out panel, including the non-image exposures. The top is a score of 100 for the original rubric and the pair visibility 4 and artifact severity 0 for the alternative rubric, which we do not remap to 100. “Replies at top score” counts all valid replies at the top, and “eligible” counts the subset that also meets the rubric’s eligibility conditions. These counts are exposure-level counts, and they differ from the distinct-pattern passes in Table S7 .

Origin classifier	NIH flagged	Cheff rejected	SynthCheX rejected	Unavailable (NIH/C/S)
Pilot CLS head	131 (4.8–6.7)	1,000 (99.6–100.0)	4,436 (87.8–89.6)	0/0/0
DINOv3 CLS, raw (primary)	110 (4.0–5.7)	1,000 (99.6–100.0)	4,677 (92.8–94.2)	0/0/0
DINOv3 mean, raw	123 (4.5–6.4)	1,000 (99.6–100.0)	4,407 (87.2–89.0)	0/0/0
Medical mean, raw	107 (3.9–5.6)	1,000 (99.6–100.0)	4,673 (92.7–94.1)	0/0/0
DINOv3 CLS, augmented	97 (3.5–5.1)	1,000 (99.6–100.0)	4,644 (92.1–93.6)	0/0/0
Rubric fields	22 (0.6–1.4)	13 (0.8–2.2)	99 (1.6–2.4)	78/12/80

Table S11: Origin check on canonical test images at the nominal 5% false-alarm rate, without the quality check, for all seven heads. Cells give counts with Wilson 95% intervals in percent. The thresholds use 3,000 genuine calibration images, except for the pilot head (2,914) and the rubric fields (2,930 valid replies).

Method	$\alpha$ (%)	NIH reference N; U	NIH flags count; % [95%]	Cheff rejections count; % [95%]	SynthCheX rejections count; % [95%]	Unavailable N/C/S
Pilot CLS	1	2914 U=0	39/2297; 1.70 [1.24, 2.31]	997/1000; 99.70 [99.12, 99.90]	3944/5000; 78.88 [77.73, 79.99]	0/0/0
Pilot CLS	10	2914 U=0	237/2297; 10.32 [9.14, 11.63]	1000/1000; 100.00 [99.62, 100.00]	4636/5000; 92.72 [91.97, 93.41]	0/0/0
DINOv3 CLS raw	1	3000 U=0	26/2297; 1.13 [0.77, 1.65]	999/1000; 99.90 [99.44, 99.98]	4339/5000; 86.78 [85.81, 87.69]	0/0/0
DINOv3 CLS raw	10	3000 U=0	200/2297; 8.71 [7.62, 9.93]	1000/1000; 100.00 [99.62, 100.00]	4780/5000; 95.60 [95.00, 96.13]	0/0/0
DINOv3 mean raw	1	3000 U=0	29/2297; 1.26 [0.88, 1.81]	999/1000; 99.90 [99.44, 99.98]	3808/5000; 76.16 [74.96, 77.32]	0/0/0
DINOv3 mean raw	10	3000 U=0	255/2297; 11.10 [9.88, 12.45]	1000/1000; 100.00 [99.62, 100.00]	4631/5000; 92.62 [91.86, 93.31]	0/0/0

Table S12: The seven core heads at the nominal 1% and 10% levels (Table S11 gives the 5% level). Cells give count/attempted, percentage, and Wilson 95% interval. NIH counts are false alarms, and the generated columns are rejections. U gives unavailable outputs (NIH/Cheff/SynthCheX). The reference columns give the finite reference size and the unavailable references: the pilot head uses 2,914 images after 86 exclusions, and the rubric fields use 2,930 valid replies after 10 exclusions and 60 unavailable replies.

Method	Generated collection	AUROC [95%]	Std. partial AUC [95%]	Finite N NIH/generated	Unavailable U NIH/generated
Pilot CLS	Cheff	0.9996 [0.9992, 0.9999]	0.9981 [0.9960, 0.9995]	2297/1000	0/0
Pilot CLS	SynthCheX	0.9704 [0.9671, 0.9736]	0.9182 [0.9085, 0.9270]	2297/5000	0/0
DINOv3 CLS raw	Cheff	0.9999 [0.9997, 1.0000]	0.9994 [0.9982, 1.0000]	2297/1000	0/0
DINOv3 CLS raw	SynthCheX	0.9849 [0.9825, 0.9871]	0.9569 [0.9496, 0.9630]	2297/5000	0/0
DINOv3 mean raw	Cheff	0.9997 [0.9993, 1.0000]	0.9986 [0.9962, 0.9999]	2297/1000	0/0
DINOv3 mean raw	SynthCheX	0.9699 [0.9665, 0.9732]	0.9182 [0.9083, 0.9272]	2297/5000	0/0

Table S13: AUROC and standardized partial AUC (false-positive rates up to 0.10) under canonical processing, for the seven heads and both generated collections, with bootstrap 95% intervals. Finite N and unavailable U are given as NIH/generated. Unavailable outputs are not detections.

First minus second	Collection	AUROC difference [95% CI]	Std. partial AUC difference [95% CI]	Finite in both NIH / gen.	Not finite NIH / gen.
DINOv3 mean $-$ DINOv3 CLS	Cheff	$-0.000156726$ [ $-0.000404495$ , $-6.51937\times 10^{-6}$ ]	$-0.000824875$ [ $-0.00212892$ , $-3.43125\times 10^{-5}$ ]	2297 / 1000	0 / 0
DINOv3 mean $-$ DINOv3 CLS	SynthCheX	$-0.0149568$ [ $-0.0173688$ , $-0.012378$ ]	$-0.0386599$ [ $-0.0456914$ , $-0.0320829$ ]	2297 / 5000	0 / 0
Medical mean $-$ DINOv3 CLS	Cheff	$6.53026\times 10^{-5}$ [ $-6.66195\times 10^{-5}$ , $0.000289987$ ]	$0.000343698$ [ $-0.000350629$ , $0.00152625$ ]	2297 / 1000	0 / 0
Medical mean $-$ DINOv3 CLS	SynthCheX	$0.00422125$ [ $0.0017536$ , $0.00678711$ ]	$0.000384208$ [ $-0.0074951$ , $0.00813689$ ]	2297 / 5000	0 / 0
DINOv3 CLS aug. $-$ DINOv3 CLS	Cheff	$4.78886\times 10^{-6}$ [ $-1.8731\times 10^{-5}$ , $3.96169\times 10^{-5}$ ]	$2.52045\times 10^{-5}$ [ $-9.8584\times 10^{-5}$ , $0.00020851$ ]	2297 / 1000	0 / 0
DINOv3 CLS aug. $-$ DINOv3 CLS	SynthCheX	$0.00152634$ [ $0.000202819$ , $0.00294557$ ]	$0.00124176$ [ $-0.0022921$ , $0.00470716$ ]	2297 / 5000	0 / 0

Table S14: Paired differences in AUROC and standardized partial AUC between core heads (first-named head minus second), on images with finite scores in both heads. Each cell gives the estimate, then the lower and upper bounds of the paired bootstrap interval. The last two columns count the genuine and generated images that are finite in both heads or not. DINOv3 CLS and Medical mean are the raw heads, and DINOv3 CLS aug. is the augmented CLS head (Section S2 ).

Method	NIH reference N; U	NIH flags count; % [95%]	Cheff passes origin check count; % [95%]	SynthCheX passes origin check count; % [95%]	Unavailable N/C/S
CLS / Cheff	3000 U=0	101/2297; 4.40 [3.63, 5.31]	0/1000; 0.00 [0.00, 0.38]	311/5000; 6.22 [5.58, 6.92]	0/0/0
CLS / SynthCheX	3000 U=0	165/2297; 7.18 [6.20, 8.31]	114/1000; 11.40 [9.58, 13.52]	0/5000; 0.00 [0.00, 0.08]	0/0/0
CLS / Mixed	3000 U=0	134/2297; 5.83 [4.95, 6.87]	0/1000; 0.00 [0.00, 0.38]	0/5000; 0.00 [0.00, 0.08]	0/0/0
Mean / Cheff	3000 U=0	114/2297; 4.96 [4.15, 5.93]	0/1000; 0.00 [0.00, 0.38]	673/5000; 13.46 [12.54, 14.43]	0/0/0
Mean / SynthCheX	3000 U=0	135/2297; 5.88 [4.99, 6.91]	177/1000; 17.70 [15.46, 20.19]	0/5000; 0.00 [0.00, 0.08]	0/0/0
Mean / Mixed	3000 U=0	162/2297; 7.05 [6.08, 8.17]	4/1000; 0.40 [0.16, 1.02]	1/5000; 0.02 [0.00, 0.11]	0/0/0

Table S15: Secondary heads at the nominal 5% level. Cells give count/attempted, percentage, and Wilson 95% interval. NIH counts are false alarms, and the generated columns count generated images that pass the origin check. U gives unavailable outputs (NIH/Cheff/SynthCheX). All nine reference sets hold 3,000 images.

Head	Nominal $\alpha$ (%)	NIH flags [95% CI]	Cheff passes origin check [95% CI]	SynthCheX passes origin check [95% CI]	Reference n; unavailable NIH/Cheff/SynthCheX
CLS / Cheff	1	20/2297 [0.56, 1.34]	1/1000 [0.02, 0.56]	695/5000 [12.97, 14.89]	Ref. 3000 U: 0 / 0 / 0
CLS / Cheff	10	215/2297 [8.24, 10.62]	0/1000 [0.00, 0.38]	196/5000 [3.42, 4.49]	Ref. 3000 U: 0 / 0 / 0
CLS / SynthCheX	1	33/2297 [1.02, 2.01]	213/1000 [18.87, 23.94]	0/5000 [0.00, 0.08]	Ref. 3000 U: 0 / 0 / 0
CLS / SynthCheX	10	333/2297 [13.12, 16.00]	69/1000 [5.49, 8.64]	0/5000 [0.00, 0.08]	Ref. 3000 U: 0 / 0 / 0
CLS / Mixed	1	19/2297 [0.53, 1.29]	2/1000 [0.05, 0.73]	2/5000 [0.01, 0.15]	Ref. 3000 U: 0 / 0 / 0
CLS / Mixed	10	232/2297 [8.93, 11.40]	0/1000 [0.00, 0.38]	0/5000 [0.00, 0.08]	Ref. 3000 U: 0 / 0 / 0

Table S16: Secondary heads at the nominal 1% and 10% levels, part 1 of 2. NIH flags are false alarms, and the generated columns count generated images that pass the origin check. All heads use 3,000 genuine reference images. Cells give counts with Wilson 95% intervals in percent, and U gives unavailable outputs (genuine/Cheff/SynthCheX). A shared nominal level does not mean equal achieved false-alarm rates.

Head	Nominal $\alpha$ (%)	NIH flags [95% CI]	Cheff passes origin check [95% CI]	SynthCheX passes origin check [95% CI]	Reference n; unavailable NIH/Cheff/SynthCheX
Mean / SynthCheX	10	300/2297 [11.74, 14.50]	119/1000 [10.04, 14.05]	0/5000 [0.00, 0.08]	Ref. 3000 U: 0 / 0 / 0
Mean / Mixed	1	34/2297 [1.06, 2.06]	15/1000 [0.91, 2.46]	9/5000 [0.09, 0.34]	Ref. 3000 U: 0 / 0 / 0
Mean / Mixed	10	299/2297 [11.70, 14.46]	1/1000 [0.02, 0.56]	0/5000 [0.00, 0.08]	Ref. 3000 U: 0 / 0 / 0
Medical / Cheff	1	26/2297 [0.77, 1.65]	1/1000 [0.02, 0.56]	1123/5000 [21.32, 23.64]	Ref. 3000 U: 0 / 0 / 0
Medical / Cheff	10	188/2297 [7.13, 9.38]	0/1000 [0.00, 0.38]	301/5000 [5.39, 6.71]	Ref. 3000 U: 0 / 0 / 0
Medical / SynthCheX	1	57/2297 [1.92, 3.20]	149/1000 [12.83, 17.24]	1/5000 [0.00, 0.11]	Ref. 3000 U: 0 / 0 / 0

Table S17: Continuation of Table S16 , part 2 of 2.

Head	Collection	AUROC [95% CI]	Std. partial AUC [95% CI]	Finite n NIH / gen.	Unavailable n NIH / gen.
CLS / Cheff	Cheff	$0.999872$ [ $0.99964$ , $0.999998$ ]	$0.999329$ [ $0.998107$ , $0.999989$ ]	2297 / 1000	0 / 0
CLS / Cheff	SynthCheX	$0.986047$ [ $0.983776$ , $0.988085$ ]	$0.959035$ [ $0.951916$ , $0.964967$ ]	2297 / 5000	0 / 0
CLS / SynthCheX	Cheff	$0.966237$ [ $0.959058$ , $0.973158$ ]	$0.919539$ [ $0.907532$ , $0.931643$ ]	2297 / 1000	0 / 0
CLS / SynthCheX	SynthCheX	$0.999996$ [ $0.99999$ , $1$ ]	$0.99998$ [ $0.99995$ , $0.999998$ ]	2297 / 5000	0 / 0
CLS / Mixed	Cheff	$0.99971$ [ $0.999378$ , $0.999929$ ]	$0.998474$ [ $0.996728$ , $0.999629$ ]	2297 / 1000	0 / 0
CLS / Mixed	SynthCheX	$0.999792$ [ $0.999399$ , $0.99999$ ]	$0.998905$ [ $0.996837$ , $0.999949$ ]	2297 / 5000	0 / 0

Table S18: AUROC and standardized partial AUC (false-positive rates up to 0.10) for the nine secondary heads on canonical test images, part 1 of 2. Heads are named by their features and by the generators used in fitting. Intervals come from the paired bootstrap on test images. Unavailable outputs are not detections.

Head	Collection	AUROC [95% CI]	Std. partial AUC [95% CI]	Finite n NIH / gen.	Unavailable n NIH / gen.
Mean / SynthCheX	SynthCheX	$0.999975$ [ $0.999955$ , $0.999991$ ]	$0.999867$ [ $0.999761$ , $0.99995$ ]	2297 / 5000	0 / 0
Mean / Mixed	Cheff	$0.998768$ [ $0.998087$ , $0.999353$ ]	$0.993942$ [ $0.990759$ , $0.99671$ ]	2297 / 1000	0 / 0
Mean / Mixed	SynthCheX	$0.999619$ [ $0.999124$ , $0.999914$ ]	$0.998005$ [ $0.995409$ , $0.999554$ ]	2297 / 5000	0 / 0
Medical / Cheff	Cheff	$0.999897$ [ $0.99981$ , $0.999963$ ]	$0.999459$ [ $0.998999$ , $0.999805$ ]	2297 / 1000	0 / 0
Medical / Cheff	SynthCheX	$0.982348$ [ $0.979953$ , $0.984484$ ]	$0.935143$ [ $0.926342$ , $0.943188$ ]	2297 / 5000	0 / 0
Medical / SynthCheX	Cheff	$0.979815$ [ $0.975402$ , $0.983743$ ]	$0.935848$ [ $0.924825$ , $0.945991$ ]	2297 / 1000	0 / 0

Table S19: Continuation of Table S18 , part 2 of 2.

First minus second	Collection	AUROC difference [95% CI]	Std. partial AUC difference [95% CI]	Finite in both NIH / gen.	Not finite NIH / gen.
CLS / SynthCheX $-$ CLS / Cheff	Cheff	$-0.0336352$ [ $-0.0407283$ , $-0.0267199$ ]	$-0.0797901$ [ $-0.0918953$ , $-0.0676765$ ]	2297 / 1000	0 / 0
CLS / SynthCheX $-$ CLS / Cheff	SynthCheX	$0.013949$ [ $0.0119077$ , $0.0162161$ ]	$0.0409448$ [ $0.0350172$ , $0.0480639$ ]	2297 / 5000	0 / 0
CLS / Mixed $-$ CLS / Cheff	Cheff	$-0.000162386$ [ $-0.000311276$ , $-4.22181\times 10^{-5}$ ]	$-0.000854662$ [ $-0.00163829$ , $-0.000222201$ ]	2297 / 1000	0 / 0
CLS / Mixed $-$ CLS / Cheff	SynthCheX	$0.0137448$ [ $0.01182$ , $0.0159114$ ]	$0.0398701$ [ $0.0347462$ , $0.0454903$ ]	2297 / 5000	0 / 0
Mean / SynthCheX $-$ Mean / Cheff	Cheff	$-0.0477701$ [ $-0.0558783$ , $-0.0397965$ ]	$-0.108361$ [ $-0.122278$ , $-0.0954331$ ]	2297 / 1000	0 / 0
Mean / SynthCheX $-$ Mean / Cheff	SynthCheX	$0.0332486$ [ $0.0298327$ , $0.0368343$ ]	$0.091582$ [ $0.0823968$ , $0.101947$ ]	2297 / 5000	0 / 0

Table S20: Paired differences in AUROC and standardized partial AUC between secondary heads (first-named head minus second), for heads with the same features fitted on different generators. Each cell gives the estimate, then the lower and upper bounds of the paired bootstrap interval, and the last two columns count images that are finite in both heads or not.

Method	NIH reference N; U	1% policy flags; % [95%]	5% policy flags; % [95%]	10% policy flags; % [95%]
Pilot CLS	2914 U=0	257/957; 26.85 [24.14, 29.75]	491/957; 51.31 [48.14, 54.46]	611/957; 63.85 [60.75, 66.83]
DINOv3 CLS raw	3000 U=0	374/957; 39.08 [36.04, 42.21]	568/957; 59.35 [56.21, 62.42]	663/957; 69.28 [66.28, 72.12]
DINOv3 mean raw	3000 U=0	125/957; 13.06 [11.07, 15.35]	292/957; 30.51 [27.68, 33.50]	413/957; 43.16 [40.05, 46.31]
Medical mean raw	3000 U=0	61/957; 6.37 [4.99, 8.10]	152/957; 15.88 [13.70, 18.33]	226/957; 23.62 [21.03, 26.41]
DINOv3 CLS augmented	3000 U=0	312/957; 32.60 [29.71, 35.64]	493/957; 51.52 [48.35, 54.67]	607/957; 63.43 [60.33, 66.42]
Pixel statistics	3000 U=0	11/957; 1.15 [0.64, 2.05]	27/957; 2.82 [1.95, 4.07]	40/957; 4.18 [3.08, 5.64]

Table S21: Open-I genuine images flagged at the NIH-derived 1%, 5%, and 10% thresholds, for six heads and 957 images each. Cells give flags/957, percentage, and Wilson 95% interval. U is unavailable outputs. The Open-I images took no part in fitting or tuning, and the set has no generated images, so it gives no AUROC.

Collection	Endpoint	Count	Rate (%)	Wilson 95% (%)
NIH genuine	Genuine flagged	178/300	59.3	53.7–64.7
Cheff-identified	Generated passed	127/300	42.3	36.9–48.0
SynthCheX	Generated passed	50/300	16.7	12.9–21.3

Table S22: MedForge at its own categorical operating point on 300 fixed canonical images per collection. Every verdict is available. These outcomes are origin-only, without the quality check, and MedForge has no matched false-alarm target.

Collection	Selection	Passes	95% (%)
Cheff-identified	Surrogate	4/20	8.1–41.6
Cheff-identified	Quality	4/20	8.1–41.6
Cheff-identified	Uniform	9/20	25.8–65.8
SynthCheX	Surrogate	1/20	0.9–23.6
SynthCheX	Quality	0/20	0.0–16.1
SynthCheX	Uniform	5/20	11.2–46.9

Table S23: Images that MedForge passes as real, among the fixed submitted lists at

B=1{,}000

and

K=20

. Wilson intervals describe these fixed lists, not the whole selection procedure. A table with all 29 conditions is in the machine-readable output.

Condition	Raw/canonical	Raw/matched	Augmented/matched	Augmented/global
Canonical	45 / 1000 / 954 U: 0 / 0 / 0	45 / 1000 / 954 U: 0 / 0 / 0	43 / 1000 / 940 U: 0 / 0 / 0	19 / 1000 / 913 U: 0 / 0 / 0
JPEG95	503 / 1000 / 988 U: 0 / 0 / 0	36 / 998 / 907 U: 0 / 0 / 0	37 / 1000 / 937 U: 0 / 0 / 0	52 / 1000 / 947 U: 0 / 0 / 0
JPEG85	706 / 999 / 991 U: 0 / 0 / 0	55 / 966 / 702 U: 0 / 0 / 0	40 / 998 / 916 U: 0 / 0 / 0	25 / 998 / 899 U: 0 / 0 / 0
JPEG75	820 / 1000 / 987 U: 0 / 0 / 0	65 / 889 / 486 U: 0 / 0 / 0	41 / 998 / 906 U: 0 / 0 / 0	76 / 999 / 935 U: 0 / 0 / 0
JPEG50	950 / 1000 / 998 U: 0 / 0 / 0	68 / 800 / 355 U: 0 / 0 / 0	52 / 992 / 845 U: 0 / 0 / 0	614 / 1000 / 996 U: 0 / 0 / 0
Resize192	651 / 1000 / 989 U: 0 / 0 / 0	68 / 995 / 841 U: 0 / 0 / 0	41 / 998 / 922 U: 0 / 0 / 0	144 / 1000 / 971 U: 0 / 0 / 0

Table S24: All 13 processing conditions at the nominal 5% level. Each cell gives NIH flags / Cheff rejections / SynthCheX rejections out of 1,000 images per collection, with unavailable counts (U) on the second line. The same images appear in every row. Raw and augmented are the two CLS heads. Raw/canonical uses the canonical threshold, and matched uses a threshold from genuine images processed in the same way. Global is the augmented head with one threshold from a mixture of five conditions (579, 621, 579, 627, and 594 calibration images for canonical, JPEG 75, resize 128, native 512, and padding). The global threshold’s false-alarm guarantee holds only on average over this mixture. Machine-readable files hold the intervals for each condition.

Comparison	Collection	Endpoint	Baseline achieved %	Other achieved %	$\Delta$ pp [95%]	Unavailable views baseline/other
A: head change	NIH	Genuine flags	5.89	4.57	-1.32 [-1.96, -0.67]	0/0
A: head change	Cheff	Generated rejection	94.31	99.28	+4.97 [+4.45, +5.51]	0/0
A: head change	SynthCheX	Generated rejection	73.84	89.28	+15.45 [+14.47, +16.51]	0/0
B: full policy	NIH	Genuine flags	39.90	4.57	-35.33 [-36.32, -34.31]	0/0
B: full policy	Cheff	Generated rejection	99.55	99.28	-0.28 [-0.43, -0.14]	0/0
B: full policy	SynthCheX	Generated rejection	93.28	89.28	-4.00 [-4.78, -3.18]	0/0

Table S25: Condition-averaged percentages and paired differences (other minus baseline, in percentage points) at the nominal 5% level, with bootstrap 95% intervals over 1,000 images. Each row uses 1,000 images seen under 13 conditions, not 13,000 independent images. Comparison A changes the raw head to the augmented head, with matched thresholds on both sides. B changes from the raw head with the canonical threshold to the augmented head with matched thresholds. C keeps the raw head and changes the canonical threshold to matched thresholds. The last column gives unavailable views (baseline/other).

Method	Strategy	Cheff: passed both / submitted	SynthCheX: passed both / submitted
Pilot CLS	Uniform	0/20 [0.0, 16.1]%	1/20 [0.9, 23.6]%
Pilot CLS	Quality	0/20 [0.0, 16.1]%	0/20 [0.0, 16.1]%
Pilot CLS	Surrogate	0/20 [0.0, 16.1]%	1/20 [0.9, 23.6]%
DINOv3 CLS raw	Uniform	0/20 [0.0, 16.1]%	2/20 [2.8, 30.1]%
DINOv3 CLS raw	Quality	0/20 [0.0, 16.1]%	1/20 [0.9, 23.6]%
DINOv3 CLS raw	Surrogate	0/20 [0.0, 16.1]%	2/20 [2.8, 30.1]%

Table S26: Images that pass both checks among the fixed submitted lists at

B=1{,}000

and

K=20

, part 1 of 2. Cells give passes over actual submissions, with Wilson 95% intervals in percent. Together the two parts cover all seven heads and three strategies. Surrogate selection is constrained by Qwen quality, which differs from the independent MedGemma check. Each head uses its own nominal 5% threshold. Unavailable replies cannot pass and stay in the denominators.

Method	Strategy	Cheff: passed both / submitted	SynthCheX: passed both / submitted
DINOv3 CLS augmented	Uniform	0/20 [0.0, 16.1]%	2/20 [2.8, 30.1]%
DINOv3 CLS augmented	Quality	0/20 [0.0, 16.1]%	1/20 [0.9, 23.6]%
DINOv3 CLS augmented	Surrogate	0/20 [0.0, 16.1]%	1/20 [0.9, 23.6]%
Rubric fields	Uniform	12/20 [38.7, 78.1]%	8/20 [21.9, 61.3]%
Rubric fields	Quality	19/20 [76.4, 99.1]%	17/20 [64.0, 94.8]%
Rubric fields	Surrogate	19/20 [76.4, 99.1]%	14/20 [48.1, 85.5]%

Table S27: Continuation of Table S26 , part 2 of 2.

Method / endpoint	AP count; %; CI	PA count; %; CI	Direct std. rate	No Finding count; %; CI	Other finding count; %; CI	Finite / U of 2297
DINOv3 CLS raw False flags	28/541 5.18% [3.60, 7.38]	82/1756 4.67% [3.78, 5.76]	4.76%	71/1594 4.45% [3.55, 5.58]	39/703 5.55% [4.08, 7.49]	2297/0
DINOv3 CLS aug. False flags	20/541 3.70% [2.41, 5.64]	77/1756 4.38% [3.52, 5.45]	4.26%	57/1594 3.58% [2.77, 4.60]	40/703 5.69% [4.21, 7.66]	2297/0
DINOv3 mean False flags	25/541 4.62% [3.15, 6.73]	98/1756 5.58% [4.60, 6.75]	5.41%	81/1594 5.08% [4.11, 6.27]	42/703 5.97% [4.45, 7.98]	2297/0
Medical mean False flags	35/541 6.47% [4.69, 8.86]	72/1756 4.10% [3.27, 5.13]	4.52%	38/1594 2.38% [1.74, 3.26]	69/703 9.82% [7.83, 12.24]	2297/0
MedGemma alt. Quality passes	257/541 47.50% [43.33, 51.72]	1460/1756 83.14% [81.32, 84.82]	76.80%	1330/1594 83.44% [81.53, 85.18]	387/703 55.05% [51.36, 58.69]	2219/78

Table S28: NIH images by recorded projection (AP, PA) and recorded findings (No Finding, other finding). The table shows origin flags for four heads at their nominal 5% thresholds and passes of the primary quality check. Cells give count/attempted, percentage, and Wilson 95% interval. Attempted counts are 541 AP, 1,756 PA, 1,594 No Finding, and 703 other-finding images. Finite/U gives the finite scores (schema-valid replies for quality) and the unavailable outputs out of 2,297. The standardized rate uses weights 0.178 (AP) and 0.822 (PA) and has no interval. The strata are observational.

Condition	Raw median shift	Raw canonical IQR	Augmented median shift	Augmented canonical IQR
Border4	$-4.77\times 10^{-6}$	$0.00604$	$+1.19\times 10^{-5}$	$0.00106$
Padding4	$+0.000434$	$0.00604$	$+9.69\times 10^{-5}$	$0.00106$
Central64	$-0.000759$	$0.00604$	$-1.37\times 10^{-5}$	$0.00106$
Border26	$+0.000918$	$0.00604$	$+0.000285$	$0.00106$
JPEG75	$+0.2$	$0.00604$	$+6.11\times 10^{-5}$	$0.00106$
Resize128	$+0.295$	$0.00604$	$+0.000114$	$0.00106$

Table S29: Paired shifts in NIH scores (condition minus canonical) for the same 1,000 images, for the raw and the augmented head. IQR is the head’s canonical score interquartile range, which gives a scale for the shifts. All 1,000 pairs are finite. Score scales differ between heads, and the scores are not probabilities of acquisition. Border4 and Central64 mask the stated pixels, Border26 covers 36.5% of the image, and Padding4 rescales the content before it adds a frame.

Collection	Pixel statistic	DINOv3 CLS raw $\rho$ [95% CI] finite/attempted; U	DINOv3 CLS aug. $\rho$ [95% CI] finite/attempted; U
NIH	Mean	0.225 [0.185, 0.265] n=2297/2297; U=0	0.298 [0.262, 0.336] n=2297/2297; U=0
NIH	Std.	0.163 [0.124, 0.203] n=2297/2297; U=0	0.264 [0.226, 0.302] n=2297/2297; U=0
NIH	Entropy	0.285 [0.245, 0.322] n=2297/2297; U=0	0.396 [0.361, 0.430] n=2297/2297; U=0
NIH	Border	0.116 [0.076, 0.156] n=2297/2297; U=0	0.129 [0.087, 0.167] n=2297/2297; U=0
NIH	JPEG	-0.083 [-0.123, -0.042] n=2297/2297; U=0	-0.129 [-0.170, -0.087] n=2297/2297; U=0
NIH	High freq.	-0.060 [-0.100, -0.017] n=2297/2297; U=0	-0.034 [-0.075, 0.010] n=2297/2297; U=0

Table S30: Spearman correlations between origin scores and six pixel statistics within each collection, part 1 of 4. The four parts together give all 72 canonical correlations (six statistics, three collections, four heads). Cells give the coefficient, its bootstrap 95% interval, the finite/attempted counts, and the unavailable count U. The correlations are observational and do not show a causal mechanism. Border is the border-to-interior gradient ratio, JPEG is the blockiness ratio, and High freq. is the high-frequency fraction.

Collection	Pixel statistic	DINOv3 CLS raw $\rho$ [95% CI] finite/attempted; U	DINOv3 CLS aug. $\rho$ [95% CI] finite/attempted; U
Cheff	Border	-0.143 [-0.202, -0.083] n=1000/1000; U=0	-0.146 [-0.206, -0.087] n=1000/1000; U=0
Cheff	JPEG	0.191 [0.128, 0.251] n=1000/1000; U=0	0.179 [0.118, 0.236] n=1000/1000; U=0
Cheff	High freq.	0.060 [-0.004, 0.124] n=1000/1000; U=0	0.038 [-0.024, 0.101] n=1000/1000; U=0
SynthCheX	Mean	0.035 [0.005, 0.062] n=5000/5000; U=0	0.057 [0.028, 0.086] n=5000/5000; U=0
SynthCheX	Std.	-0.068 [-0.095, -0.039] n=5000/5000; U=0	-0.098 [-0.125, -0.069] n=5000/5000; U=0
SynthCheX	Entropy	0.034 [0.006, 0.062] n=5000/5000; U=0	0.047 [0.020, 0.074] n=5000/5000; U=0

Table S31: Continuation of Table S30 , part 2 of 4.

Collection	Pixel statistic	DINOv3 mean $\rho$ [95% CI] finite/attempted; U	Medical mean $\rho$ [95% CI] finite/attempted; U
NIH	Mean	0.178 [0.139, 0.219] n=2297/2297; U=0	0.325 [0.289, 0.364] n=2297/2297; U=0
NIH	Std.	0.047 [0.005, 0.090] n=2297/2297; U=0	0.274 [0.237, 0.312] n=2297/2297; U=0
NIH	Entropy	0.194 [0.154, 0.234] n=2297/2297; U=0	0.415 [0.382, 0.447] n=2297/2297; U=0
NIH	Border	0.054 [0.012, 0.095] n=2297/2297; U=0	0.099 [0.056, 0.141] n=2297/2297; U=0
NIH	JPEG	-0.028 [-0.069, 0.012] n=2297/2297; U=0	-0.055 [-0.100, -0.013] n=2297/2297; U=0
NIH	High freq.	-0.006 [-0.047, 0.036] n=2297/2297; U=0	-0.111 [-0.152, -0.069] n=2297/2297; U=0

Table S32: Continuation of Table S30 , part 3 of 4.

Collection	Pixel statistic	DINOv3 mean $\rho$ [95% CI] finite/attempted; U	Medical mean $\rho$ [95% CI] finite/attempted; U
Cheff	Border	-0.122 [-0.182, -0.061] n=1000/1000; U=0	-0.051 [-0.113, 0.013] n=1000/1000; U=0
Cheff	JPEG	0.150 [0.085, 0.210] n=1000/1000; U=0	0.158 [0.096, 0.217] n=1000/1000; U=0
Cheff	High freq.	0.019 [-0.043, 0.084] n=1000/1000; U=0	-0.012 [-0.075, 0.052] n=1000/1000; U=0
SynthCheX	Mean	-0.003 [-0.033, 0.024] n=5000/5000; U=0	0.219 [0.193, 0.246] n=5000/5000; U=0
SynthCheX	Std.	-0.126 [-0.153, -0.097] n=5000/5000; U=0	-0.165 [-0.193, -0.137] n=5000/5000; U=0
SynthCheX	Entropy	0.108 [0.081, 0.135] n=5000/5000; U=0	0.292 [0.265, 0.317] n=5000/5000; U=0

Table S33: Continuation of Table S30 , part 4 of 4.

Encoder	Normalization	Contrast	N per panel	Unbiased MMD 2	Fixed-panel 95% interval
DINOv3 CLS	Raw	NIH–NIH	500	3.7091e-06	[4.6171e-06, 4.0745e-05]
DINOv3 CLS	Raw	Cheff–NIH	500	0.0014219	[0.0012872, 0.0016208]
DINOv3 CLS	Raw	SynthCheX–NIH	500	0.002268	[0.002109, 0.0024757]
DINOv3 CLS	Angular	NIH–NIH	500	0.00074518	[0.00088802, 0.0073874]
DINOv3 CLS	Angular	Cheff–NIH	500	0.24395	[0.22127, 0.2778]
DINOv3 CLS	Angular	SynthCheX–NIH	500	0.39077	[0.36331, 0.42667]

Table S34: Unbiased squared MMD between fixed panels of 500 images, for raw and angular-normalized features of each encoder. The panels are drawn from the encoded pools (2,297 NIH, 1,000 Cheff-identified, 5,000 SynthCheX images). Each generated collection is compared with NIH, and the NIH–NIH rows are the reference. Intervals come from 2,000 resamples of base images, and the intervals for the NIH–NIH rows show spread only. Values are comparable only within one encoder and normalization, and the unbiased estimate can be negative.

Encoder	Normalization	N per panel	Cheff minus SynthCheX MMD 2 difference	Paired fixed-panel 95% interval
DINOv3 CLS	Raw	500	-0.00084617	[-0.0010546, -0.00062756]
DINOv3 CLS	Angular	500	-0.14682	[-0.18179, -0.10982]
DINOv3 mean	Raw	500	-3.5471e-05	[-8.6209e-05, 1.9759e-05]
DINOv3 mean	Angular	500	-0.010572	[-0.022885, 0.0029512]
Medical mean	Raw	500	0.019727	[0.005187, 0.033548]
Medical mean	Angular	500	0.00023329	[-0.0021904, 0.0024338]

Table S35: Cheff minus SynthCheX difference in squared MMD to the NIH panel, for each encoder and normalization. A negative value means that Cheff-identified images are closer to NIH. Intervals come from joint resampling of the same panels, not from the marginal intervals.

Figure S5: Nearest-neighbor examples chosen by a fixed rule. Neighbors come from the NIH fit images only, and NIH queries and fit images come from different patients because the splits are patient-disjoint. “DINO” in the figure means DINOv3. Exact identity checks and proximity in a representation do not establish that memorization is absent, that the images are clinically faithful, or that they were excluded from model pretraining.

Figure S6: Examples for the four outcomes of the two checks under the primary canonical policy. A fixed rule picks up to three examples with the lowest hashes in each cell: generated images that pass both checks, genuine images that pass quality but are flagged, genuine images that fail quality with a valid reply but are not flagged, and generated images that are flagged. Unavailable replies are excluded from the quality-failure cell, and the rate tables keep them. In the figure, “origin screen” is the origin check, “quality gate” is the quality check, “suspicion” is the origin score, and “matching bases” is the number of test images in each outcome (both generators together for generated images). The examples are model outcomes, not clinical assessments. Open-I images are not redistributed.

Stage	Attempted tasks	Generation (s)	Last-process wall (s)
Qwen original quality	8297	25391.5	25476.9
Qwen surrogate	6000	20397.2	–
Qwen bounded pairwise	584	2291.8	2321.8
MedForge fixed panel	900	8315.4	8402.0
MedForge selected extras	195	1772.9	1797.2

Table S36: Measured inference costs for selected stages, copied from the run records. Generation is the recorded time of batch generation, added over all runs. Wall time includes loading in the last process, and it is not a total for the whole allocation when a run was resumed. A dash means unavailable, not zero. Tasks in different stages may reuse the same images. These are not a project-wide wall time or GPU-hour estimate.

Data Provenance

Data provenance tracks where data and AI outputs come from, how they are transformed, and what evidence supports their attribution and use.

See all 185 papers

Jun 17, 2026

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

We introduce the first generative foundation model for chest radiograph synthesis trained from scratch at the billion-parameter scale. Existing radiographic AI models often suffer from poor generalisation across patient subpopulations, institutions, and acquisition settings, resulting in limited real-world clinical utility. Controlled, high-fidelity synthesis of chest radiographs is a promising path toward diversifying clinical datasets and evaluating the robustness of diagnostic models. Therefore, we present the largest specialist generative foundation model for chest radiographs to date, with over 1.3B parameters, trained for 1.6T tokens on a curated, heterogeneous dataset comprising 1.2M radiographs and clinical expert-guided metadata. Our model supports controllable radiograph generation and editing across multiple demographic subgroups, acquisition views, and a dozen pathologies. Moreover, we significantly advance the state of the art in radiograph synthesis fidelity, producing images that are indistinguishable from real radiographs to clinical experts.

Fabio De Sousa Ribeiro, Emma A. M. Stanley, Charles Jones +7

Imperial College London · Causality in Healthcare AI Hub · Cleveland Clinic London +4

Medical Imaging Foundation Models Medical Image Synthesis Conditional Generative Modeling

Ask PDF

Aug 8, 2026

JustLLMGRPO: Radiographic Control for Chest X-Ray Generation

Text-conditioned chest X-ray generation aims to synthesize realistic radiographs that faithfully depict specified findings. Existing work has primarily improved quality by updating image generators, implicitly treating prompts as fixed after CXR-domain adaptation. We show that this generator-centric view leaves a substantial optimization dimension underexplored. With a CXR-adapted Sana generator frozen, one-pass reformulation by an unmodified LLM reduces RadDINO-FID from 54.225 to 27.572. Prompt analysis shows that the LLM suppresses temporal comparisons, uncertainty, and other non-renderable report content while emphasizing visible radiographic findings. However, unconstrained reformulation reduces BioViL-T alignment with source prompts from 0.695 to 0.609. We therefore introduce JustLLMGRPO, which applies standard Group Relative Policy Optimization (GRPO) only to the LLM prompt policy while keeping Sana frozen. Group-relative radiology-aware image feedback retains visual focus while preserving source-prompt alignment. On CheXGenBench, JustLLMGRPO reduces RadDINO-FID to 26.780, a 50.6% improvement over direct prompting, while maintaining alignment (0.696 versus 0.695). It also achieves state-of-the-art distribution coverage and downstream classification utility. These results show that substantial performance can remain latent in how radiographic information is expressed to an adapted generator. Code is publicly available at https://github.com/pxcai/JustLLMGRPO.

Pengxiang Cai, Xiaohan Li, Anglin Liu +3

The Hong Kong University of Science and Technology (Guangzhou)

T2I Generation RL for Image Generation Prompt Optimization for T2I Generation Medical Image Synthesis

Ask PDF

Apr 13, 2026

GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rays

We introduce GazeVaLM, a public eye-tracking dataset for studying clinical perception during chest radiograph authenticity assessment. The dataset comprises 960 gaze recordings from 16 expert radiologists interpreting 30 real and 30 synthetic chest X-rays (generated by diffusion based generative AI) under two conditions: diagnostic assessment and real-fake classification (Visual Turing test). For each image-observer pair, we provide raw gaze samples, fixation maps, scanpaths, saliency density maps, structured diagnostic labels, and authenticity judgments. We extend the protocol to 6 state-of-the-art multimodal LLMs, releasing their predicted diagnoses, authenticity labels, and confidence scores under matched conditions - enabling direct human-AI comparison at both decision and uncertainty levels. We further provide analyses of gaze agreement, inter-observer consistency, and benchmarking of radiologists versus LLMs in diagnostic accuracy and authenticity detection. GazeVaLM supports research in gaze modeling, clinical decision-making, human-AI comparison, generative image realism assessment, and uncertainty quantification. By jointly releasing visual attention data, clinical labels, and model predictions, we aim to facilitate reproducible research on how experts and AI systems perceive, interpret, and evaluate medical images. The dataset is available at https://huggingface.co/datasets/davidcwong/GazeVaLM.

David Wong, Zeynep Isik, Bin Wang +22

Northwestern University, Evanston, IL, United States · Northwestern University, Evanston, IL, USA · Université Sorbonne Paris Nord, Paris, France +5

Medical Image Benchmarks VLM Evaluation Eye Tracking Chest X-Ray Classification

Ask PDF

Abstract

Data Provenance

Explore similar work

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

JustLLMGRPO: Radiographic Control for Chest X-Ray Generation

GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rays