The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond

Andreas Maier; Monica Hinrichs-Mayer; Franziska Weber; Niklas Lackner; Matthias May; Bernhard Kainz; Siming Bayer

cs.CLSep 27, 2026

The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond

Authors: Andreas Maier, Monica Hinrichs-Mayer, Franziska Weber, Niklas Lackner, Matthias May, Bernhard Kainz, Siming Bayer

Organizations: Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Universitätsklinikum Erlangen, Germany · Imperial College London, United Kingdom

Abstract

Medical images are released with the reports that describe them, and protecting the image does not protect the report. This paper measures the text component of such releases. We measure identifier detection, downstream utility and residual identity leakage on the same documents, with the pseudonymisation policy as the variable under test: 15 detectors, three release conditions and four corpora of medical reports, legal judgments, news and other genres, and e-mail, in German, English, Chinese and Arabic. A fixed 13-detector union reaches a person sensitivity of 0.9998 at specificity 0.8686 on the medical reports, 0.9958 at 0.8504 on the legal judgments, 0.9352 at 0.9318 on news and other genres, and 0.9906 at 0.6235 on e-mail. With this ensemble, frequency matching with a public name list recovers zero identities by alignment across the four corpora; the names it got right were ones the detector missed, left in clear text. Cross-document linkage ranks the correct person first for 0.93% of e-mail queries without training and 3.94% with it, against 1/3697 chance and 71.98% on unmodified text. On the medical reports it recovers nothing without training and 0.71% of 138 queries with it, against 1/207 chance and a 2.73% ceiling on unmodified text.

Figures & tables

Corpus	Docs	Tokens	Ment./doc	Lang.	Reference annotation
CARDIO:DE	$400$	$885\,059$	$78.3$	de	constructed
TAB	$1268$	$1\,844\,804$	$81.2$	en	manual
OntoNotes	$5994$	$4\,726\,594$	$45.5$	en/zh/ar	linguistic
Enron	$58\,636$	$15\,507\,925$	$12.2$	en	header-derived

Table 1 : The corpora separate domain, language, and where the reference annotation came from.

Operating point	Sens.	Spec.	Cost	Sens.	Spec.	Cost	Sens.	Spec.	Cost	Sens.	Spec.	Cost
	CARDIO:DE			TAB			OntoNotes			Enron
max. sensitivity	$0.9919$	$0.9670$	$57.633^{\cup}$	$0.9253$	$0.8986$	$14.210^{\cup}$	$0.7466$	$0.9763$	$9.450^{\cup}$	$0.9968$	$0.7745$	$25.557^{\cup}$
fast sensitivity	$0.8947$	$0.9769$	$0.301^{\cup}$	$0.8926$	$0.9242$	$0.219^{\cup}$	$0.6843$	$0.9731$	$5.376^{\cup}$	$0.9222$	$0.7420$	$0.057^{\cup}$
max. specificity	$0.5230$	$0.9997$	$116.875^{v}$	$0.5014$	$0.9959$	$98.110^{v}$	$0.5034$	$0.9960$	$0.103^{\cup}$	$0.5292$	$0.9448$	$20.931^{\cap}$
fast specificity	$0.7807$	$0.9862$	$0.263^{s}$	$0.5198$	$0.9934$	$0.117^{s}$	$0.5034$	$0.9960$	$0.103^{\cup}$	$0.6920$	$0.8781$	$0.026^{s}$

Table 2 : Operating points on both error rates. Sensitivity is over the identifier types the conditions replace, specificity over every other token. Cost is seconds per document; the rule is

\cup

union,

\cap

intersection,

v

at-least-

k

vote,

s

a single detector.

Corpus	Sens.	Spec.	Exp. (%)	Frequency	Context link.	Learned link.	Chance
CARDIO:DE	$0.9998$	$0.8686$	$0.05$	0 (1)	$0$	$0.71\pm 1.60$	$1/207$
Enron	$0.9906$	$0.6235$	$1.60$	0 (0)	$0.93\pm 0.10$	$3.94\pm 0.30$	$1/3697$
TAB	$0.9958$	$0.8504$	$0.83$	0 (1)	no links $\rightarrow$ not measurable		–
OntoNotes	$0.9352$	$0.9318$	$5.43$	0 (1)	no links $\rightarrow$ not measurable		–

Table 3 : Leakage from the recommended 13-detector union; all columns concern people. Sens. is person sensitivity and Spec. the share of non-identifier tokens left alone: neither is readable without the other. Exp. is the risk that a person is still named in a given document after processing, Frequency the identities named from frequency alone with a public list and, bracketed, with the corpus’s own distribution (an upper bound, not a risk). Linkage is the percentage of queries ranking the correct person first, over the complete candidate list, against chance

1/\text{gallery}

Explore similar work

MirrorNet: Can Medical Image Anonymization Really Protect Patient Identity?
Aug 6, 2026Attila SimkóMedical Images Biometrics
Department of Diagnostics and Intervention Umeå University, Umeå, Sweden
MedDeID enables locally governed clinical-text de-identification from real or synthetic training data
Sep 9, 2026Stig Hellemans, Tom Stroobants, Elyne Scheurwegs +3De-Identification Clinical Notes
Adrem Data Lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium · Antwerp University Hospital (UZA), Edegem, Belgium · Laboratory of Experimental Medicine and Pediatrics (LEMP), University of Antwerp, Antwerp, Belgium
Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
Jun 24, 2026Ching-Hao Chiu, Hao-Wei Chung, Gelei Xu +7Synthetic Image Detection Multimodal Clinical Data
University of Notre Dame, Notre Dame, IN, USA. · IBM Research, Yorktown Heights, NY, USA. · Department of Cardiology, Boston Children’s Hospital, Boston, MA, USA. +2

Aug 6, 2026, cs.CVJ/K move · Enter open · S save

MirrorNet: Can Medical Image Anonymization Really Protect Patient Identity?

Attila Simkó

Department of Diagnostics and Intervention Umeå University, Umeå, Sweden

Medical Images Biometrics Patients Cone-Beam Computed Tomography Conditional Variational Autoencoder

Open paper AskPDF arXiv

Medical images are routinely de-identified---names, dates, and other metadata removed---and then shared for research, teaching, and public benchmarks under the assumption that this renders them anonymous. Such de-identification protects the metadata but not the pixels, and---apart from scans that directly contain facial structures---whether the image content itself identifies the patient has received little scrutiny. We investigate this question by learning a cycle-consistent correspondence between a cross-sectional medical image and a non-medical, patient-identifying image, using a pair of coupled, cycle-consistent variational autoencoders. From a held-out scan, the model recovers a recognisable likeness of the patient (identity-region MAE = 0.163); conversely, it synthesises a scan from such an image. These results indicate that a de-identified medical scan remains identifying---it is, in effect, a photograph of the patient---and that imaging data should be governed as biometric data rather than as anonymisable records. To support reproducibility, the code and trained models are shared at https://github.com/attilasimko/public-repository.

Abstract

Explore similar work

MirrorNet: Can Medical Image Anonymization Really Protect Patient Identity?

MedDeID enables locally governed clinical-text de-identification from real or synthetic training data

Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection