When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Images
Authors: Andrea Posada, Wenke Karbole, Bach Ngoc Doan, Alexander Weers, Solmaz Abdolrahimzadeh, Maria Patsiamanidi, Kahkashan Haider, Vaishali Khare, +4 more
Organizations: Technical University of Munich and TUM University Hospital, Germany · Munich Center of Machine Learning (MCML), Munich, Germany · Munich Data Science Institute (MDSI), Munich, Germany · Sapienza University of Rome, Rome, Italy · Faculty of Medicine, University of Southampton, Southampton, United Kingdom · Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom · Imperial College London, UK
Counterfactual medical image generation aims to modify an existing image to reflect a hypothetical scenario in which certain characteristics of the imaged subject are altered, while keeping their identity fixed. Most existing works repurpose established image editing methods, which do not directly supervise identity preservation. Instead, they assume that identity is implicitly preserved by anchoring generation to the source image. This assumption is rarely tested and may fail in domains where biometric cues are subtle, such as retinal optical coherence tomography (OCT). In this work, we explicitly measure identity preservation for three groups of text-conditioned editing methods - source-anchored, structured-prompt, and paired-training - using referee classifiers, embedding alignment scores, and a blind reader study. We find that all methods produce high-quality OCT images with comparable editing success, yet their identity preservation differs markedly. Source-anchored editing frequently alters the depicted subject, while paired-training preserves it best. We argue that future work on medical counterfactual generation must explicitly measure and report identity preservation alongside image realism and editing success.
Figures & tables
Figure 1: In OCT counterfactual generation, changing global age-related macular degeneration (AMD) biomarkers requires strong edits that can disrupt the subtle cues linked to patient identity. As a result, the generated samples often fail to preserve the subject’s identity even when the edits appear faithful and realistic.
Figure 2: Per-method comparison on two selection criteria across realism, editing success, and identity preservation metrics. Selecting on realism and editing success alone (dashed line) can yield configurations with substantial loss of identity features. The balanced pick (solid line) recovers identity preservation while editing success and realism are largely retained.
Figure 3: Two displayed images (real/real) or (real/edited) were graded by four clinically trained readers as the same or different eyes with substantial agreement (Fleiss’s κ=0.67 ). While edits generated by P2P and InstructPix2Pix are mostly recognized as belonging to the same eye, those of SDEdit reveal frequent identity loss.
Figure 4: With increasing editing strength, all investigated text-conditioned editing strategies show a loss in identity preservation, most prominently revealed by Agreye,Agrsex . Above all, we find that even minor edits can cause a drastic loss of identity features, which is not equally compensated by gains in editing success metrics.