Anatomy-Preserving Unpaired Medical Image Translation via Shared Latent Anchoring
Organizations: Stony Brook University, NY, USA · University of North Carolina at Charlotte, NC, USA
Abstract
Unpaired medical image-to-image translation requires synthesizing target-modality images from source-modality images without aligned supervision. Existing methods learn a direct source-to-target mapping that entangles structure preservation and appearance generation within a single model, often distorting clinically meaningful anatomy in the translated output. We argue that these two objectives should be decoupled entirely: anatomical structure should be captured independently of generation, and the generator should operate only on a frozen structural representation. To this end, we propose a two-stage framework for unpaired medical image translation. In the first stage, we learn a shared structure space by training a structure encoder-decoder on source images, target images, and structural masks jointly, using mask reconstruction to anchor the latent space to modality-agnostic anatomy. For source images without mask annotations, a target-domain memory bank provides soft structural supervision via prototype retrieval. In the second stage, a conditional flow model learns to render target-domain appearance under fixed structural conditions. This ensures the generator inherits target appearance statistics and can focus entirely on structural conditioning rather than learning appearance from scratch. We validate our framework on OCT-to-OCTA synthesis and brain MR-to-CT translation. Our method outperforms all unpaired baselines on both tasks and, despite requiring no paired supervision, also surpasses paired baselines, improving PSNR by up to 11.2% and SSIM by up to 26.4% on OCT-to-OCTA, and achieving consistent gains across whole-image, soft-tissue, and bone regions on MR-to-CT. The project repository is available on GitHub.
Figures & tables
| Method | Paired | Whole Image | Vessel | FAZ | |||
|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||
| CycleGAN [ 6 ] | |||||||
| CUT [ 7 ] | |||||||
| MUNIT [ 9 ] | |||||||
| SynDiff [ 1 ] | |||||||
| UNSB [ 15 ] | |||||||
| Method | Paired | Whole Image | Soft Tissue | Bone | |||
|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||
| CycleGAN [ 6 ] | |||||||
| CUT [ 7 ] | |||||||
| MUNIT [ 9 ] | |||||||
| SynDiff [ 1 ] | |||||||
| UNSB [ 15 ] | |||||||
| Setting | ||||
|---|---|---|---|---|
| w/o | ||||
| w/o | ||||
| Ours |
| Method | Paired | Whole Image | Soft Tissue | Bone | |||
|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||
| SynDiff | No | 24.10 | 94.68 | 23.18 | 50.71 | 13.71 | 62.31 |
| DDBM | Yes | 25.13 | 95.79 | 23.27 | 58.71 | 14.10 | 67.49 |
| Ours | No | 27.98 | 97.05 | 27.15 | 59.17 | 17.14 | 80.45 |
| Input | AUROC | Bal. Acc. | Flip rate | |
|---|---|---|---|---|
| Acquired OCTA | – | – | ||
| MuTri | ||||
| SynDiff | ||||
| UNSB | ||||
| MUNIT | ||||
| Ours |
| Method | Vessel Dice | clDice | Density Err. | Halluc. | Omiss. | #CC Err. |
|---|---|---|---|---|---|---|
| MUNIT | ||||||
| MuTri | ||||||
| Ours |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| PSNR | SSIM | |
|---|---|---|
| 0.25 | ||
| 0.5 | ||
| 1 |
| PSNR | SSIM | |
|---|---|---|
| 0.5 | ||
| 1 | ||
| 2 |
| PSNR | SSIM | |
|---|---|---|
| 0.1 | ||
| 0.2 | ||
| 0.4 |
| Cross branch | Whole Image | Vessel | FAZ | ||||
|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||
| ✓ | |||||||
| ✓ | |||||||
| ✓ | ✓ | ||||||
| Target masks | Mask Dice | Whole Image | Vessel | FAZ | ||||
|---|---|---|---|---|---|---|---|---|
| Vessel | FAZ | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
| Official | ||||||||
| Automatic | ||||||||
| Subset | Image / Method | Own-subject Dice | Prototype Affinity | Subject Margin |
|---|---|---|---|---|
| All 50 | Acquired OCTA | |||
| MuTri | ||||
| Ours | ||||
| Lowest retrieval- confidence 10 | Acquired OCTA | |||
| MuTri | ||||
| Ours |
| Method | Fidelity | Missing | Unsupported | Acceptability |
|---|---|---|---|---|
| SynDiff | ||||
| MuTri | ||||
| Ours |