Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge

Jiaming Liu; Felix Petersen; Yunhe Gao; Yabin Zhang; Hyojin Kim; Akshay S. Chaudhari; Yu Sun; Stefano Ermon

cs.CVFeb 18, 2026

Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge

Authors: Jiaming Liu, Felix Petersen, Yunhe Gao, Yabin Zhang, Hyojin Kim, Akshay S. Chaudhari, Yu Sun, Stefano Ermon,

Organizations: Stanford University · LLNL · Johns Hopkins University

Abstract

Adversarial diffusion and diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations. Adversarial approaches require target-domain adversarial loss during training, which can limit generalization to unseen data, while diffusion-inversion methods often produce low-fidelity translations due to imperfect inversion into noise-latent representations. In this work, we propose the Self-Supervised Semantic Bridge (SSB), a versatile framework that integrates external semantic priors into diffusion bridge models to enable spatially faithful translation without cross-domain supervision. Our key idea is to leverage self-supervised visual encoders to learn representations that are invariant to appearance changes but capture geometric structure, forming a shared latent space that conditions the diffusion bridges. Extensive experiments show that SSB outperforms strong prior methods for challenging medical image synthesis in both in-domain and out-of-domain settings, and extends easily to high-quality text-guided editing.

Figures & tables

Figure 1: Overview of our Self-Supervised Semantic-Bridge (SSB) framework for unpaired image translation and editing. SSB trains without paired data or adversarial objectives, relying on a shared latent-space assumption to connect domains via a common representation; $\times$ denotes no cross-domain supervision. The top rows showcase image-to-image translation results using text-free guidance, while the bottom rows demonstrate text-guided editing based on Stable Diffusion 3-Medium Esser et al. (2024) . This single-domain training enables unpaired translation across medical and natural images with improved structural consistency and OOD robustness (e.g., unseen MRI contrasts at test-time).

Figure 2: Unlike inversion-based methods that invert toward an unstructured Gaussian noise, SSB defines a unified semantic latent endpoint ${\bm{y}}=E_{\phi}({\bm{x}})$ using a self-supervised visual encoder and trains domain-specific bridges independently to connect each domain to this shared endpoint, enabling reliable translation by composing source-to- $T$ inversion with target-bridge generation.

Figure 3: Cross-modal semantic alignment and improvement in the shared latent space. Left: PCA visualizations of intermediate feature maps for paired CT/MRI inputs. Our ViT-B/8 encoder yields smoother and more anatomically consistent representations across modalities. Middle: Quantitative alignment on paired MRI–CT validation samples, comparing global alignment (cosine similarity) and local structural consistency (CKNNA). Right: Semantic improvement $\Delta_{\text{cosim}}$ measures the net gain in feature alignment with the ground-truth CT ( gtCT ) achieved by our generated translation ( genCT ) over the original input ( MRI ). Formally, $\Delta_{\text{cosim}}=\operatorname{cosim}(E_{\phi}(\text{genCT}),E_{\phi}(\text{gtCT}))-\operatorname{cosim}(E_{\phi}(\text{MRI}),E_{\phi}(\text{gtCT}))$ . We plot this gain against the initial input alignment $x$ . The dashed line $y=1-x$ represents the theoretical ceiling, corresponding to perfect recovery of the ground-truth features (similarity $=1$ ). The improvements confirm that our method effectively minimizes semantic divergence, largely mitigating the impact of the initial domain gap.

Method	MRI $\rightarrow$ CT		MRI $\rightarrow$ CT (OOD)
Method	MS-SSIM* $\uparrow$	PSNR* $\uparrow$	FID $\downarrow$	MS-SSIM $\uparrow$
CycleGAN Zhu et al. (2017)	0.657	18.86	127.22	0.514
UNIT Liu et al. (2017)	0.712	20.88	115.84	0.458
SDEdit Meng et al. (2022)	0.628	18.87	50.17	0.385
DDIB Su et al. (2023)	0.625	18.45	48.15	0.411
Syndiff Özbey et al. (2023b)	0.805	22.65	79.56	0.405

Table 1: Quantitative comparison on MRI

\rightarrow

CT (in-domain and out-of-domain (OOD)). (*) denotes comparison to pre-registered paired CT images. Best and second-best values are highlighted among methods trained without ground-truth supervision. The

\text{I}^{2}

SB and SelfRDB are supervised baselines.

Method	Horse $\rightarrow$ Zebra / Apple $\rightarrow$ Orange (256 $\times$ 256)
Method	CLIP-T $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
CycleGAN Zhu et al. (2017)	0.225	0.416	0.630	15.57
CUT Park et al. (2020)	0.227	0.425	0.607	15.39
SDEdit Meng et al. (2022)	0.280	0.335	0.611	17.84
DDIB Su et al. (2023)	0.263	0.387	0.595	16.51
CycleNet Xu et al. (2023)	0.305	0.301	0.719	19.75

Table 2: Quantitative comparison on Natural I2I translations.

Figure 4: Our SSB ensures anatomically consistent MRI $\rightarrow$ CT translation across in-domain and out-of-domain scenarios. Segmentation masks are overlaid on MRI source images in OOD settings to provide structural reference, as paired CT ground truth is unavailable. They are not used during training or inference and serve only to illustrate anatomical fidelity without segmentation supervision.

Figure 5: Visual comparison with recent diffusion editing methods using SD3-M base model. Our approach generates visually consistent results with the source image, preserving structural integrity while convincingly applying appearance changes across diverse scenarios.

Figure 6: Numerical results based on SD3-M model. Our method achieves a favorable balance between text adherence (CLIP-T) and structural preservation (DINO and PSNR) across both scene-editing and object-change tasks. Scene editing requires substantial global modifications, whereas object change focuses on localized adjustments. Connected markers represent different hyperparameter settings.

Figure 7: Balancing structure and appearance via vector-field control. Interpolating between domain flows ( e.g., Left: “lighthouse” → “clock tower” and Right: “water” → “milk” ) enables controllable trade-offs between structural consistency and appearance fidelity.

Figure 8: More results comparing with FlowEdit and ControlNet.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Visual Results. Qualitative results on natural image translation. We show unpaired horse→zebra and apple→orange examples comparing SSB to prior GAN- and diffusion-based baselines. Our method achieves strong appearance transfer (striping / color change) while better preserving object pose, background layout, and fine structural details. The SSB results are obtained by fine-tuning SiT-XL/2.

FID Dist. $\downarrow$	U-FAT	U-WATER	S-MRI	S-CT
U-FAT	0	73.5	97.6	177.2
U-WATER	—	0	110.4	191.6
S-MRI	—	—	0	120.7
S-CT	—	—	—	0

Appendix

Table 3: FID distances between MRI and CT datasets. U-FAT = UKBB fat-suppressed MRI; U-WATER = UKBB water-suppressed MRI; S-MRI = SynthRAD2023/2025 MRI; S-CT = SynthRAD2023/2025 CT. SynthRAD MRI is substantially closer to SynthRAD CT (FID 120.7) than UKBB MRI (FID 177–192), supporting our definition of SynthRAD MRI as “in-domain” and UKBB MRI as “OOD” with respect to CT acquisition.

Figure 10: Data augmentation pipeline used for MRI-CT DINOv2 vision encoder. We adapt the widely used spatial-temporal retinainspired filter for aligning MRI-CT appearance gap as illustrated in the bottom of this figure.

Figure 11: Full trade-off curves complementing the quantitative results shown in the main paper Fig 6. As in the main text, we plot text alignment (CLIP-T, x-axis) against multiple fidelity and perceptual metrics (y-axis), including LPIPS, DINO, PSNR, SSIM, and CLIP-I. These expanded plots show the complete range of guidance settings used for each method. Consistent with the main-paper results, our method provides a favorable balance between text adherence and structural preservation, comparing competitively with—and in many cases outperforming—state-of-the-art baselines such as FlowEdit. In addition, all methods exhibit smooth and monotonic behavior as guidance strength varies, confirming the stability of our evaluation protocol. ControlNet (Canny) performs substantially worse due to its edge-only conditioning, which discards most semantic and photometric information. Structural fidelity for scene-style editing metrics (SSIM, PSNR, LPIPS) are computed on the luminance (YCbCr) channel to better reflect perceptual structure consistency. Connected markers correspond to different hyperparameter values for each baseline.

Methods	$T$ steps	$t_{\text{end}}$ (Control Strength*)	$n_{\text{avg}}$	CFG @ source	CFG @ target
SDEdit [ 26 ]	75	0.2, 0.3, 0.4, 0.5, 0.6, 0.75	1	–	7.5, 13.5, 16.5
DDIB [ 27 ]	200	0.7, 0.8, 0.9	1	1, 3.5	13.5, 16.5, 19.5
FlowEdit [ 55 ]	50	0.5, 0.56, 0.6, 0.66, 0.7, 0.8	1, 3, 5	1, 3.5	13.5, 16.5, 19.5
ControlNet [ 13 ]	75	0.2, 0.5, 0.7, 0.8, 0.9	1	–	7.5, 13.5, 16.5
iRFDS [ 98 ]	1000, 1400, 1800	–	1	–	official hyperparameters

Appendix

Table 4: Control strengths for inversion/editing baselines (unified across each method’s original notation). For SDEdit, FlowEdit, and DDIB, a larger strength corresponds to weaker preservation of the source structure, whereas for ControlNet the relationship is reversed.

Task	Setting	Inversion Step	$\bm{T}$ steps	$\bm{t_{\text{end}}}$ ( Control Strength* )	$\bm{p_{w}}$ in Eq. ( 26 )	CFG @ target
Medical I2I	MRI $\rightarrow$ CT	–	50	–	–	–
Text-free I2I	Horse $\rightarrow$ Zebra	100	50	0.2, 0.4, 0.7	0.5, 0.7	3, 4.5
Text-free I2I	Apple $\rightarrow$ Orange	100	50	0.2, 0.4, 0.7	0.5, 0.7	3, 4.5
Text-guided I2I	Scene editing	–	75	0.4, 0.7, 1	0, 0.2, 0.5, 0.7	4.5, 7.5, 13.5
Text-guided I2I	Object editing	75	75	0, 0.2, 0.4, 0.7	0, 0.2, 0.5, 0.7	4.5, 7.5, 13.5

Appendix

Table 5: SSB hyperparameter configurations across different I2I translation tasks (medical and natural images). We utilize standard diffusion hyperparameters (e.g., inversion steps, control strength) consistent with prior I2I methods, adapting settings like stochasticity (

\bm{p_{w}}

) and guidance strength solely based on the task’s intrinsic ambiguity. This demonstrates that SSB is a general framework achieving strong performance across domains using established configurations, without relying on complex, task-specific heuristics.

Backbone: UNet-based [ 108 , 93 ]
Methods	NFE $\downarrow$	Params	Inference Time (s/image) $\downarrow$
SDEdit [ 26 ]	400	552.8M	8.03
DDIB [ 65 ]	1000	552.8M	20.15
Syndiff [ 2 ]	4	156.1M	0.13
DDBM [ 35 ]	300	489.7M	5.21
I 2 SB [ 37 ]	1000	552.8M	24.97

Appendix

Table 6: Efficiency Comparison on medical MRI

\rightarrow

256\times 256

images. All methods use a UNet backbone. We evaluate SDEdit, DDIB, and DDBM using our own pre-trained diffusion models within their official frameworks. For Syndiff, SelfDRB, and I 2 SB, we utilize their official implementations.

Methods	Base-model	NFE $\downarrow$	Inference Time (s/image) $\downarrow$	Parameters
SDEdit [ 26 ]	SiT-XL/2 [ 61 ]	400	9.22	675M
DDIB [ 27 ]	SiT-XL/2 [ 61 ]	1000	22.06	675M
CycleNet [ 25 ]	SD-2.1 [ 69 ]	200	5.21	865.91M
ControlNet [ 13 ]	SD3-M [ 52 ]	150	5.75	2.02B
SSB (Ours)	SiT-XL/2 [ 61 ]	250	6.56	675M

Appendix

Table 7: Efficiency Comparison on text-free I2I translation with Transformer-based diffusion denoisers . Methods differ in base architectures (e.g., SiT/SD/SD3 backbones), reflecting the recent shift from U-Net to Transformer denoisers. We use the official implementation of CycleNet and adapt publicly available weights for ControlNet-Canny .

Methods	Inference Time (s/image) $\downarrow$	Parameters
SDEdit [ 26 ]	4.71	2.02B
DDIB [ 27 ]	28.75
FlowEdit [ 55 ]	12.21
iRFDS [ 98 ]	44.82
ControlNet [ 13 ]	5.85
SSB (Ours)	14.13

Appendix

Table 8: Efficiency Comparison on text-guided I2I translation and editing. We employ pre-trained public available SD3-M for all methods. Since the NFE varies due to hyperparameter settings (e.g., control strength) for all baselines, we omit a single NFE value and instead report the average running time across different settings to reflect the actual computational cost more accurately.

Figure 12: Additional Visual Results. Additional qualitative results for in-domain MRI $\rightarrow$ CT translation.

Figure 13: Additional Visual Results. Additional qualitative results for out-of-distribution (OOD) MRI (UKBB whole-body water-series) $\rightarrow$ CT translation. Segmentation masks are overlaid on the MRI source images in OOD settings only to provide visual structural reference, since paired CT ground truth is unavailable. These masks are not used during training or inference; they serve solely to illustrate anatomical fidelity without any segmentation supervision.

Figure 14: Additional Visual Results. Additional qualitative results for out-of-distribution (OOD) MRI (UKBB whole-body fat-series) $\rightarrow$ CT translation. Segmentation masks are overlaid on the MRI source images in OOD settings only to provide visual structural reference, since paired CT ground truth is unavailable. These masks are not used during training or inference; they serve solely to illustrate anatomical fidelity without any segmentation supervision.

Figure 15: Additional Visualization Results. We present additional qualitative results for text-guided image scene editing using SD3-M [ 52 ] . Baseline methods are shown with numerical values in parentheses $(*)$ , indicating the control strengths defined in their original formulations. For ControlNet, we use Canny edges as the conditioning signal. For example, FlowEdit (33) denotes a control parameter of $n_{\max}=33$ , while ControlNet (0.7) refers to a conditioning scale of $\alpha=0.7$ , where larger values correspond to stronger Canny-based structural guidance.

Figure 16: Additional Visualization Results. We present additional qualitative results for text-guided object editing using SD3-M [ 52 ] . Baseline methods are shown with numerical values in parentheses $(*)$ , indicating the control strengths defined in their original formulations. For ControlNet, we use Canny edges as the conditioning signal. For example, FlowEdit (33) denotes a control parameter of $n_{\max}=33$ , while ControlNet (0.7) refers to a conditioning scale of $\alpha=0.7$ , where larger values correspond to stronger Canny-based structural guidance.

Figure 17: Limitations in Natural Image Editing and Translation. Top : For large semantic changes (small lizard → large dragon), SSB trades off between preserving the source structure and fully following the target prompt, and strong edits can distort background geometry. Bottom : With highly abstract guidance such as silhouettes, the model tends to produce stylized, painterly animals rather than fully photorealistic translations.

Unpaired Image-to-Image Translation

Unpaired image-to-image translation maps images between visual domains without requiring matched examples, while aiming to preserve meaningful content.

See all 47 papers

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation
Aug 30, 2026Xing Xie, Jiawei Liu, Shijun Zhou +4Masked Diffusion Models Image-to-Image Translation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China · University of Chinese Academy of Sciences, Beijing, China · School of Computing and Data Science, The University of Hong Kong, Hong Kong
Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation
Mar 19, 2025Suhyeon Lee, Kwanyoung Kim, Jong Chul YeImage-to-Image Translation Unpaired Image-to-Image Translation
Kim Jae Chul Graduate School of AI, KAIST · Samsung Research
Seed-to-Seed: Unpaired Image Translation in Diffusion Seed Space
Sep 1, 2024Or Greenberg, Eran Kishon, Dani LischinskiUnpaired Image-to-Image Translation Generative Adversarial Networks
General Motors R&D · The Hebrew University of Jerusalem

Aug 30, 2026J/K move · Enter open · S save

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation

Xing Xie, Jiawei Liu, Shijun Zhou, Huijie Fan+3

State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China · University of Chinese Academy of Sciences, Beijing, China · School of Computing and Data Science, The University of Hong Kong, Hong Kong

Masked Diffusion Models Image-to-Image Translation Diffusion Timestep Scheduling Diffusion-Based Image Editing Diffusion Models

Open paper AskPDF arXiv

We propose Discrete Diffusion Bridges (DDB), a novel framework designed to resolve the fundamental spatiotemporal misalignment of standard discrete diffusion in image translation and generation. By corrupting data into a pure mask state via a random schedule, the conventional forward process induces a twofold misalignment: spatially, this pure-mask destination entirely discards the rich structural priors of the source image; temporally, the random masking order inherently contradicts the ``easy-first, hard-last'' decoding mechanism used during inference. To address this, DDB constructs a direct and efficient trajectory between domains. Spatially, we introduce a hybrid absorption mechanism that redefines the absorbing state to a stochastic mixture of mask and source tokens, effectively injecting source prior as spatial anchors into the latent space. Temporally, we design an information-guided noise schedule that quantifies semantic variation to prioritize the corruption of high-information regions at earlier timesteps. This ensures the model learns to resolve difficult semantic changes using robust context from invariant regions. Extensive experiments validate the versatility and robustness of our framework across diverse generative paradigms. DDB effectively balances edit alignment with structural fidelity across both text-guided semantic manipulation and pure structural image translation, while inherently complementing text-to-image generation and guaranteeing robust high-quality decoding under extremely low sampling steps. Code and models are available at https://github.com/HKU-HealthAI/DDB.

Abstract

Unpaired Image-to-Image Translation

Explore similar work

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation

Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation

Seed-to-Seed: Unpaired Image Translation in Diffusion Seed Space