Source-Agnostic Image Translation Based on Latent Aware Adaptive Masking

Tomislav Dobrički; Byung-Woo Hong

cs.CVAug 14, 2026

Source-Agnostic Image Translation Based on Latent Aware Adaptive Masking

Authors: Tomislav Dobrički, Byung-Woo Hong

Organizations: Chung-Ang University, Seoul, South Korea

Abstract

In this work, we propose a source-agnostic framework that dynamically refines a binary mask throughout the reverse diffusion process by computing the discrepancies of a pretrained diffusion model's prediction for each latent time step. Rather than relying on a fixed threshold, our method introduces a time-dependent statistical thresholding scheme derived from the empirical mean and standard deviation of prediction discrepancies across the latent noisy images from the target distribution. This allows the mask to adapt to the model's varying predictive confidence at different noise levels, effectively isolating domain-specific regions while preserving global structural coherence. Experimental results on the AFHQ and Celeba-HQ datasets demonstrate that our approach outperforms state-of-the-art unsupervised Image-to-Image methods in both realism (FID, KID) and faithfulness (SSIM, LPIPS). By requiring only a pretrained model of the target domain, our approach enables precise, automated localization and seamless translation across diverse source distributions without any specialized training. The project source code is available at: https://github.com/dtoma95/PM-Edit

Figures & tables

Figure 1: a) For a given source image (top), masks are computed at each time step (middle; only final mask is shown) and the translated image (bottom) is generated by inpainting. The generative network is only trained on the target domain in each example. Note that the displayed masks do not directly correspond to the image modifications because the masks are slightly dilated and blurred during inpainting. b) The mask at each time step (top) adapts to the latent noise level. When applied to the noised image (bottom) the mask conceals features foreign to the target domain.

Figure 2: Overview of our image editing method. a) For a given reference image $y$ an appropriate mask is computed at each time-step of the DDIM forward process. The current mask $M_{t}$ is computed by the union between the previous mask and the discrepancy between the model’s prediction for the noisy masked reference image $\tilde{y}_{t}$ and the generated sample $x_{t}$ . b) The output list of masks is used to perform inpainting to generate a more realistic final result. Any diffusion-based inpainting algorithm can be used. At each time step $t$ the appropriate mask $M_{t}$ is applied to guide generation.

Figure 3: Mean and standard deviations for L1 (left) and 1-SSIM (right) discrepancy metrics. We compare statistics for reference from the target dataset (AFHQ-Dog) and a source dataset that is unknown to the diffusion model (AFHQ-Cat). Note that source dataset statistics are not used in our method, and are displayed only for presentation purposes.

Method	FID $\downarrow$	e-0 KID $\times 10^{3}\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Wild $\rightarrow$ Dog
CUT † [ 35 ]	92.94	48.0	0.592	-
Santa † [ 54 ]	74.93	31.6	0.453	-
UNSB † [ 23 ]	86.72	40.2	0.524	-
SDDM † [ 43 ]	57.38	-	0.328	-
ILVR [ 5 ]	81.34 ± 1.44	36.6 ± 1.56	0.289 ± 0.001	0.530 ± 0.001

Table 1: Quantitative comparison of our results with baseline methods on AFHQ and Celeba-HQ I2I translation tasks. ILVR, SDEdit, EGSDE, CycleDiffusion all use the same backbone generator model, the best result among these methods is denoted in bold. Method denoted with

\dagger

require specialized training on the source domain. Our method outperformed baseline methods while remainging source agnostic.

	Structure	Fidelity		CLIP Similarity
Method	Distance ${}_{\times 10^{3}}\downarrow$	LPIPS ${}_{\times 10^{3}}\downarrow$	SSIM $\uparrow$	Whole $\uparrow$	Edited $\uparrow$
MasaCtrl. [ 3 ]	24.70	87.94	0.813	24.38	21.35
Wang et al. [ 48 ]	22.40	84.45	0.816	25.15	22.12
LatentEdit [ 30 ]	22.13	-	0.808	25.45	22.51
PnP [ 46 ]	28.22	113.46	0.791	25.41	22.55
PnPInversion [ 19 ]	24.29	106.06	0.797	25.41	22.62

Table 2: Results on the PIE-Bench (Prompt-driven Image Editing Benchmark). Our method outperform previous methods in both fidelity in prompt adherence.

Figure 4: Qualitative comparison of our method and other diffusion-based approaches.

Figure 5: Examples of text prompt guided Image translation on the PIE-Bench dataset. The left image and text for each sample represents the source image-prompt pair. With the result given by our method, according to the target prompt, shown on the right.

Figure 6: Demonstration of the final mask for different numbers of mask-computation batches $N$ . Computing discrepancies over of average of several images yields cleaner masks with less noise interference.

Table 9

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 115 papers

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation
Aug 30, 2026Xing Xie, Jiawei Liu, Shijun Zhou +4Masked Diffusion Models Image-to-Image Translation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China · University of Chinese Academy of Sciences, Beijing, China · School of Computing and Data Science, The University of Hong Kong, Hong Kong
GET: Generative Embedding Translation for Medical Image Segmentation
Aug 23, 2026Md Maklachur Rahman, Md Hasan Al Banna, Saraf Anjum +2Image Segmentation Medical Image Segmentation
Texas A&M University, College Station, TX, USA · Independent Researcher · Alfa Laval
Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation
May 31, 2026Ziyue Lin, Jiahe Hou, Hongyu Xia +6Diffusion Model Alignment Image-to-Image Translation
1The University of Hong Kong · 2Shenyang Institute of Automation, Chinese Academy of Sciences · 3The Chinese University of Hong Kong +1

Aug 30, 2026J/K move · Enter open · S save

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation

Xing Xie, Jiawei Liu, Shijun Zhou, Huijie Fan+3

State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China · University of Chinese Academy of Sciences, Beijing, China · School of Computing and Data Science, The University of Hong Kong, Hong Kong

Masked Diffusion Models Image-to-Image Translation Diffusion Timestep Scheduling Diffusion-Based Image Editing Diffusion Models

Open paper AskPDF arXiv

We propose Discrete Diffusion Bridges (DDB), a novel framework designed to resolve the fundamental spatiotemporal misalignment of standard discrete diffusion in image translation and generation. By corrupting data into a pure mask state via a random schedule, the conventional forward process induces a twofold misalignment: spatially, this pure-mask destination entirely discards the rich structural priors of the source image; temporally, the random masking order inherently contradicts the ``easy-first, hard-last'' decoding mechanism used during inference. To address this, DDB constructs a direct and efficient trajectory between domains. Spatially, we introduce a hybrid absorption mechanism that redefines the absorbing state to a stochastic mixture of mask and source tokens, effectively injecting source prior as spatial anchors into the latent space. Temporally, we design an information-guided noise schedule that quantifies semantic variation to prioritize the corruption of high-information regions at earlier timesteps. This ensures the model learns to resolve difficult semantic changes using robust context from invariant regions. Extensive experiments validate the versatility and robustness of our framework across diverse generative paradigms. DDB effectively balances edit alignment with structural fidelity across both text-guided semantic manipulation and pure structural image translation, while inherently complementing text-to-image generation and guaranteeing robust high-quality decoding under extremely low sampling steps. Code and models are available at https://github.com/HKU-HealthAI/DDB.

Abstract

Image-to-Image Translation

Explore similar work

Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation

GET: Generative Embedding Translation for Medical Image Segmentation

Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation