Day-to-night unpaired image translation is important to downstream tasks but remains challenging due to large appearance shifts and the lack of direct pixel-level supervision. Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized. These hallucinations significantly degrade downstream performance. We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation. To detect hallucination, we design a dual-head discriminator that additionally performs semantic segmentation to identify hallucinated content in background regions. To suppress these hallucinations, we introduce class-specific prototypes, constructed by aggregating features of annotated target-domain objects, which act as semantic anchors for each class. Built upon a Schrodinger Bridge-based translation model, our framework performs iterative refinement, where detected hallucination features are explicitly pushed away from class prototypes in feature space, thus preserving object semantics across the translation trajectory.Experiments show that our method outperforms existing approaches both qualitatively and quantitatively. On the BDD100K dataset, it improves mAP by 15.5% for day-to-night domain adaptation, with a notable 31.7% gain for classes such as traffic lights that are prone to hallucinations.
Figures & tables
Figure 1: Existing unpaired image-to-image (I2I) translation methods often introduce hallucinations that mimic or falsely suggest target classes, such as fake taillights for cars or spurious green and red signals for traffic lights (see yellow boxes).
Figure 2: Model overview. Given a source image x0 , our framework models image-to-image translation as a multi-step transport process that progressively refines the translation. At each step j∈[0,i] , the generator processes the current state xtj and predicts an intermediate target image x1(xtj) . The next state xtj+1 is obtained by mixing xtj , x1(xtj) , and Gaussian noise. This process continues until xti , the final intermediate state before training. Hallucinations in the x1(xti) are detected by the segmentation head Dseg , while reference images provide annotated objects to generate target-class prototypes of the target domain. Intermediate hallucination suppression then pushes detected hallucinated features away from the prototypes.
Figure 3: Discriminator training. Left panel: The segmentation head Dseg is trained using SAM2 ( Ravi et al. 2024 ) -generated pseudo labels, with the segmentation loss Lseg . Right panel: In the generator training stage, translated images are evaluated by the discriminator. Hallucinations are segmented by comparing the segmentation prediction with the annotations. The segmented hallucinations will be utilized by the intermediate hallucination suppression. The style head is not shown in this figure.
Figure 4: Qualitative comparison of day-to-night translation results on the BDD100K dataset. Yellow boxes highlight regions containing target-class hallucinations or inconsistencies with the original annotations, which may introduce label noise. Our method (e) preserves both the realistic nighttime style and semantic consistency with the input annotations, whereas existing state-of-the-art methods (b–d) often produce hallucinated objects or fail to align with the source semantics.
Methods
mAP
Person
Car
Rider
Bus
Truck
Bike
Motor
T. Light
T. Sign
Lower Bound
13.75
12.99
25.21
8.71
18.93
15.73
8.45
6.88
8.28
18.55
Upper Bound
17.86
14.43
32.59
10.25
26.50
23.83
8.96
8.43
11.93
23.83
DRIT
12.68
11.84
25.16
8.24
14.89
15.46
9.05
3.84
6.49
19.18
CycleGAN
13.16
12.52
26.08
7.46
17.36
16.01
9.46
5.70
4.60
19.29
CUT
14.10
14.13
28.31
8.27
20.22
16.21
9.96
5.29
5.36
19.19
Cycle-Diffusion
13.62
13.54
26.49
8.31
20.32
17.02
10.11
6.22
4.91
17.88
Table 1: Comparison on the BDD100K Day-to-Night Translation Task. The Lower Bound represents the detector trained in the daytime, while the Upper Bound represents the detector trained in the nighttime. All methods are trained on daytime images and translated nighttime images and tested at nighttime. Reported AP is the averaged AP over IoUs 0.5 to 0.95.
Methods
mAP
Person
Car
Rider
Bus
Truck
Bike
Motor
T. Light
T. Sign
Lower Bound
13.75
12.99
25.21
8.71
18.93
15.73
8.45
6.88
8.28
18.55
w/o Lhl & Lsupp
14.11
14.08
27.14
8.31
19.66
18.42
9.66
4.48
5.48
19.82
w/o Lsupp
15.55
14.80
27.69
9.00
22.15
20.10
11.84
8.42
7.01
18.94
w/o Lhl
16.43
15.26
29.65
10.22
25.49
20.42
11.12
7.97
7.45
20.34
All Components
17.40
15.35
30.01
10.92
26.02
23.48
11.74
8.44
8.55
22.01
Table 2: Ablation study on the mAP for BDD100K day-to-night domain adaptation. Lhl refers to the hallucination segmentation loss, and Lsupp denotes the hallucination feature suppression loss.
Adversarial diffusion and diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations. Adversarial approaches require target-domain adversarial loss during training, which can limit generalization to unseen data, while diffusion-inversion methods often produce low-fidelity translations due to imperfect inversion into noise-latent representations. In this work, we propose the Self-Supervised Semantic Bridge (SSB), a versatile framework that integrates external semantic priors into diffusion bridge models to enable spatially faithful translation without cross-domain supervision. Our key idea is to leverage self-supervised visual encoders to learn representations that are invariant to appearance changes but capture geometric structure, forming a shared latent space that conditions the diffusion bridges. Extensive experiments show that SSB outperforms strong prior methods for challenging medical image synthesis in both in-domain and out-of-domain settings, and extends easily to high-quality text-guided editing.
Jiaming Liu, Felix Petersen, Yunhe Gao +6
Stanford University · LLNL · Johns Hopkins University
Unpaired image-to-image translation has seen significant progress since the introduction of CycleGAN. However, methods based on diffusion models or Schrödinger bridges have yet to be widely adopted in real-world applications due to their iterative sampling nature. To address this challenge, we propose Implicit Bridge Consistency Distillation (IBCD), which enables single-step bidirectional unpaired translation without using adversarial loss. IBCD reformulates consistency distillation by using a diffusion implicit bridge model that connects PF-ODE trajectories between distributions, parametrized on a signed time axis whose sign selects the target domain, so that one network translates in both directions in a single step. Additionally, we introduce two key improvements: 1) distribution matching for consistency distillation and 2) adaptive weighting method based on distillation difficulty. Experimental results demonstrate that IBCD achieves state-of-the-art performance on benchmark datasets in a single generation step, and that it extends to brain MRI contrast translation.
Suhyeon Lee, Kwanyoung Kim, Jong Chul Ye
Kim Jae Chul Graduate School of AI, KAIST · Samsung Research
In this work, we propose a source-agnostic framework that dynamically refines a binary mask throughout the reverse diffusion process by computing the discrepancies of a pretrained diffusion model's prediction for each latent time step. Rather than relying on a fixed threshold, our method introduces a time-dependent statistical thresholding scheme derived from the empirical mean and standard deviation of prediction discrepancies across the latent noisy images from the target distribution. This allows the mask to adapt to the model's varying predictive confidence at different noise levels, effectively isolating domain-specific regions while preserving global structural coherence. Experimental results on the AFHQ and Celeba-HQ datasets demonstrate that our approach outperforms state-of-the-art unsupervised Image-to-Image methods in both realism (FID, KID) and faithfulness (SSIM, LPIPS). By requiring only a pretrained model of the target domain, our approach enables precise, automated localization and seamless translation across diverse source distributions without any specialized training. The project source code is available at: https://github.com/dtoma95/PM-Edit