Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation

Shuwei Li; Lei Tan; Robby T. Tan

cs.CVFeb 17, 2026

Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation

Authors: Shuwei Li, Lei Tan, Robby T. Tan

Organizations: National University of Singapore · ASUS Intelligent Cloud Services (AICS)

Abstract

Day-to-night unpaired image translation is important to downstream tasks but remains challenging due to large appearance shifts and the lack of direct pixel-level supervision. Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized. These hallucinations significantly degrade downstream performance. We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation. To detect hallucination, we design a dual-head discriminator that additionally performs semantic segmentation to identify hallucinated content in background regions. To suppress these hallucinations, we introduce class-specific prototypes, constructed by aggregating features of annotated target-domain objects, which act as semantic anchors for each class. Built upon a Schrodinger Bridge-based translation model, our framework performs iterative refinement, where detected hallucination features are explicitly pushed away from class prototypes in feature space, thus preserving object semantics across the translation trajectory.Experiments show that our method outperforms existing approaches both qualitatively and quantitatively. On the BDD100K dataset, it improves mAP by 15.5% for day-to-night domain adaptation, with a notable 31.7% gain for classes such as traffic lights that are prone to hallucinations.

Figures & tables

Figure 1: Existing unpaired image-to-image (I2I) translation methods often introduce hallucinations that mimic or falsely suggest target classes, such as fake taillights for cars or spurious green and red signals for traffic lights (see yellow boxes).

Figure 2: Model overview. Given a source image $x_{0}$ , our framework models image-to-image translation as a multi-step transport process that progressively refines the translation. At each step $j\in[0,i]$ , the generator processes the current state $x_{t_{j}}$ and predicts an intermediate target image $x_{1}(x_{t_{j}})$ . The next state $x_{t_{j+1}}$ is obtained by mixing $x_{t_{j}}$ , $x_{1}(x_{t_{j}})$ , and Gaussian noise. This process continues until $x_{t_{i}}$ , the final intermediate state before training. Hallucinations in the $x_{1}(x_{t_{i}})$ are detected by the segmentation head $D_{\text{seg}}$ , while reference images provide annotated objects to generate target-class prototypes of the target domain. Intermediate hallucination suppression then pushes detected hallucinated features away from the prototypes.

Figure 3: Discriminator training. Left panel: The segmentation head $D_{\text{seg}}$ is trained using SAM2 ( Ravi et al. 2024 ) -generated pseudo labels, with the segmentation loss $L_{\text{seg}}$ . Right panel: In the generator training stage, translated images are evaluated by the discriminator. Hallucinations are segmented by comparing the segmentation prediction with the annotations. The segmented hallucinations will be utilized by the intermediate hallucination suppression. The style head is not shown in this figure.

Figure 4: Qualitative comparison of day-to-night translation results on the BDD100K dataset. Yellow boxes highlight regions containing target-class hallucinations or inconsistencies with the original annotations, which may introduce label noise. Our method (e) preserves both the realistic nighttime style and semantic consistency with the input annotations, whereas existing state-of-the-art methods (b–d) often produce hallucinated objects or fail to align with the source semantics.

Methods	mAP	Person	Car	Rider	Bus	Truck	Bike	Motor	T. Light	T. Sign
Lower Bound	13.75	12.99	25.21	8.71	18.93	15.73	8.45	6.88	8.28	18.55
Upper Bound	17.86	14.43	32.59	10.25	26.50	23.83	8.96	8.43	11.93	23.83
DRIT	12.68	11.84	25.16	8.24	14.89	15.46	9.05	3.84	6.49	19.18
CycleGAN	13.16	12.52	26.08	7.46	17.36	16.01	9.46	5.70	4.60	19.29
CUT	14.10	14.13	28.31	8.27	20.22	16.21	9.96	5.29	5.36	19.19
Cycle-Diffusion	13.62	13.54	26.49	8.31	20.32	17.02	10.11	6.22	4.91	17.88

Table 1: Comparison on the BDD100K Day-to-Night Translation Task. The Lower Bound represents the detector trained in the daytime, while the Upper Bound represents the detector trained in the nighttime. All methods are trained on daytime images and translated nighttime images and tested at nighttime. Reported AP is the averaged AP over IoUs 0.5 to 0.95.

Methods	mAP	Person	Car	Rider	Bus	Truck	Bike	Motor	T. Light	T. Sign
Lower Bound	13.75	12.99	25.21	8.71	18.93	15.73	8.45	6.88	8.28	18.55
w/o $\mathcal{L}_{\text{hl}}$ & $\mathcal{L}_{\text{supp}}$	14.11	14.08	27.14	8.31	19.66	18.42	9.66	4.48	5.48	19.82
w/o $\mathcal{L}_{\text{supp}}$	15.55	14.80	27.69	9.00	22.15	20.10	11.84	8.42	7.01	18.94
w/o $\mathcal{L}_{\text{hl}}$	16.43	15.26	29.65	10.22	25.49	20.42	11.12	7.97	7.45	20.34
All Components	17.40	15.35	30.01	10.92	26.02	23.48	11.74	8.44	8.55	22.01

Table 2: Ablation study on the mAP for BDD100K day-to-night domain adaptation.

\mathcal{L}_{\text{hl}}

refers to the hallucination segmentation loss, and

\mathcal{L}_{\text{supp}}

denotes the hallucination feature suppression loss.

Methods	mAP	Person	Car	Truck	Bike
DT	$31.2$	$28.5$	$40.7$	$25.9$	$29.7$
DAF	$38.5$	$39.2$	$40.2$	$25.7$	$48.9$
DARL	$45.3$	$46.4$	$58.7$	$27.0$	$49.1$
DAOD	$46.1$	$47.3$	$59.1$	$28.3$	$49.6$
DUNIT	$54.1$	$60.7$	$65.1$	$32.7$	$57.7$
MGUIT	$54.6$	$58.3$	$68.2$	$33.4$	$58.4$

Table 3: KITTI → Cityscapes Domain Adaptation. Per-class comparison is shown.

Figure 5: Ablation study . The fewest hallucinations are observed when both components are incorporated.

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 115 papers

Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge
Feb 18, 2026Jiaming Liu, Felix Petersen, Yunhe Gao +6Unpaired Image-to-Image Translation Diffusion Models
Stanford University · LLNL · Johns Hopkins University
Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation
Mar 19, 2025Suhyeon Lee, Kwanyoung Kim, Jong Chul YeImage-to-Image Translation Unpaired Image-to-Image Translation
Kim Jae Chul Graduate School of AI, KAIST · Samsung Research
Source-Agnostic Image Translation Based on Latent Aware Adaptive Masking
Aug 14, 2026Tomislav Dobrički, Byung-Woo HongImage-to-Image Translation Unpaired Image-to-Image Translation
Chung-Ang University, Seoul, South Korea

Feb 18, 2026J/K move · Enter open · S save

Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge

Jiaming Liu, Felix Petersen, Yunhe Gao, Yabin Zhang+5

Stanford University · LLNL · Johns Hopkins University

Unpaired Image-to-Image Translation Diffusion Models Medical Image Synthesis

Open paper AskPDF arXiv

Adversarial diffusion and diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations. Adversarial approaches require target-domain adversarial loss during training, which can limit generalization to unseen data, while diffusion-inversion methods often produce low-fidelity translations due to imperfect inversion into noise-latent representations. In this work, we propose the Self-Supervised Semantic Bridge (SSB), a versatile framework that integrates external semantic priors into diffusion bridge models to enable spatially faithful translation without cross-domain supervision. Our key idea is to leverage self-supervised visual encoders to learn representations that are invariant to appearance changes but capture geometric structure, forming a shared latent space that conditions the diffusion bridges. Extensive experiments show that SSB outperforms strong prior methods for challenging medical image synthesis in both in-domain and out-of-domain settings, and extends easily to high-quality text-guided editing.

Abstract

Image-to-Image Translation

Explore similar work

Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge

Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation

Source-Agnostic Image Translation Based on Latent Aware Adaptive Masking