Simulation-to-reality translation must bridge the appearance gap between synthetic and real domains while preserving structural and semantic consistency. Conditioning-based methods achieve spatial alignment but introduce computationally expensive control modules. Paired-data methods achieve realism but rely on complex synthesis pipelines, often altering scene geometry and semantics. Training-free editing methods avoid both constraints but lack a learned appearance prior, limiting their perceptual quality. Recently proposed phase-preserving diffusion presents a promising alternative, but Fourier-domain formulations are constrained by global spectral coupling. This coupling induces spatial artifacts such as ringing and boundary leakage, thereby degrading structural and semantic consistency. We introduce Wavelet Phase Diffusion, which addresses this through two components. First, we operate in the Dual-Tree Complex Wavelet Packet Transform domain, whose localized wavelet packets enable spatially adaptive phase injection without global spectral interference. Second, Low-Frequency Randomization (LFR) replaces the low-frequency packet, decoupling the model from the synthetic illumination prior and enabling in-distribution real-world appearance. Both components train on unpaired open-domain data, and introduce negligible inference overhead. The spatial locality further enables instance-level translation, where individual objects or regions are translated to photorealistic appearance independently while the surrounding scene remains untranslated. On vKITTI → KITTI image translation, ψ-PD outperforms prior methods in realism and semantic consistency while maintaining competitive structural alignment. For CARLA video translation, ψ-PD approaches the realism of paired-data methods while reducing VLM planner ADE and FDE by 5.4% and 5.1%, respectively.
Figures & tables
Figure 2: Fourier- vs. Wavelet-domain low-pass filtering. Fourier exhibits non-local ringing near edges, while DT- C WPT preserves local geometry and edge structure more faithfully.
Figure 3: Overview of ψ-PD . Source and noise latents are decomposed by DT- C WPT into a low-frequency packet and multiple high-frequency packets. Source phase (green) is injected per packet under the cutoff map F while the low-frequency packet is randomized (LFR), and the inverse transform yields the structured noise ϵ^ .
Method
KID ↓
FID ↓
CLIP- IQA ↑
mIoU ↑
Dep- SSIM ↑
AbsRel ↓
Input
6.06
97.29
0.281
50.39
0.900
0.157
FlowEdit [ 23 ]
4.85
82.41
0.401
42.72
0.812
0.260
DNAEdit [ 44 ]
4.78
85.47
0.322
41.22
0.827
0.254
NeuralRemaster [ 46 ]
4.87
78.95
0.527
38.32
0.811
0.344
Cosmos Transfer 2.5 [ 27 ]
4.52
73.52
0.469
39.36
0.870
0.202
ψ-PD ( Ours )
4.41
73.84
0.561
43.50
0.839
0.229
Table 1: Quantitative evaluations for image translation (vKITTI → KITTI). Bold indicates best, underlined second best. VACE and Ditto are video-native and thus not evaluated here. KID is reported ×102 .
Method
sKID
sFID
CLIP-
MS
ADE (m) ↓
FDE
↓
↓
IQA ↑
(%) ↑
1s
2s
3s
Avg.
(m) ↓
Input
1.97
48.89
0.367
98.58
0.520
2.008
4.484
2.337
5.223
DNAEdit [ 44 ]
2.24
50.22
0.315
98.20
0.581 (+11.8%)
2.260 (+12.5%)
5.039 (+12.4%)
2.627 (+12.4%)
5.872 (+12.4%)
VACE [ 21 ]
2.81
59.91
0.405
98.47
0.503 (-3.3%)
1.952 (-2.8%)
4.385 (-2.2%)
2.280 (-2.5%)
5.116 (-2.1%)
Cosmos Transfer 2.5 [ 27 ]
2.42
51.98
0.429
97.84
0.516 (-0.9%)
1.999 (-0.5%)
4.444 (-0.9%)
2.319 (-0.8%)
5.173 (-1.0%)
NeuralRemaster [ 46 ]
1.45
37.65
0.450
98.30
0.532 (+2.2%)
2.079 (+3.5%)
4.630 (+3.2%)
2.413 (+3.3%)
5.389 (+3.2%)
Table 2: Quantitative evaluation on video translation (CARLA). Relative changes w.r.t. the input are reported in %. Bold indicates best, underlined second best. FlowEdit is an image-only method and is not applicable to video. sKID is reported ×102 .
Figure 4: Qualitative comparison on vKITTI → KITTI. Competing methods either hallucinate structure or distort lane/vehicle semantics; ψ-PD preserves both while achieving realistic appearance. See Sec. 5.4 for per-method analysis.
Figure 5: Qualitative comparison on CARLA video translation. Baselines either retain CARLA’s synthetic look or trade realism for altered semantics (weather, traffic lights, lanes); ψ-PD matches Ditto’s realism while keeping semantics intact. See Sec. 5.4 .
Figure 6: VLM planning on translated CARLA frames. Predicted waypoints ( red ) vs. ground-truth ( green ).
Figure 7: Zero-shot instance-level video translation. The task requires only the character’s mirror reflection (highlighted in green in the first input frame) to appear photorealistic while the surrounding comic-style scene remains untranslated. Ditto translates the entire frame indiscriminately and cannot isolate the edit to the reflection region; ψ-PD confines the translation to the target region via a per-pixel map.
Figure 8: Ablation: Pareto frontiers (realism vs. consistency).
Positive (real)
Negative (synthetic)
A photo of real cars on a road.
Computer graphics of cars on a road.
Realistic materials.
Plastic-looking materials.
A sharp real photo.
A blurry computer render.
A photograph of a real place.
A CGI scene.
A photo with natural colors.
An image with artificial colors.
Table S1 : CLIP-IQA prompt pairs. Each row is one antonym pair; scores are averaged over all five pairs.
Figure S1 : Qualitative results on downstream utility of translated videos.
Controls
KID ↓
FID ↓
CLIP- IQA ↑
mIoU ↑
Dep- SSIM ↑
AbsRel ↓
Input
6.06
97.29
0.281
50.39
0.900
0.157
D
5.41
77.18
0.466
38.37
0.852
0.216
D+E
4.52
73.52
0.469
39.36
0.870
0.202
D+V
6.91
97.49
0.370
40.99
0.872
0.163
D+S
6.94
92.34
0.427
36.11
0.870
0.202
S+E
5.52
79.78
0.497
37.32
0.844
0.275
Table S2 : Cosmos Transfer 2.5 sensitivity to control-signal choice (vKITTI → KITTI). D=depth, E=edge, S=seg, V=vis. Bold indicates best, underlined second best (excluding raw input). KID is reported ×102 .
r
J
KID ↓
CLIP- IQA ↑
Dep- SSIM ↑
mIoU ↑
12
3
4.19
0.593
0.818
37.00
4
4.41
0.561
0.839
43.50
5
5.23
0.536
0.847
44.81
16
3
4.50
0.562
0.838
38.73
4
4.74
0.531
0.851
44.31
5
5.74
0.507
0.855
45.84
Table S3 : Effect of decomposition depth J at r=12 and r=16 on vKITTI → KITTI. J=4 (default) best balances realism (KID) and structural fidelity (DepSSIM, mIoU). KID is reported ×102 .
Figure S2 : vKITTI → KITTI qualitative comparison, Scene 1.
Figure S3 : vKITTI → KITTI qualitative comparison, Scene 2.
Figure S4 : vKITTI → KITTI qualitative comparison, Scene 3.
Figure S5 : vKITTI → KITTI qualitative comparison, Scene 4.
Figure S6 : vKITTI → KITTI qualitative comparison, Scene 5.
Figure S7 : Zero-shot instance-level video translation. Five consecutive frames from three scenes. In all cases, the target instance is highlighted in green in the first input frame.
Bridging the simulation-to-reality (sim2real) gap remains challenging as labelled real-world data is scarce. Existing diffusion-based approaches rely on unstructured prompts or statistical alignment, which do not capture the structured factors that make images look real. We introduce Ontology- Guided Diffusion (OGD), a neuro-symbolic zero-shot sim2real image translation framework that represents realism as structured knowledge. OGD decomposes realism into an ontology of interpretable traits -- such as lighting and material properties -- and encodes their relationships in a knowledge graph. From a synthetic image, OGD infers trait activations and uses a graph neural network to produce a global embedding. In parallel, a symbolic planner uses the ontology traits to compute a consistent sequence of visual edits needed to narrow the realism gap. The graph embedding conditions a pretrained instruction-guided diffusion model via cross-attention, while the planned edits are converted into a structured instruction prompt. Across benchmarks, our graph-based embeddings better distinguish real from synthetic imagery than baselines, and OGD outperforms state-of-the-art diffusion methods in sim2real image translations. Overall, OGD shows that explicitly encoding realism structure enables interpretable, data-efficient, and generalisable zero-shot sim2real transfer.
Mohamed Youssef, Mayar Elfares, Anna-Maria Meer +2
University of Stuttgart, Germany · Fraunhofer IPA, Germany
Image-to-image (I2I) translation has achieved strong results in tasks like human relighting and driving scene translation using latent diffusion models (LDMs). However, compact LDMs often struggle to preserve fine-grained structures because the encoder compresses high-resolution inputs into a spatially downsampled latent space. To address this issue, we propose a simple saliency-guided warp-unwarp framework that reallocates spatial representation toward salient regions before encoding, enabling better preservation of structural details without increasing latent resolution. The warped image is processed by the original diffusion model and then mapped back via an inverse warp. In addition, we propose a simple and efficient outpainting-based synthetic data generation pipeline to produce high-quality paired data for image relighting. Our method is model-agnostic, requires no architectural modification, and introduces negligible computational overhead. Experiments on human relighting, driving scene relighting, and translation demonstrate improved structural preservation, lighting faithfulness, and image quality, with our framework extending naturally to video via frame-by-frame application with good temporal stability. Project Webpage: https://shenzheng2000.github.io/WarpI2I.github.io
Adversarial diffusion and diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations. Adversarial approaches require target-domain adversarial loss during training, which can limit generalization to unseen data, while diffusion-inversion methods often produce low-fidelity translations due to imperfect inversion into noise-latent representations. In this work, we propose the Self-Supervised Semantic Bridge (SSB), a versatile framework that integrates external semantic priors into diffusion bridge models to enable spatially faithful translation without cross-domain supervision. Our key idea is to leverage self-supervised visual encoders to learn representations that are invariant to appearance changes but capture geometric structure, forming a shared latent space that conditions the diffusion bridges. Extensive experiments show that SSB outperforms strong prior methods for challenging medical image synthesis in both in-domain and out-of-domain settings, and extends easily to high-quality text-guided editing.
Jiaming Liu, Felix Petersen, Yunhe Gao +6
Stanford University · LLNL · Johns Hopkins University