Cross-modal image translation in remote sensing must preserve source-observed content while matching the target-domain distribution. Existing methods jointly learn the target prior and cross-modal dependence from scarce paired data, overlooking a key asymmetry: only the latter intrinsically requires cross-modal correspondence. We formalize this distinction through conditional-score and denoising-risk analyses and propose Learning the Target Priors Before Image Translation (LTP-BIT), a prior-first paradigm that decouples the two learning tasks. LTP-BIT first learns a target-domain generative prior from large-scale unpaired imagery, then retains the pretrained backbone weights and learns source-conditioned control through P-DART, a parameter-efficient dual-stream architecture. Controlled experiments show that prior matching and scaling primarily improve target-domain realism, whereas instance fidelity relies more strongly on conditional adaptation. LTP-BIT achieves state-of-the-art performance across SAR-to-RGB and NIR-to-RGB benchmarks using only 9.81% task-specific parameters. On QXS-SAROPT, it retains near-full-data instance fidelity with only 25% of the paired samples.
Figures & tables
Figure 1: Overview of the prior-first learning paradigm in LTP-BIT for remote sensing cross-modal translation.
Figure 2: Target-domain prior initialization and scaling on QXS-SAROPT. Panels (a, b) compare random, ImageNet, and remote-sensing initialization under full-parameter fine-tuning; arrows report the epoch speedups achieved by remote-sensing initialization in reaching the respective final baseline levels. Under frozen-prior conditional adaptation, panel (c) varies the prior-backbone size with fixed DDT-head capacity, while panel (d) varies a strictly nested pretraining pool with the L prior and adaptation protocol fixed.
Prior Params CMMD ↓ PSNR ↑ S 10.0M 0.265 15.569 M 10.0M 0.254 15.422 L 10.0M 0.232 15.437 XL 10.0M 0.232 15.009
Prior Params CMMD ↓ PSNR ↑ S 10.6M 0.273 15.888 M 14.6M 0.258 15.965 L 22.0M 0.248 16.109 XL 45.4M 0.226 16.194
Table 1: Prior and data scaling. Panels (a) and (b) apply P-DART conditioning to the DDT head only and the full backbone, respectively; trainable parameters are therefore fixed in (a) and scale with the backbone in (b). Panel (c) scales nested pretraining data with the L prior and adaptation protocol fixed. Darker shading denotes better results.
Figure 3: Prior and Translation TFRR across nested pretraining-pool sizes. Panels (a) and (b) show Translation and Prior TFRR, respectively, versus normalized target-feature perturbation; panels (c)–(f) show their correspondence for 50K, 250K, 500K, and 1M pretraining samples. Color denotes perturbation magnitude; the dashed line marks equality.
Data scale
Manifold
Distribution-level
Instance-level
Cov.@3 (%) ↑
Dens.@3 ↑
FID ↓
CMMD ↓
PSNR ↑
LPIPS ↓
50K
7.675
0.062
29.70
0.253
16.093
0.469
250K
7.750
0.055
22.23
0.231
16.009
0.460
500K
7.925
0.052
20.34
0.226
15.912
0.459
1M
8.000
0.053
18.98
0.224
15.845
0.461
Table 2: Target-manifold metrics and paired translation performance across nested pretraining-pool sizes.
QXS
Manifold
Distribution-level
Instance-level
Cov.@3 (%) ↑
Dens.@3 ↑
FID ↓
CMMD ↓
PSNR ↑
LPIPS ↓
0
7.075
0.0481
28.18
0.239
16.187
0.467
4K
21.275
0.1317
27.75
0.241
16.138
0.464
8K
29.075
0.2028
26.41
0.234
16.220
0.460
16K
36.900
0.2938
26.75
0.236
16.268
0.457
Table 3: Target-manifold metrics and paired translation performance under fixed-pool target-domain enrichment. All priors are initialized from the same model trained for 300 epochs on a 50K-sample pool and are then trained for 100 additional epochs; Coverage and Density are computed from 5K generated samples.
Figure 4: Dual-Stream Conditional Adaptation Architecture of P-DART. (a) Propagation path of the source-modality reference stream through the DiTDH architecture, comprising a DiT backbone and a DDT head; (b) bidirectional attention interactions between the reference and generation streams in adjacent DiT blocks, where the reference-stream attention is fully trainable, whereas the generation-stream attention is adapted using LoRA; and (c) architecture of the DDT head and the locations where LoRA is applied.
Chesapeake
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
CMMD ↓
No pretrained generative prior
CycleGAN
15.621
0.536
0.405
83.94
1.514
CFCA-SET
18.569
0.572 0.572
0.351
85.82
2.388
StegoGAN
16.099
0.559
0.390
48.91
0.625
BBDM
17.801
0.437
0.355
53.29
0.828
Table 5: Comparison on Chesapeake. Method variants and notation follow Table 4 .
Figure 5: Qualitative comparisons on the QXS-SAROPT dataset.
Figure 6: Effect of paired-sample scale under matched trainable parameters and 5,040 training steps. Concentric rings report CMMD (left; lower is better) and PSNR (right; higher is better) for RS-XL and IN-XL. From outer to inner, ring pairs correspond to 1K, 2K, 4K, 8K, and 16K paired samples. Annotations report the relative CMMD reduction and absolute PSNR gain of RS-XL over IN-XL.
Method
Base
Param
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
CMMD ↓
Add.
FT
873M
16.349
0.380
0.459
55.03
0.358
Concat.
FT
877M
16.240
0.377
0.472
62.09
0.368
LoRA
Frozen
95M
15.936
0.361
0.455
20.55
0.222
DoRA
Frozen
95M
15.903
0.359
0.455
20.67
0.220
Ours-XL
Frozen
95M
16.194
0.362
0.445
20.68
0.226
Table 6: Full fine-tuning (FT) versus frozen-base adaptation. LoRA and DoRA use the Addition pathway.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Reconstruction errors across frequency components for RAE decoder variants. Following the frequency-domain analysis method in PRISM ( Fan et al. 2026 ) , we first average the RGB reconstruction residuals across the color channels, partition them into 8×8 non-overlapping patches, and compute the DCT patch by patch. The resulting 64 frequency coefficients are arranged in a zigzag order from the DC component to the high-frequency components. (a) Comparison among RAE-PT, RAE-RS, RAEv2-IN, RAEv2-Mix, and representative VAEs using 5K images randomly sampled from Git-10M. (b) Comparison of the four RAE decoders using 5K images from the ImageNet validation set.
Figure 2: Comparison of decoder reconstructions on remote sensing images. For two sets of Git-10M samples, the reconstruction results from left to right correspond to Ground Truth (GT), RAE-PT, RAE-RS, RAEv2-IN, RAEv2-Mix, SD1.5, SD3, SDXL, and Text2Earth VAE. The set-level FID, computed across 5K Git-10M evaluation images for each method, is annotated in the bottom-left corner of the first row (lower is better).
Figure 3: Comparison of RAE decoder reconstructions on ImageNet. From left to right, the results correspond to Ground Truth (GT), RAE-PT, RAE-RS, RAEv2-IN, and RAEv2-Mix. The set-level FID, computed across 5K ImageNet validation images for each method, is annotated in the bottom-left corner of the first row (lower is better).
Figure 4: Comprehensive evaluation of prior scaling and fixed-pool continued training controls. Blue and brownish-red represent the prior generation FID@5K and the translation FID@4K, respectively (lower is better). (a) Scaling up the prior model size while keeping the conditional capacity of the P-DART Decoder fixed; (b) Solid lines and solid markers indicate the continued training trajectory with progressively expanded data pools, whereas hollow markers denote the continued training controls initialized from the same parent exponential moving average (EMA) checkpoint while maintaining the original data pool. Dashed lines connect each hollow control point to the solid expansion point of the subsequent stage that shares an identical cumulative number of processed images. In the legend, “Equal compute” solely refers to the aforementioned matching of processed images, rather than implying strictly identical FLOPs or optimization trajectories; “Paired runs” indicates that a blue prior point and a brownish-red translation point with the same fill status originate from the same experimental group.
Setting
S
M
L
XL
Prior backbone
Parameters (M)
204.6
282.4
423.4
873.8
DiT backbone depth / width / heads
12 / 512 / 8
16 / 768 / 12
20 / 1024 / 16
28 / 1440 / 20
DDT head depth / width / heads
2 / 2048 / 16
2 / 2048 / 16
2 / 2048 / 16
2 / 2048 / 16
Auxiliary head attachment depth
4
5
6
8
MLP ratio
4.0
4.0
4.0
4.0
Appendix
Table 1: Prior model scales and their paired conditional adaptation configurations. The four prior tiers employ the same pretraining data and training schedule. The P-DART Decoder maintains a constant conditional capacity, whereas the encoder-side reference stream of the P-DART Codec scales synchronously with the backbone prior.
Cumulative images
LR schedule (epochs)
Prior images
Training strategy
Initialization
Epochs
seen
Warm-up
Decay start
Decay end
50K
From scratch
Random
300
15M
100
–
200
250K
Expanded pool
50K checkpoint
100
40M
5
30
60
50K
Fixed-pool control
50K checkpoint
500
40M
5
30
60
500K
Expanded pool
250K checkpoint
100
90M
5
30
60
250K
Fixed-pool control
250K checkpoint
200
90M
5
30
60
Appendix
Table 2: Configurations for the Prior Data Scaling experiments. The expanded data pool branch and the fixed data pool branch are initialized from the same parent checkpoint and are matched by the cumulative number of processed images.
Prior
Translation
Cumulative images
Init. from
Branch
Pool
FID ↓
CMMD ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
CMMD ↓
15M
Random
Initial
50K
22.35
0.856
16.093
0.373
0.469
29.70
0.253
40M
50K
Expanded
250K
14.81
0.794
16.009
0.361
0.460
22.23
0.231
40M
50K
Fixed
50K
20.25
0.833
16.057
0.369
0.466
27.21
0.241
90M
250K
Expanded
500K
13.89
0.773
15.912
0.356
0.459
20.34
0.226
90M
250K
Fixed
250K
14.37
0.782
15.946
0.359
0.460
20.86
0.222
Appendix
Table 3: Prior data scaling results for the expanded data trajectory and fixed pool continued training controls at matched cumulative training volumes. Each row reports the parent checkpoint, training route, current data pool, prior quality, and paired translation performance.
Resolution
Number of Images
0.5m/pixel
344,669
1m/pixel
206,801
2m/pixel
124,081
4m/pixel
74,449
8 – 128m/pixel
50,000 per resolution
Total
1,000,000
Appendix
Table 4: Spatial-resolution distribution of the constructed RS-1M dataset.
Manifold metrics
Translation metrics
Pool
Cov.@3 (%) ↑
Dens.@3 ↑
FID ↓
CMMD ↓
PSNR ↑
50K
8.025
0.071
27.21
0.241
16.057
250K
8.200
0.058
20.86
0.222
15.946
500K
8.350
0.059
19.15
0.220
15.805
Appendix
Table 5: Target manifold structure and paired translation performance of the fixed-pool continued training controls. The cumulative numbers of processed images for the 50K, 250K, and 500K groups are 40M, 90M, and 190M, respectively.
Manifold metrics
Translation metrics
QXS
Cov.@3 (%) ↑
Dens.@3 ↑
FID ↓
CMMD ↓
PSNR ↑
4K
21.275
0.132
24.33
0.241
14.748
8K
29.075
0.203
23.76
0.234
14.726
16K
36.900
0.294
23.81
0.228
14.677
Appendix
Table 6: Target-domain data enrichment controls with a frozen prior backbone. Coverage and Density are computed from the corresponding priors, while only the P-DART conditional side chain is trained during paired conditional adaptation.
Prior
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
CMMD ↓
Random
14.566
0.425
0.829
453.73
6.270
ImageNet
16.1782
0.3658
0.4514
23.6509
0.2466
RS-1M
16.2187
0.3613
0.4390
19.9728
0.2273
Appendix
Table 7: Effect of backbone generative prior initialization on paired translation performance. We compare random initialization with generative pretraining on ImageNet and RS-1M.
Figure 5: Qualitative comparison of ImageNet and RS-1M prior initialization on QXS.
Dataset
End Epoch
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
CMMD ↓
Chesapeake
80
17.5835
0.3463
0.2935
18.5616
0.8395
20
17.7171
0.3500
0.2904
18.1939
0.8247
SpaceNet6
80
18.9241
0.3511
0.2962
44.2808
0.8327
20
18.9237
0.3526
0.2926
43.7616
0.8084
QXS
80
16.0406
0.3518
0.4441
17.1970
0.2046
20
15.9555
0.3505
0.4457
16.5363
0.2009
Appendix
Table 8: Quantitative comparison of different curriculum endpoints on translation performance. The pure-noise replacement probability is kept at 100% for the first 10 epochs, followed by a linear decay to 25%.
Figure 6: Qualitative comparison of the fine-tuning strategies evaluated in the main text on the QXS dataset.
Figure 7: Qualitative comparisons on the QXS dataset.
Figure 8: Qualitative comparisons on the SpaceNet6 dataset.
Figure 9: Qualitative comparisons on the Chesapeake dataset.
Multi-modal remote sensing imagery provides complementary observations of the same geographic scene, yet such observations are frequently incomplete in practice. Existing cross-modal translation methods treat each modality pair as an independent task, resulting in quadratic complexity and limited generalization to unseen modality combinations. We formulate Any-to-Any translation as inference over a shared latent representation of the scene, where different modalities correspond to partial observations of the same underlying semantics. Based on this formulation, we propose Any2Any, a unified latent diffusion framework that projects heterogeneous inputs into a geometrically aligned latent space. Such structure performs anchored latent regression with a shared backbone, decoupling modality-specific representation learning from semantic mapping. Moreover, lightweight target-specific residual adapters are used to correct systematic latent mismatches without increasing inference complexity. To support learning under sparse but connected supervision, we introduce RST-1M, the first million-scale remote sensing dataset with paired observations across five sensing modalities, providing supervision anchors for any-to-any translation. Experiments across 14 translation tasks show that Any2Any consistently outperforms pairwise translation methods and exhibits strong zero-shot generalization to unseen modality pairs. Code and models are available at https://github.com/MiliLab/Any2Any.
Haoyang Chen, Jing Zhang, Hebaixu Wang +7
National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University, and Hubei Key Laboratory of Multimedia and Network Communication Engineering · Zhongguancun Academy, Beijing, China. 100094 · School of Electronic Information, Wuhan University, Wuhan, China +2
The ability to leverage images from co-available modalities to inform target-domain reconstruction is highly desirable in imaging algorithms. In this work, we introduce Generative Translation Priors (GTP)--a Bayesian framework that transforms diffusion-based image-to-image translation models into cross-modality image priors for ill-posed imaging inverse problems. GTP incorporates target-domain measurements through likelihood guidance, steering the translation process toward the desired posterior distribution. The framework is grounded in a theoretical analysis of the resulting posterior dynamics, which reveals an intrinsic bias introduced by likelihood guidance. We further characterize this bias and derive a ground-truth-free formulation for its estimation, enabling it to serve as a practical metric for assessing posterior sampling quality. Building on this analysis, we derive two discretized GTP algorithms based on gradient and proximal likelihood guidance, respectively. We validate GTP on computed tomography reconstruction with magnetic resonance side information, and on positron emission tomography reconstruction with computed tomography side information. Experiments demonstrate that GTP effectively incorporates complementary cross-modality information and achieves high-fidelity reconstruction even under severely undersampled measurements.
Evan Bell, Jiaming Liu, Yifan Chen +1
Johns Hopkins University · Stanford University · University of California, Los Angeles
In this work, we propose a source-agnostic framework that dynamically refines a binary mask throughout the reverse diffusion process by computing the discrepancies of a pretrained diffusion model's prediction for each latent time step. Rather than relying on a fixed threshold, our method introduces a time-dependent statistical thresholding scheme derived from the empirical mean and standard deviation of prediction discrepancies across the latent noisy images from the target distribution. This allows the mask to adapt to the model's varying predictive confidence at different noise levels, effectively isolating domain-specific regions while preserving global structural coherence. Experimental results on the AFHQ and Celeba-HQ datasets demonstrate that our approach outperforms state-of-the-art unsupervised Image-to-Image methods in both realism (FID, KID) and faithfulness (SSIM, LPIPS). By requiring only a pretrained model of the target domain, our approach enables precise, automated localization and seamless translation across diverse source distributions without any specialized training. The project source code is available at: https://github.com/dtoma95/PM-Edit