GeoSET: Generalist Foundation Model for SAR-to-EO Image Translation

Jeonghyeok Do; Munchurl Kim

cs.CVSep 26, 2026

GeoSET: Generalist Foundation Model for SAR-to-EO Image Translation

Authors: Jeonghyeok Do, Munchurl Kim

Organizations: Korea Advanced Institute of Science and Technology (KAIST)

Abstract

Paired synthetic aperture radar (SAR) and electro-optical (EO) imagery is increasingly available across sensors, resolutions, and geographic regions. Yet existing SAR-to-EO image translation (SET) methods are typically trained on a single, limited-scale dataset, producing models specialized to particular sensing conditions. We introduce GeoSET, the first generalist model for SET, built around a single pretrained parent that is adapted to downstream datasets under a common protocol. We curate over 3 million high-quality SAR--EO pairs from a collection of more than 10 million SAR observations, spanning diverse sensors, spatial resolutions, and ground sampling distances. To bridge the modality gap between SAR observations and a pretrained image generator, we develop a speckle-robust SAR encoder and pretrain the conditional generator on this heterogeneous corpus. The resulting parent supports efficient adaptation across downstream datasets through low-rank adaptation (LoRA), updating only 0.60% of the generator parameters and requiring approximately one hour per dataset. Across six downstream benchmarks, GeoSET achieves state-of-the-art results in FID and DISTS with full fine-tuning or LoRA, demonstrating effective transfer across heterogeneous SAR-EO domains.

Figures & tables

Figure 2: Cross-dataset comparison of SAR-to-EO image translation methods. We report FID and DISTS on six benchmarks, independently normalized for each dataset–metric pair as

100\times\text{best}/\text{value}

Figure 3: GeoSET framework. Stage 1 initializes a SAR encoder from a pretrained autoencoder and trains it to reconstruct the original SAR observation from a domain-aware speckle perturbation while the shared decoder remains frozen. Stage 2 replaces the pretrained generator’s text condition with a spatial SAR stream initialized from its image stream and learns a conditional flow from Gaussian noise to the EO latent. Stage 3 adapts the resulting generalist to each downstream benchmark with LoRA or full fine-tuning.

Method	Venue	QXS-SAROPT							SAR2Opt
Method	Venue	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
General image-to-image translation methods
pix2pix	CVPR’17	174.6	0.373	0.1713	0.275	0.665	0.203	12.33	261.9	0.347	0.2164	0.277	0.657	0.199	13.39
CycleGAN	ICCV’17	115.5	0.362	0.0802	0.283	0.647	0.278	13.25	139.1	0.323	0.0343	0.374	0.642	0.188	12.68
pix2pixHD	CVPR’18	85.7	0.298	0.0492	0.403	0.573	0.358	16.13	146.3	0.283	0.0654	0.475	0.567	0.268	15.95
SPADE	CVPR’19	90.7	0.292	0.0607	0.366	0.599	0.320	14.53	142.5	0.265	0.0518	0.447	0.597	0.234	14.47

Table 2: Quantitative comparison on QXS-SAROPT and SAR2Opt. We retrain (evaluate) all competing methods on the same training (test) splits. Bold and underlined values indicate the best and second-best results, respectively.

Method	SAR2EO							SpaceNet6
Method	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
General image-to-image translation methods
pix2pix	169.9	0.315	0.1551	0.400	0.573	0.455	18.06	166.7	0.242	0.1091	0.589	0.419	0.459	17.07
CycleGAN	360.7	0.522	0.3903	0.369	0.683	0.300	15.17	119.3	0.219	0.0436	0.660	0.373	0.478	16.88
pix2pixHD	106.0	0.254	0.0638	0.543	0.439	0.619	21.68	175.1	0.246	0.1136	0.682	0.361	0.525	19.21
SPADE	98.5	0.260	0.0646	0.504	0.501	0.562	19.81	179.7	0.242	0.1228	0.662	0.385	0.489	17.77

Table 3: Quantitative comparison on SAR2EO and SpaceNet6. We retrain (evaluate) all competing methods on the same training (test) splits. Bold is the best, and underlined is the 2nd best.

Figure 5: Qualitative comparison on SAR-to-EO image translation (SET) benchmarks. All methods receive the same SAR observation, and the paired EO image is shown as a reference.

Source	Original SAR			Perturbed SAR			EO
Source	Before	After	$\Delta$	Before	After	$\Delta$	EO
GUSO	27.47	29.18	+1.71	23.54	26.96	+3.42	31.02
TerraMesh	20.11	32.04	+11.93	19.12	30.48	+11.37	33.95
SARLO-80	17.39	19.21	+1.82	16.71	18.49	+1.78	27.81
SAR-1M	29.95	31.25	+1.30	22.01	26.99	+4.98	33.59
3MOS	35.15	36.07	+0.92	27.30	31.92	+4.61	34.62

Table 4: Stage 1 encoder diagnostics and Stage 3 adaptation efficiency. (a) Reconstruction PSNR on original and perturbed SAR inputs before and after encoder training. EO reports reconstruction with the frozen pretrained codec. (b) Computational costs of full fine-tuning and LoRA.

Configuration	SAR2EO			SpaceNet6
Configuration	FID $\downarrow$	DISTS $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	DISTS $\downarrow$	LPIPS $\downarrow$
SD2.1 Full FT	67.7	0.270	0.487	124.7	0.196	0.348
FLUX.2 LoRA (Backbone)	71.2	0.275	0.481	117.4	0.201	0.361
+ Pretraining corpus	50.5	0.250	0.470	107.9	0.194	0.351
+ SAR encoder	39.8	0.220	0.448	94.2	0.193	0.347
+ Speckle aug. ( GeoSET )	29.5	0.202	0.440	87.9	0.189	0.338

Table 5: Cumulative component comparison on SAR2EO and SpaceNet6.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Section	Contents
Section A	Additional results and discussion
Section B	Datasets and evaluation protocols
Section C	Implementation details
Section D	Evaluation metrics

Appendix

Table 6: Overview of the Appendix .

	PSNR $\uparrow$			LPIPS $\downarrow$
Source	True	Shuffled	Null	True	Shuffled	Null
GUSO	13.36	10.24	10.58	0.548	0.678	0.690
SARLO-80	15.88	11.96	11.65	0.514	0.678	0.693
TerraMesh	13.69	10.94	10.80	0.491	0.670	0.695

Appendix

Table 7: SAR conditioning intervention on pretraining sources. True denotes the corresponding SAR observation, shuffled uses a mismatched SAR observation, and null uses an all-zero SAR latent. The null condition is included during training through conditioning dropout.

Method	CAP-BSG							KOMPSAT
Method	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	FID $\downarrow$	DISTS $\downarrow$	KID $\downarrow$	DINO $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
General image-to-image translation methods
pix2pix	245.8	0.402	0.2086	0.254	0.626	0.251	13.77	255.6	0.346	0.2892	0.298	0.669	0.116	10.33
CycleGAN	100.0	0.327	0.0485	0.365	0.569	0.353	16.84	160.8	0.359	0.1105	0.299	0.653	0.139	10.14
pix2pixHD	212.7	0.335	0.1673	0.301	0.570	0.367	17.23	223.2	0.379	0.2004	0.305	0.649	0.128	11.24
SPADE	187.5	0.354	0.1407	0.287	0.593	0.348	16.20	226.3	0.377	0.2106	0.303	0.670	0.157	11.47

Appendix

Table 8: Quantitative comparison on CAP-BSG and KOMPSAT. We retrain (evaluate) all competing methods on the same training (test) splits. Bold is the best, and underlined is the 2nd best.

Source	SAR source	EO source	Spatial scale	Tile size	SAR input
GUSO ( Yan et al., 2026 )	ICEYE, Capella, Umbra	Supplied optical imagery	0.16–0.98 $\mathrm{m}$	$512^{2}$	1-ch display
TerraMesh ( Blumenstiel et al., 2025 )	Sentinel-1 RTC, VV/VH	Sentinel-2 L2A RGB	10 m grid	$264^{2}$	2-ch dB
SARLO-80 ( Debuysère et al., 2026 )	Umbra spotlight SLC	Google satellite imagery	0.8 $\mathrm{m}$ slant-range grid	$1024^{2}$	1-ch amplitude
SAR-1M ( Liu et al., 2026 )	Multi-source paired subset	Paired optical counterparts	Source-dependent	$256^{2}$	1-ch display
3MOS mr ( Ye et al., 2025 )	Sentinel-1, ALOS-family	Google Earth	10 / 12.5 $\mathrm{m}$	$256^{2}$	1-ch display
3MOS hr ( Ye et al., 2025 )	GF-3, RADARSAT-2, RCM	Google Earth	3.5 / 6.3 / 12.5 $\mathrm{m}$	$256^{2}$	1-ch display

Appendix

Table 9: Pretraining corpus and representations. Five dataset families form six streams, with 3MOS divided into mr (mid-resolution) and hr (high-resolution). Spatial scales denote reported resolutions or sampling intervals. Tile sizes are in pixels; display denotes rendered SAR imagery.

Dataset	SAR source	EO source	GSD	Eval. size	Split protocol
QXS-SAROPT ( Huang et al., 2021 )	GF-3	Google Earth	1 $\mathrm{m}$	$256^{2}$	C-DiffSET splits ( Do et al., 2026 )
SAR2Opt ( Zhao et al., 2022 )	TerraSAR-X	Google Earth	1 $\mathrm{m}$	$600^{2}\!\to\!512^{2}$	Official train/test folders
SAR2EO ( Du et al., 2023 )	Airborne UNICORN SAR	Monochrome WAMI	–	$256^{2}$	E3Diff split ( Qin et al., 2024 )
SpaceNet6 ( Shermeyer et al., 2020 )	Airborne Capella X-band	WorldView-2	0.5 $\mathrm{m}$	$256^{2}$	UTM-easting blocks; 450 m guard band
CAP-BSG	Capella X-band GEC	BlackSky	0.9–1.25 $\mathrm{m}$	$256^{2}$	Location-group split
KOMPSAT	X-band SAR	KOMPSAT-3 RGB	–	$256^{2}$	Acquisition-group split

Appendix

Table 10: Downstream datasets and evaluation protocols. GSDs refer to the SAR observations where specified; – denotes an unspecified value. SAR2EO uses the fixed first 4,000 samples of the 21,260-pair E3Diff test split. SAR2Opt uses a deterministic center crop.

Figure 6: Qualitative comparison on the QXS-SAROPT dataset. All methods receive identical SAR inputs, with paired EO images shown as references.

Figure 7: Qualitative comparison on the SAR2Opt dataset. All methods receive identical SAR inputs, with paired EO images shown as references.

Figure 8: Qualitative comparison on the SAR2EO dataset. All methods receive identical SAR inputs, with paired EO images shown as references.

Figure 9: Qualitative comparison on the SpaceNet6 dataset. All methods receive identical SAR inputs, with paired EO images shown as references.

Figure 10: SAR and EO reconstruction across pretraining sources. Top: reconstructions of original SAR and EO inputs. Bottom: speckle-perturbed SAR inputs ( $L=4$ , left) and their reconstructions (right). Values indicate PSNR (dB) relative to the corresponding original observations; parenthesized values refer to perturbed inputs.

Figure 11: Domain-aware speckle augmentation across pretraining sources. From left to right: original SAR inputs and perturbed inputs with $L=16$ , $8$ , and $4$ . Smaller $L$ yields stronger perturbations. Values indicate PSNR (dB) relative to the original SAR observations.

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 111 papers

C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation
Nov 16, 2024Jeonghyeok Do, Jaehyup Lee, Munchurl KimImage-to-Image Translation Latent Diffusion Models
KAIST · KNU
ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation
Sep 1, 2026Jeonghyeok Do, Seungchul Lee, Munchurl KimFlow Matching Image-to-Image Translation
KAIST · Stellarvision Inc.
SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models
Oct 7, 2026Jeonghyeok Do, Munchurl KimRemote Sensing Image Understanding Multimodal Pretraining
Korea Advanced Institute of Science and Technology (KAIST)

Nov 16, 2024J/K move · Enter open · S save

C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation

Jeonghyeok Do, Jaehyup Lee, Munchurl Kim

KAIST · KNU

Image-to-Image Translation Latent Diffusion Models Diffusion Model Fine-Tuning Synthetic Aperture Radar

Open paper AskPDF arXiv

Synthetic Aperture Radar (SAR) imagery provides robust environmental and temporal coverage (e.g., during clouds, seasons, day-night cycles), yet its noise and unique structural patterns pose interpretation challenges, especially for non-experts. SAR-to-EO (Electro-Optical) image translation (SET) has emerged to make SAR images more perceptually interpretable. However, traditional approaches trained from scratch on limited SAR-EO datasets are prone to overfitting. To address these challenges, we introduce Confidence Diffusion for SAR-to-EO Translation, called C-DiffSET, a framework leveraging pretrained Latent Diffusion Model (LDM) extensively trained on natural images, thus enabling effective adaptation to the EO domain. Remarkably, we find that the pretrained VAE encoder aligns SAR and EO images in the same latent space, even with varying noise levels in SAR inputs. To further improve pixel-wise fidelity for SET, we propose a confidence-guided diffusion (C-Diff) loss that mitigates artifacts from temporal discrepancies, such as appearing or disappearing objects, thereby enhancing structural accuracy. C-DiffSET achieves state-of-the-art (SOTA) results on multiple datasets, significantly outperforming the very recent image-to-image translation methods and SET methods with large margins.

Abstract

Image-to-Image Translation

Explore similar work

C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation

ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation

SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models