KAUSTics-Net: Diffusion-Based Through-Water UAV Restoration with a Semi-Synthetic Dataset

Dario Scilla; Omar A. Lopez; Kasper Johansen; Brian O. Nieuwenhuis; Matthew F. McCabe

cs.CVNov 2, 2026

KAUSTics-Net: Diffusion-Based Through-Water UAV Restoration with a Semi-Synthetic Dataset

Authors: Dario Scilla, Omar A. Lopez, Kasper Johansen, Brian O. Nieuwenhuis, Matthew F. McCabe

Organizations: Biological and Environmental Science and Engineering Division, Climate and Livability Initiative, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia · Red Sea Research Center, Biological and Environmental Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia

Abstract

Accurate mapping of shallow-water environments using unmanned aerial vehicle (UAV) imagery is challenged by water-induced distortions, including refraction, caustics, and wavelength-dependent attenuation. The scarcity of paired distorted and reference imagery further limits the development of supervised restoration methods. To address this challenge, we introduce KAUSTics, a semi-synthetic dataset combining 1,280 reference patches derived from 32 high-resolution UAV images with 12,800 degraded observations generated through Blender-based water-surface rendering and empirical color and blur augmentations. We further propose KAUSTics-Net, a pixel-space conditional diffusion framework integrating a multi-scale conditional encoder, feature-wise linear modulation (FiLM), cross-attention, and a composite reconstruction objective to restore shallow-water imagery. We evaluated the framework against eight image-restoration methods, complemented by architectural ablations and cross-dataset experiments. On KAUSTics, KAUSTics-Net achieved a mean peak signal-to-noise ratio (PSNR) of 31.715 +/- 0.439 dB, structural similarity index (SSIM) of 0.849 +/- 0.002, and learned perceptual image patch similarity (LPIPS) of 0.145 +/- 0.007 across three training seeds. On the independent Sea-Undistort dataset, sampler-aware fine-tuning improved restoration fidelity, reaching 19.469 +/- 0.332 dB PSNR and 0.554 +/- 0.005 SSIM under combined water-surface distortions, although perceptual trade-offs remained. Additional experiments on real UAV imagery examined feature preservation and segmentation behavior. These findings demonstrate the potential of semi-synthetic training and conditional diffusion for shallow-water image restoration while highlighting the importance of evaluating both reconstruction fidelity and downstream remote-sensing utility.

Figures & tables

Fig. 1: Area of interest for the KAUSTics dataset. The background shows a Sentinel-2 RGB composite of the coastal waters near Thuwal, Saudi Arabia, in the central Red Sea. The orange lines indicate the approximate area of interest of the UAV surveys used to acquire the calm-water reference imagery over shallow reef flats and sandy substrates.

Fig. 2: Rendering setup used to generate the synthetic component of KAUSTics. (a) Blender scene showing the UAV-derived seafloor texture mapped onto the bottom plane and viewed by a nadir-looking camera. (b) Mesh of the simulated water surface generated using the Ocean Modifier, with a central zoom showing its spatial sampling density.

Fig. 3: Overview of the KAUSTics semi-synthetic dataset. UAV imagery is divided into $1000\times 1000$ pixel reference patches and rendered in Blender to generate two caustic patterns. Gaussian blur and three empirical color-transfer configurations (Color 1–3) expand the dataset to ten degraded observations per reference patch.

Fig. 4: Examples from the Sea-Undistort dataset illustrating four scenes under three distortion configurations: all effects, no waves, and no sun glint. The bottom row shows the corresponding original water-free reference images, which are identical across the three configurations.

Distortion	KAUSTics	Sea-Undistort
Caustics	Rendered	—
Refraction	Rendered	Rendered
Sun glint	—	Rendered
Light absorption	Augmented	Rendered
Turbidity	Augmented	Rendered

TABLE I: Distortions represented in KAUSTics and Sea-Undistort, distinguishing physically rendered effects from empirical data augmentations.

Fig. 5: Architecture of KAUSTics-Net. The distorted observation $\mathbf{x}_{\mathrm{cond}}$ is processed by a multi-scale conditional encoder and concatenated with the noisy target $\mathbf{x}_{t}$ at the U-Net input. Time- and image-conditioned FiLM residual blocks integrate conditioning information throughout the network, while single-head cross-attention fuses U-Net and conditional features at the bottleneck. The decoder predicts diffusion velocity $\mathbf{v}_{\theta}(\mathbf{x}_{t},\mathbf{x}_{\mathrm{cond}},t)$ for reference-image reconstruction.

Partition	UAV images	Patches	Pairs
Training	26	1,040	10,400
Validation	3	120	1,200
Test	3	120	1,200
Total	32	1,280	12,800

TABLE II: KAUSTics source-image splits. Each UAV image contributes 40 reference patches, with ten degraded observations per patch.

Method	LR	Batch	GPU	Time (h)
CycleGAN	$2\times 10^{-4}$	16	A100	7.455
UGAN	$1\times 10^{-4}$	128	V100	1.302
FUnIE-GAN	$3\times 10^{-4}$	128	V100	1.212
DeWaterGAN	$1\times 10^{-4}$	8	A100	21.392
DiffWater	$3\times 10^{-6}$	1	A100	20.931
ResShift	$5\times 10^{-5}$	8	V100	26.470

TABLE III: Training configurations and measured runtimes on KAUSTics. All models used

256\times 256

images. Times represent mean full-run training durations across three seeds, including validation and resume overhead.

Fig. 6: Qualitative comparison of restoration methods on ten selected KAUSTics test patches. Rows represent the two caustic patterns under the base, blurred, and three color-transfer configurations (Color 1–3). Columns show the distorted input, calm-water reference, eight benchmark outputs, and KAUSTics-Net. All methods use identical $256\times 256$ inputs and seed-1 checkpoints. The central zoomed regions illustrate differences in local texture and distortion suppression.

Fig. 7: Qualitative comparison of restoration methods on the Sea-Undistort test set. Rows show three scenes under the all-effects, no-waves, and no-sunglint configurations. Columns correspond to the distorted input, original water-free reference, eight benchmark methods, and KAUSTics-Net. All methods were evaluated at $256\times 256$ resolution using the corresponding seed-1 checkpoints selected through validation.

Fig. 8: Real-UAV structural diagnostics for the GCP (0.600 m depth) and bucket (0.900 m depth). Columns show the temporal computer-vision reference, distorted input, and eight benchmark restorations alongside KAUSTics-Net. Each object is evaluated using SAM segmentation regions and SIFT keypoints. All model outputs receive identical low-frequency color harmonization ( $\alpha=1$ , $\sigma=5$ pixels), while the input and reference remain unchanged. Red boxes identify corresponding object-centered regions, and insets enlarge these regions for visualization. Histograms show the spatial distribution of SIFT keypoints using 20 bins and a common count scale within each object. SAM colors distinguish automatically generated regions and do not represent semantic classes. All methods use identical SAM and SIFT settings.

Fig. 9: SIFT feature correspondence between the distorted input or restored GCP images and the temporal computer-vision reference. Each panel shows the evaluated image on the left and the common reference on the right. Orange lines indicate mutual nearest-neighbor descriptor matches obtained using a bidirectional ratio threshold of 0.75. SIFT features were extracted with a contrast threshold of 0.040. Match counts represent putative correspondences without RANSAC-based geometric verification. No spatial registration was applied to the restored images.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. S1: Sea-Undistort zero-shot comparison using seed-1 checkpoints at 256 x 256 RGB resolution. Row groups show all effects, no waves, and no sunglint; each group contains scenes 944, 143, and 646 in that order. Columns show the distorted input, original water-free reference, and eight benchmark methods alongside KAUSTics-Net. Each panel uses the same central 26 x 26 pixel inset. No panel-specific colour correction was applied. All methods use the completed KAUSTics-trained checkpoints from Tables S1 and S2.

Method	mPSNR $\uparrow$	mSSIM $\uparrow$	mRMSE $\downarrow$	mMSE $[10^{-3}]\downarrow$	LPIPS $\downarrow$	$\Delta E_{00}\downarrow$
All effects
Unprocessed input	$10.257$	$0.311$	$0.341$	$136.826$	$0.698$	$32.385$
CycleGAN [ 37 ]	$\mathbf{10.157\pm 0.064}$	$\mathbf{0.324\pm 0.007}$	$\mathbf{0.317\pm 0.002}$	$\mathbf{105.059\pm 1.236}$	$0.714\pm 0.029$	$28.996\pm 0.282$
UGAN [ 27 ]	$10.140\pm 0.095$	$0.305\pm 0.005$	$0.325\pm 0.003$	$114.512\pm 2.043$	$0.771\pm 0.014$	$\mathbf{28.376\pm 0.517}$
FUnIE-GAN [ 38 ]	$9.249\pm 0.156$	$0.204\pm 0.011$	$0.369\pm 0.015$	$152.392\pm 16.087$	$0.744\pm 0.011$	$34.011\pm 1.487$
DeWaterGAN [ 23 ]	$10.081\pm 0.185$	$0.309\pm 0.009$	$0.343\pm 0.009$	$134.131\pm 7.198$	$0.719\pm 0.028$	$31.500\pm 1.046$

Appendix

TABLE S1: Sea-Undistort zero-shot full-reference metrics on 120 held-out scenes per configuration. Results are mean

\pm

sample standard deviation over three source-training seeds; all results are evaluated against the original water-free reference. MSE is shown in units of

10^{-3}

, and RMSE uses

[0,1]

RGB units. The unprocessed input is provided for context.

Method	NIQE $\downarrow$	BRISQUE $\downarrow$	PIQE $\downarrow$	ILNIQE $\downarrow$	LIQE $\uparrow$	CLIPIQ $\uparrow$	MUSIQ_AVA $\uparrow$	Sharpness	Entropy
All effects
Unprocessed input	$8.348$	$30.378$	$23.864$	$64.283$	$1.935$	$0.388$	$3.205$	$1380.087$	$5.211$
CycleGAN [ 37 ]	$\mathbf{6.710\pm 0.286}$	$20.166\pm 6.844$	$\mathbf{10.621\pm 1.945}$	$74.590\pm 6.220$	$1.525\pm 0.120$	$0.325\pm 0.043$	$3.072\pm 0.033$	$191.879\pm 47.371$	$5.257\pm 0.294$
UGAN [ 27 ]	$7.389\pm 0.166$	$26.358\pm 2.449$	$16.154\pm 0.495$	$98.233\pm 0.339$	$1.033\pm 0.004$	$0.193\pm 0.004$	$2.688\pm 0.028$	$106.640\pm 14.310$	$5.379\pm 0.106$
FUnIE-GAN [ 38 ]	$10.580\pm 0.142$	$\mathbf{20.007\pm 12.755}$	$37.940\pm 7.249$	$112.441\pm 3.868$	$1.165\pm 0.066$	$0.301\pm 0.036$	$3.075\pm 0.065$	$708.780\pm 432.149$	$5.732\pm 0.297$
DeWaterGAN [ 23 ]	$9.392\pm 0.626$	$42.390\pm 3.089$	$39.822\pm 4.841$	$79.853\pm 3.556$	$1.629\pm 0.071$	$0.296\pm 0.026$	$2.904\pm 0.019$	$428.506\pm 51.456$	$4.782\pm 0.126$

Appendix

TABLE S2: Sea-Undistort zero-shot no-reference metrics on the same 120 held-out scenes per configuration and the same checkpoints as Table S1 . Results are mean

\pm

sample standard deviation over three source-training seeds. Sharpness and entropy are descriptive diagnostics and are not ranked.

Fig. S2: Verified reference-image pixel inventory. Training, validation, and test contain 1,040, 120, and 120 reference patches. Original 1000 x 1000 references total 1,280.000 million spatial pixels; resized 256 x 256 references total 83.886 million. References are counted once without their ten degraded observations. Axes have different vertical scales; pixel totals do not measure unique geographic or habitat coverage.

Fig. S3: Representative calm-water KAUSTics reference patches grouped by descriptive appearance: (a) hard-bottom or rock-like, (b) sand-dominated, (c) vegetation-like or mixed benthic, and (d) dark or low-visibility. These labels are qualitative descriptions and do not indicate field-validated habitat classes, species, or measured water depths.

Fig. S4: KAUSTics references and their rendered or augmented counterparts. Rows follow the descriptive appearance groups of Supplementary Fig. S3. Columns show (a) calm-water references, (b–c) two rendered caustic patterns, (d–e) their Gaussian-blurred variants, and (f–h) empirical color-transfer presets. Color presets are not physical depth classes.

Image Restoration

Image restoration uses computational methods to recover useful visual content from noisy, blurred, low-resolution, or otherwise degraded images.

See all 386 papers

Zero-Shot Benchmarking of Unified Multimodal Models for Shallow-Water UAV Image Distortion Assessment and Restoration
Nov 6, 2026Dario Scilla, Omar A. Lopez, Mattia Rosso +2Image Restoration Underwater Image Enhancement
Biological and Environmental Science and Engineering Division, Climate and Livability Initiative, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia · Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia
Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark
Apr 10, 2026Jinquan Yan, Zhicheng Zhao, Zhengzheng Tu +3Image Restoration All-in-One Image Restoration
Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei 230601, China. · Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University, Hefei 230601, China.
CoRE-UIR: Prior-guided common and residual experts for efficient all-in-one remote sensing image restoration
Jul 30, 2026Zaiyan Zhang, Qiangqiang Yuan, Jie Li +5Mixture of Experts All-in-One Image Restoration
School of Geodesy and Geomatics, Wuhan University, Wuhan 430079, Hubei, China · School of Artificial Intelligence, Wuhan University, Wuhan 430072, Hubei, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, Hubei, China

Nov 6, 2026J/K move · Enter open · S save

Zero-Shot Benchmarking of Unified Multimodal Models for Shallow-Water UAV Image Distortion Assessment and Restoration

Dario Scilla, Omar A. Lopez, Mattia Rosso, Kasper Johansen+1

Biological and Environmental Science and Engineering Division, Climate and Livability Initiative, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia · Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia

Image Restoration Underwater Image Enhancement Unified Multimodal Models Multimodal Model Evaluation

Open paper AskPDF arXiv

Shallow-water UAV imagery provides centimeter-scale observations of benthic environments, but water-induced distortions complicate interpretation. To our knowledge, we present the first systematic zero-shot benchmark of unified multimodal large language models (MLLMs) for both assessing water-induced distortions and restoring shallow-water imagery within the same model framework. We evaluate five open-weight unified MLLMs with frozen pretrained weights and without access to paired reference images or descriptions of the true seabed. The benchmark contains 1,000 image pairs: 600 semi-synthetic UAV scenes created by applying synthetic water distortions to real shallow-water imagery and 400 fully synthetic through-water scenes. Three independent human annotations cover five distortion types. We evaluate four assessment prompts (P0-P3), from direct instruction to structured task decomposition, on all 1,000 images and four diagnosis-guided restoration prompts (R0-R3) on a common 100-image subset. Assessment varies substantially by model and prompt, with systematic prediction biases and a best pooled balanced accuracy of 0.654 for distortion presence. The best pooled PSNR gain is +0.255 dB, no unified MLLM improves pooled LPIPS, and some outputs alter scene structure or hallucinate content. Two restoration-only baselines also show limited transfer: InstructIR achieves a pooled PSNR gain of +0.040 dB, while PromptFix degrades all four pooled metrics under every prompt. These results show that recognizing shallow-water degradation does not imply reliable physical restoration and provide a reproducible baseline for future work.

Abstract

Image Restoration

Explore similar work

Zero-Shot Benchmarking of Unified Multimodal Models for Shallow-Water UAV Image Distortion Assessment and Restoration

Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark

CoRE-UIR: Prior-guided common and residual experts for efficient all-in-one remote sensing image restoration