cs.CVNov 6, 2026
SaveZero-Shot Benchmarking of Unified Multimodal Models for Shallow-Water UAV Image Distortion Assessment and Restoration
Organizations: Biological and Environmental Science and Engineering Division, Climate and Livability Initiative, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia · Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia
Abstract
Shallow-water UAV imagery provides centimeter-scale observations of benthic environments, but water-induced distortions complicate interpretation. To our knowledge, we present the first systematic zero-shot benchmark of unified multimodal large language models (MLLMs) for both assessing water-induced distortions and restoring shallow-water imagery within the same model framework. We evaluate five open-weight unified MLLMs with frozen pretrained weights and without access to paired reference images or descriptions of the true seabed. The benchmark contains 1,000 image pairs: 600 semi-synthetic UAV scenes created by applying synthetic water distortions to real shallow-water imagery and 400 fully synthetic through-water scenes. Three independent human annotations cover five distortion types. We evaluate four assessment prompts (P0-P3), from direct instruction to structured task decomposition, on all 1,000 images and four diagnosis-guided restoration prompts (R0-R3) on a common 100-image subset. Assessment varies substantially by model and prompt, with systematic prediction biases and a best pooled balanced accuracy of 0.654 for distortion presence. The best pooled PSNR gain is +0.255 dB, no unified MLLM improves pooled LPIPS, and some outputs alter scene structure or hallucinate content. Two restoration-only baselines also show limited transfer: InstructIR achieves a pooled PSNR gain of +0.040 dB, while PromptFix degrades all four pooled metrics under every prompt. These results show that recognizing shallow-water degradation does not imply reliable physical restoration and provide a reproducible baseline for future work.
Figures & tables
Fig. 1: Overview of the proposed benchmarking framework. (I) : The 1,000-image dataset is first independently annotated by three human annotators for five water distortions. (II) : Five unified MLLMs then assess all images using four zero-shot assessment prompts (P0–P3). (III) : Model-generated diagnoses are then used to guide restoration on a 100-image subset using four restoration prompts (R0–R3).
Fig. 2: Representative distorted images from the two source datasets. Top row: KAUSTics UAV semi-synthetic scenes ( pixels). Bottom row: Sea-Undistort synthetic through-water scenes ( pixels). Both sources show distortion patterns characteristic of shallow-water imaging, alongside variation in scene appearance and distortion severity.
| Task | Name | Method | Description | Operative prompt |
| Assessment | P0 Direct | Zero-shot baseline | Minimal instruction; establishes performance without domain scaffolding. | “Analyze this UAV shallow-water image. For caustics, refraction, sun glint, turbidity, and light absorption, assign none, low, medium, or high. Select the dominant distortion and overall restoration need. Judge only visible evidence; return one valid JSON object.” |
| Assessment | P1 Expert | Role/context | Adds a shallow-water remote-sensing and aquatic-optics role without definitions. | “Assess the water-induced degradation in this UAV shallow-water image as a remote-sensing scientist specializing in aquatic optics. Grade the five distortions from visible evidence and identify what most limits seabed interpretability. Return one valid JSON object.” |
| Assessment | P2 Rubric | Definition-guided | Adds physical definitions, severity anchors, and counterexamples for common confounders. | “Assess using operational definitions: caustics are focused bright networks; refraction is water-surface geometric displacement; sun glint is saturated specular reflection; turbidity is a scattering veil; light absorption is broad attenuation/color loss. Use none/low/medium/high anchors and do not confuse seabed appearance with an artifact. Return one valid JSON object.” |
| Assessment | P3 Checklist | Task decomposition | Orders radiometric, illumination, geometry, and visibility checks; only the final answer is exposed. | “Internally check: A) global radiometry, B) caustics versus sun glint, C) geometric instability, and D) scattering/visibility. Then grade all five distortions and restoration need. Do not output chain-of-thought; return only the final JSON.” |
| Restoration | R0 Priority-direct | Direct editing | Applies the score-derived correction order with the least additional scaffolding. | “Restore image [ID] into a plausible undistorted view of the same scene. Apply [ORDERED ACTIONS] from highest to lowest priority using [DIAGNOSIS] and [PRIORITY PLAN]. Correct only diagnosed artifacts and return one natural restored image.” |
| Restoration | R1 Expert optics | Role/context | Requests physically conservative, artifact-specific corrections. | “As an underwater-optics scientist, create a scientifically plausible undistorted version. Correct higher-scoring distortions first: suppress glint without inventing seabed; stabilize refraction; flatten caustics; reduce turbidity; then correct absorption. Preserve radiometric and geometric realism.” |
TABLE II: Assessment and restoration prompt design. The fixed assessment JSON schema and the restoration diagnosis and priority placeholders are applied consistently to all prompt variants within their corresponding stage. [ID], [ORDERED ACTIONS], [PRIORITY PLAN] and [DIAGNOSIS] are retrieved from the assessment output.
| Severity level | Distortion type | ||
| None | 0 | Sun glint | 5 |
| Low | 1 | Refraction | 4 |
| Medium | 2 | Caustics | 3 |
| High | 3 | Turbidity | 3 |
| – | – | Light absorption | 2 |
| – | – | None | 0 |
TABLE III: Ordinal severity scores and fixed restoration priorities used to construct the diagnosis-guided restoration plan. The priority values determine correction order and should not be interpreted as measures of physical distortion magnitude.
Fig. 3: Agreement among the three annotators for each water-induced distortion. Left: four-class severity agreement, categorized as unanimous, 2–1 majority, or no majority. Right: binary presence agreement after grouping low , medium , and high as present. Each distortion has paired bars for KAUSTics (dark; 600 images) and Sea-Undistort (light; 400 images). Percentages are normalized within each dataset and labels are shown for segments of at least 8%.
Fig. 4: Assessment performance across the four prompt variants, separated by dataset. Each model–prompt pair uses dark KAUSTics and light Sea-Undistort bars. Scores are macro-averaged over defined distortion-specific metrics; caustics presence BA is undefined within both datasets because one reference class is absent. Error bars show within-dataset image-bootstrap 95% confidence intervals. Higher values indicate better performance for all metrics. An ‘ ’ in the conditional-severity panel indicates zero detection coverage, for which conditional severity is undefined.
Fig. 5: Assessment performance by prompt, model, and distortion over the pooled 1,000 images. Rows show presence balanced accuracy, exact four-class severity accuracy, and the joint ordinal score ( ). Values are reported as percentages. Both presence classes occur in the pooled data for every distortion, so all presence BA values are defined. Bold numbers identify the highest score across models for each distortion within each metric–prompt panel.
Fig. 6: Fraction of evidence descriptions containing at least one frequently used hedge term (’slight’, ’subtle’, ’faint’, ’appear’, ’slightly’, ’mild’), separated by model, correctness, and dataset.
Fig. 7: Predicted severity distribution conditioned on each frequently occurring hedge term, with dark KAUSTics and light Sea-Undistort bars. The baseline uses all evidence descriptions within the corresponding dataset. Each bar sums to 100%; segments of at least 8% are labeled.
Fig. 8: Restoration change from the matching input baseline for R0–R3, for KAUSTics (dark) and Sea-Undistort (light). Error bars show paired image-bootstrap 95% confidence intervals. Positive change indicates improvement. The dotted line separates the five unified MLLMs from InstructIR and PromptFix.
Fig. 9: Five common restoration examples using restoration prompt R0. Columns show the distorted input, paired reference, five unified MLLMs, InstructIR, and PromptFix. Suffixes _k and _s identify KAUSTics and Sea-Undistort datasets.The paired reference is displayed for comparison.
Fig. 10: Representative restoration failures grouped by visible behavior: artifacts, hallucinations, text over image, text only, underwater scenarios, and mixed failures. Two examples per category are shown. Labels identify the model, restoration prompt, and image. Text-only examples are dominated by document or interface content; underwater scenarios depict a generated submerged viewpoint. Categories can overlap.
Fig. 11: Best-case downstream diagnostics for each model’s own three highest-SSIM-gain restorations per dataset. Each group contains three input/restoration pairs; short black marks show the corresponding reference counts. The SIFT count axis is logarithmic above one and linear near zero. Moving the restored value closer to the reference indicates improved count agreement, but does not by itself establish recovery of the same physical features or regions.
Fig. 12: Matched-image downstream diagnostics for Sea-Undistort 0277_s and KAUSTics 0467_k. Columns show the common reference, distorted input, and R0 restoration prompt from each model. The SAM row shows mask overlays and the SIFT row shows all detected keypoints together with top/right spatial histograms. Insets report , the number of masks or keypoints. Histograms use 20 bins. Viridis mask colors distinguish regions locally and do not represent semantic classes. Similar feature or mask counts therefore do not necessarily indicate recovery of the same reference structure.
Fig. S1: Median evidence length by model and dataset, using all assessment evidence descriptions. Dark/light bars denote KAUSTics/Sea-Undistort; error bars show within-dataset interquartile ranges. The pooled comparison differs across models (Kruskal–Wallis , ).
Fig. S2: Predicted severity distributions for each model’s best pooled presence-BA prompt, as in Table VI , compared with the three human annotations and stratified by dataset. Dark/light bars denote KAUSTics/Sea-Undistort. Each bar shows within-dataset percentages assigned to none , low , medium , and high , and sums to 100%.
Fig. S3: Row-normalized severity confusion matrices for each model and distortion using that model’s best pooled presence-BA prompt, as in Table VI . Rows represent the human severity grade and columns the model prediction. Each row sums to 100%; stronger values along the diagonal indicate better severity discrimination, while a consistently dominant column indicates a systematic prediction bias. Bold diagonal values identify the highest correct-class rate across models for each distortion and human severity.
| Category | Operational interpretation |
| Caustics | Focused or network-like bright patterns produced by surface-wave focusing; severity reflects their visual prominence rather than total scene brightness. |
| Refraction | Apparent spatial displacement, warping, or waviness of scene structure caused by refraction at the water surface. |
| Sun glint | Bright specular surface reflections, including saturated highlights that conceal underlying scene content. |
| Turbidity | Scattering-induced haze or veiling, reduced contrast, and loss of visibility attributable to suspended material. |
| Light absorption | Broad attenuation and wavelength-dependent loss of color or detail as light propagates through water. |
| None | No visible evidence of the distortion. |
TABLE S1: Operational interpretation of the distortion and severity categories. No numerical thresholds or example anchors were embedded in the annotation interface.
Fig. S4: Annotation interface used to present one distorted image and collect one mandatory severity selection for each distortion. The screenshot shows the unselected state; the notes field was optional.
More in this topic
Image Restoration
Image restoration uses computational methods to recover useful visual content from noisy, blurred, low-resolution, or otherwise degraded images.
See all 386 papers