Computational imaging through turbid, bubbly and flowing water near the telescope
Authors: Alexander Ingold, Sabina D. Menon, Aadhi Umamageswaran, Alec Ikei, May H. Hlaing, Syed N. Qadri, Rajesh Menon
Organizations: Department of Electrical and Computer Engineering, University of Utah, Salt Lake City, UT 84112, USA · U.S. Naval Research Laboratory Remote Sensing Division, Washington DC 20375, USA. · Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139 USA.
Cameras in the surf zone or behind ship wakes look through turbid, bubbly, moving water close to the lens, yet restoration networks are usually scored only on displayed images. We introduce a measured dataset of a telescope viewing displayed images and physical objects through a 25mm water layer 45mm in front of its objective, under seven combinations of flow, bubbles and turbidity, with clear-water recordings as a second reference. Because the suspension scatters light far outside the 1.3\textdegree{} field of view, turbidity mainly removes contrast, 4.4- to 4.8-fold on a Siemens star. Restoring the mean of eight frames improves structural similarity (SSIM) by 0.032 over a single frame, and learned fusion of the frames adds little. One raw frame supports five-category recognition (96.8%) as well as restored images. Adding synthetic training pairs leaves display-image scores unchanged but raises the correlation of a restored Siemens star with its clear-water recording from below 0.5 to above 0.9. These networks also improve ordinary underwater photographs without retraining, where training on measured pairs alone fails. Scoring against the digital source or a clear-water recording reverses the ranking of raw and restored images. These results support a simple pipeline: average the frames, classify raw frames directly, and restore with networks trained on measured and synthetic pairs.
Figures & tables
Figure 1: Applications, apparatus and measurements. (a) A camera looking through water: up through a wavy, bubbly surface, through breaking surf, through a ship’s wake, and through turbid water. (b) A Seestar S50 telescope (50 mm aperture) views a display or physical object about 15.2 m away through a 25 mm water path whose near face is 45 mm from the objective. A motor drives flow, an air pump adds bubbles and milk adds turbidity. Photographs show the tank in five conditions; see also Visualizations 1–4. (c) The displayed digital image and recorded frames of it in the seven conditions. Note that the clear image is a single frame, and degradation is due to the intrinsic telescope optics and water in the path. This image is reserved for illustration and excluded from all quantitative results.
Figure 2: Restoration of display images. (a) The reserved image restored by networks trained on the frame mean and on eight frames, in each condition. (b) SSIM in reference to the displayed image: a measured single frame, restored from mean of 8 frames, and restored from burst of frames (video); n is the number of evaluation images. See also Visualizations 5–11.
Figure 3: Physical objects through turbid water with bubbles and high motor (T+B+HM). Rows: recorded frame; contrast stretch of the eight-frame mean, no training; Restormer trained on measured data only; Restormer trained with synthetic data on the frame mean; the same on all eight frames. See Visualizations 12 and 13.
Figure 4: Restoration of ordinary underwater photographs without retraining. Two difficult UIEB photographs that have no reference images: (a) a dark fish scene and (b) a reef slope. Columns: unchanged input, and outputs of EDVR (single-frame input), Restormer (trained mean) and NAFNet (single-frame input), all trained with synthetic pairs. For each architecture, the model shown is the one with the highest mean SSIM on the 890 referenced UIEB images. Each photograph is repeated as all eight input frames. All panels use the same display range, without brightness or color adjustment.
Computational imaging combines optical and sensor design with algorithms to recover or infer scene information from indirect, compressed, or noisy measurements.
Visibility in underwater environments degrades rapidly under turbid conditions, yet the effects on computer-vision models remain unclear. This issue is compounded by reliance on synthetic turbidity datasets, which may misrepresent real-world information loss. To address this gap, we introduce the Turbid Underwater Baseline (TUB) dataset, comprising 1,320 images captured under extreme turbidity and over 16,000 high-confidence ground-truth segmentation masks. We additionally propose PCD, a metric derived from phase congruency maps that is invariant to contrast and aims to capture the loss of structural information in real turbidity. We show that PCD correlates strongly with the performance of instance segmentation models on both real and synthetic turbid images, whereas common metrics in the field show weak to no correlation at all. The dataset and relevant code can be found on the project page: https://vap.aau.dk/pcd
Vasiliki Ismiroglou, Stefan H. Bengtson, Tasos Benos +2
Visual Analysis and Perception Laboratory, Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark
We address the problem of looking into the water from the air, where we seek to remove image distortions caused by refractions at the water surface. Our approach is based on modeling the different water surface structures at various points in time, assuming the underlying image is constant. To this end, we propose a model that consists of two neural-field networks. The first network predicts the height of the water surface at each spatial position and time, and the second network predicts the image color at each position. Using both networks, we reconstruct the observed sequence of images and can therefore use unsupervised training. We show that using implicit neural representations with periodic activation functions (SIREN) leads to effective modeling of the surface height spatio-temporal signal and its derivative, as required for image reconstruction. Using both simulated and real data we show that our method outperforms the latest unsupervised image restoration approach. In addition, it provides an estimate of the water surface.
Ori Lifschitz, Tali Treibitz, Dan Rosenbaum
Hatter Department of Marine Technologies, Charney School of Marine Sciences, University of Haifa, Haifa, Israel · Department of Computer Science, University of Haifa, Haifa, Israel
Underwater image restoration consists of recovering an image which looks like there is no water present. To date, evaluation has not been systematic. This paper describes a systematic evaluation pipeline for underwater reconstruction, which can be used to assess a method for accuracy; consistency of reconstruction over camera moves; and the effect of water parameters. We use this pipeline to evaluate a range of current procedures, from models constructed using explicit but approximate physical models of scattering to Vision-Language Models (VLMs which are not currently trained with explicit physical models). Overall, VLMs wholly and significantly outperform physically based models in our evaluation, likely because of the importance of a strong image prior. Results on images of real underwater scenes strongly confirm the evaluation.
Sara Aghajanzadeh, Yingxue Wang, Ieva Bagdonaviciute +1