Fluorescence-enhanced Whisker Array with Vision-based Deformation Analysis for Underwater Source Localization
Authors: Xiaochi Xie, Hao Li, Shixuan Zhao, Siyue Yao, Shuran Song, Mark R. Cutkosky
Organizations: Department of Mechanical Engineering, Stanford University, USA · State Key Laboratory of Mechanical Systems and Vibration; META Robotics Institute, Shanghai Jiao Tong University, Shanghai 200240, China · Department of Electrical Engineering, Stanford University, USA
Deep-water biological observation is essential for understanding marine organisms and their interactions with the environment. However, conventional optical and acoustic approaches can introduce stimuli that alter animal behavior and bias biological observations. This paper proposes a fluorescence-enhanced whisker array sensing system that pinpoints underwater hydrodynamic sources through local optical readout rather than direct source imaging. Five spatially oriented whiskers, fabricated with nitinol cores and fluorescent urethane shells, are integrated with ultraviolet excitation and a monocular camera. Image enhancement and segmentation are applied to track the whisker deformation. A lightweight convolutional neural network captures temporal and cross-whisker features from 2 s sequences to estimate source localization. Pool experiments achieve a mean spatial localization error of 88 mm, with 73.5 mm in radius and 2.5∘ in angle, across a test region of 600 mm with ±30∘. Real-time localization of a moving thruster demonstrates the capability of the proposed method in dynamic scenarios, highlighting its potential for integration into underwater robots for hydrodynamic source detection, localization, and tracking in low-light environments.
Figures & tables
Fig. 1: Overview of the proposed fluorescence-enhanced whisker sensing framework. (A) Bio-inspired sensing concept, where flow disturbances generated by an underwater source induce whisker deformation that is captured by monocular vision and used for source localization. (B) Representative moving-source experiments demonstrating real-time localization along different source trajectories.
Fig. 2: Design and configuration of the fluorescence-enhanced whisker-array sensing system. (A) Bio-inspired compliant whisker with a wavy tapered profile and an elliptical cross-section. (B) Compact sensing platform consisting of a monocular camera, two UV light sources, and an array with five spatially oriented whiskers. The top and side views show the different yaw and pitch orientations of whiskers.
Fig. 3: Fabrication process and performance of the fluorescence-enhanced compliant whiskers. (A) Two-stage molding procedure with fluorescence incorporated. (B) Illustrative comparison of whiskers with and without fluorescent pigment under conditions with and without UV illumination.
Fig. 4: Pipeline of the proposed vision-based deformation analysis and underwater source localization method, including pre-processing, learning-based regression using a lightweight CNN, and source localization.
Fig. 5: Experimental setup of underwater source localization. (A) Overall configuration, where a thruster mounted on a three-axis gantry is positioned within a fan-shaped test region and continuously oriented toward the whisker array. (B) Side view of the setup. (C) Actual brightness under the illumination condition during data acquisition.
Fig. 6: Source localization performance. (A) Spatial distribution of the predicted source locations. Filled and hollow markers indicate the on-grid and off-grid locations. The radius of each error circle denotes the mean absolute localization error at the corresponding location. (B) Predicted versus actual distance r and angle θ . The diagonal line indicates the ideal prediction y=x , and the box plots summarize the 25% , 50% , and 75% percentiles of the predictions at each test coordinate. (C) Spatial distributions of the radial error Δr and angular error Δθ over the test region.
ΔrMAE (mm)
ΔθMAE ( ∘ )
ΔdMAE (mm)
(Avg/Min/Max)
(Avg/Min/Max)
(Avg/Min/Max)
On-grid
45.2 / 0.1 / 202.6
0.47 / 0.02 / 3.76
47.2 / 0.9 / 202.8
Off-grid
77.7 / 0.1 / 278.6
2.78 / 0.01 / 11.75
94.2 / 5.5 / 279.2
All ( Avg )
73.5
2.48
88.1
TABLE I: Quantitative localization errors on the testing set.
Fig. 7: Demonstrations and performance of a moving-source dynamic localization. (A) Comparison under different motion speeds, from quasi-static to more dynamic conditions. (B) Repeated trials demonstrating repeatability performance of localization. (C) Representative real-time localization of different source trajectories, together with the estimated radius r and angle θ .
Fig. 8: Comparisons of whisker deformation under thruster with different power. (A) Original data sequence. (B) Average displacement.
Underwater environments impose severe constraints on conventional imaging systems and demand solutions that balance high-quality sensing with strict resource efficiency. While emerging event cameras offer a promising alternative, their potential in aquatic scenarios remains largely unexplored. Through the lens of neuromorphic vision, this work pioneers the investigation of motion fields that serve as key media for agile underwater perception. Built upon spiking neural networks, we introduce a self-supervised framework to estimate per-pixel optical flow from asynchronous event streams, elegantly bypassing the long-standing bottleneck of underwater data scarcity. Extensive evaluations demonstrate that our method achieves competitive visual and quantitative results against leading techniques while operating with superior computational efficiency. By bridging neuromorphic sensing and aquatic intelligence, this work opens new frontiers for lightweight, real-time, and low-cost perception on resource-constrained underwater edge platforms.
Pei Zhang, Yunkai Liang, Kaiqiang Wang
School of Electrical Engineering, Guangxi University · Baise Artificial Intelligence Innovation and Development Center · School of Physical Science and Technology, Northwestern Polytechnical University
This study presents a bio inspired signal processing framework for robust Underwater Acoustic Target Recognition (UATR). The latest state of the art methods often fail to resolve dense low frequency harmonic structures in vessel propulsion signals under high noise conditions, which is addressed by the proposed framework using a biologically inspired Gammatone filter bank that emulates the cochlea nonlinear frequency selectivity. By distributing filters according to the Equivalent Rectangular Bandwidth (ERB) scale, the framework achieves a high fidelity representation of engine radiated tonals while effectively suppressing isotropic ambient interference. The resulting Cochleagram features are processed by a lightweight, custom designed Convolutional Neural Network (CNN) that leverages large receptive fields to integrate spectral-temporal continuities. Experimental results on the VTUAD dataset demonstrate a state of the art classification accuracy of 98.41%, outperforming Continuous Wavelet Transform and Mel Frequency Cepstral Coefficients baselines by 3.5% and 7.7% respectively. Furthermore, the framework achieves an inference latency of only 0.77 ms and a 0.971 Cohen Kappa score, validating its efficacy for real time deployment on autonomous, low-power sonar hardware.
Submarine fiber-optic cables instrumented with distributed acoustic sensing (DAS) provide an effective approach for large-scale monitoring of fin whales. We present an end-to-end workflow for detecting, characterizing, and localizing fin whale notes, tested on two submarine telecom cables in the Strait of Gibraltar and western Alboran Sea. The workflow applies a kurtosis-value picker adapted to narrow-band fin whale notes. Channel-wise detections are grouped into individual notes using density-based spatio-temporal clustering, cluster agglomeration, and hyperbolic fitting to reject incoherent picks. The retained clusters are characterized through temporal, spectral, and energy-related descriptors that support note-type discrimination and estimation of inter-note intervals. Relative arrival times across DAS channels are then used in a grid-search procedure to estimate candidate source locations. Evaluation against manually annotated detections from six fin whale songs yielded median pick-level precision of 0.990 and recall of 0.744, and median cluster-level precision of 0.880 and recall of 0.806. Representative applications demonstrate separation of overlapping vocalizations, characterization of type-A and type-B notes, and the inference of apparent source movement. By transforming dense DAS recordings into compact note-level bioacoustic information, the workflow provides an integrated framework for fin whale monitoring and a basis for adaptation to other synchronized acoustic receiver arrays.