DINO-Explorer: Active Underwater Discovery via Ego-Motion Compensated Semantic Predictive Coding
Authors: Yuhan Jin, Nayari Marie Lessa, Mariela De Lucas Alvarez, Melvin Laux, Lucas Amparo Barbosa, Frank Kirchner, Rebecca Adam
Organizations: Robotics Innovation Center, German Research Center for Artificial Intelligence, Bremen, Germany · Robotics Research Group, University of Bremen, Bremen, Germany · Computing Department, SENAI CIMATEC, Salvador, Brazil
Marine ecosystem degradation necessitates continuous, scientifically selective underwater monitoring. However, most autonomous underwater vehicles (AUVs) operate as passive data loggers, capturing exhaustive video for offline review and frequently missing transient events of high scientific value. Transitioning to active perception requires a causal, online signal that highlights significant phenomena while suppressing maneuver-induced visual changes. We propose DINO-Explorer, a novelty-aware perception framework driven by a continuous semantic surprise signal. Operating within the latent space of a frozen DINOv3 foundation model, it leverages a lightweight, action-conditioned recurrent predictor to anticipate short-horizon semantic evolution. An efference-copy-inspired module utilizes globally pooled optical flow to discount self-induced visual changes without suppressing genuine environmental novelty. We evaluate this signal on the downstream task of asynchronous event triage under variant telemetry constraints. Results demonstrate that DINO-Explorer provides a robust, bandwidth-efficient attention mechanism. At a fixed operating point, the system retains 78.8% of post-discovery human-reviewer consensus events with a 56.8% trigger confirmation rate, effectively surfacing mission-relevant phenomena. Crucially, ego-motion conditioning suppresses 45.5% of false positives relative to an uncompensated surprise signal baseline. In a replay-side Pareto ablation study, DINO-Explorer robustly dominates the validated peak F1 versus telemetry bandwidth frontier, reducing telemetry bandwidth by 48.2% at the selected operating point while maintaining a 62.2% peak F1 score, successfully concentrating data transmission around human-verified novelty events.
Figures & tables
Fig. 1: Conceptual overview of DINO-Explorer: Inspired by predictive coding, the framework generates an intrinsic surprise signal that acts as an indicator of semantic novelty to serve active discovery. Downstream tasks, such as detecting scientifically relevant events (e.g., biological anomalies or habitat transitions), can then be driven by this signal. Furthermore, to mitigate false positives caused by the AUV’s own motion, the system isolates genuine environmental changes by implementing an efference copy-inspired ego-motion compensation module.
Fig. 2: System architecture of DINO-Explorer: The pipeline comprises Feature Extraction (left/yellow), which produces actual semantic state zt , history semantic states Zt−L:t−1 and RAFT-based motion estimates Mt−L:t−1 , and Surprise Signal Generation (right/pink). Within a predictive coding framework, the GRU-based ego motion compensated semantic predictor ( Pcomp,θ ) generates an internal expectation z^t conditioned on the motion-aware temporal context Ct−L:t−1 , which acts as an efference copy. The resulting surprise signal St , quantifying the mismatch between sensory reality zt and internal expectation z^t , is used both to drive continuous online adaptation via self-supervised backpropagation and to provide a real-time novelty indicator for downstream active discovery tasks.
Fig. 3: Qualitative analysis of the semantic surprise signal across a continuous sequence. The plot illustrates the efficacy of our predictive coding framework in isolating genuine novelty: (left/red) DINO-Explorer detects significant deviations from the predicted semantic path caused by a fish rapidly changing its trajectory, generating a valid surprise peak that is correctly preserved after ego-motion compensation; and (right/blue) the efference copy-inspired module effectively filters out predictable reafferent noise induced by an AUV maneuver, suppressing a “naive” false alarm to ensure the autonomous trigger remains robustly focused on scientifically relevant phenomena.
Fig. 4: Ablation benchmark for the proposed DINO-Explorer under matched telemetry budgets. Sweeping α avoids tying the comparison to a single downstream event-extraction hyperparameter and shows the ablation benchmark across extraction sensitivities. Each panel is a budget-controlled frontier: at the same bandwidth saving, a higher curve indicates stronger proposal quality or better preservation of the corresponding diagnostic. The sweep compares DINO-Explorer with the uncompensated surprise signal, Direct Δzt , and uniform replay, showing that the ego-motion-compensated surprise signal improves human-reviewer-facing proposal quality while preserving the Phase 1 recall, proposal-confirmation, and DINO-latent semantic-change diagnostics needed for underwater triage.
Setting
SPCR (%)
TCR (%)
FPSR (%)
DR (%)
P1
P1+P2
P1
P1+P2
α=2.5
47.5
78.8
11.8
56.8
45.5
147.8
TABLE I: Operating-point consensus summary. Phase 1 (P1) uses blind same-category consensus intervals; P1+P2 additionally counts reviewer-confirmed model-guided discoveries. FPSR compares against the uncompensated predictive variant, and DR reports additional event coverage with model discovering.
Method
Setting
BSR (%)
Peak F1 (%)
SPCR P1 (%)
TCR P1 (%)
LER (%)
Scomp
α=2.5
48.2
62.2
47.5
11.8
70.8
Suncomp
α=2.5
47.6
46.4
46.3
12.0
68.2
Direct diff ( Δzt )
α=2.5
45.9
59.5
48.2
11.3
71.2
Uniform periodic
Δt=12 s
48.0
19.5
26.5
8.6
53.0
TABLE II: Operating-point replay comparison. Peak F1 is measured against reviewer-confirmed proposal peaks; BSR , SPCR / TCR , and LER report bandwidth, recall/precision diagnostics, and latent-energy retention. Bold values mark the primary DINO-Explorer operating-point readouts.
Metric summary
Value
Avg / median / p95 latency (ms)
227.2 / 228.7 / 272.9
Average FPS
4.40
TABLE III: Runtime context for asynchronous mission-time triage on an RTX A6000.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Fig. 5: Qualitative surprise-event examples. Rows show the compensated surprise trace for a habitat transition, turbidity plume, and illumination shift, illustrating low response to predictable underwater noise and peaks on event-level semantic changes.
Autonomous underwater vehicles (AUVs) are increasingly used to survey coral reefs, yet efficiently locating specific coral species of interest remains difficult: target species are often sparsely distributed across the reef, and an AUV with limited battery life cannot afford to search everywhere. When detections of the target itself are too sparse to provide directional guidance, the robot benefits from an additional signal to decide where to look next. We propose using the visual environmental context -- the habitat features that tend to co-occur with a target species -- as that signal. Because context features are spatially denser and often vary more smoothly than target detections, we hypothesize that a reward function targeted at broader environmental context will enable adaptive planners to make better decisions on where to go next, even in regions where no target has yet been observed. Starting from a single labeled image, our method uses patch-level DINOv2 embeddings to perform one-shot detections of both the target species and its surrounding context online. We validate our approach using real imagery collected by an AUV at two reef sites in St. John, U.S. Virgin Islands, simulating the robot's motion offline. Our results demonstrate that one-shot detection combined with adaptive context modeling enables efficient autonomous surveying, sampling up to 75% of the target in roughly half the time required by exhaustive coverage when the target is sparsely distributed, and outperforming search strategies that only use target detections.
Eric Chen, Travis Manderson, Nare Karapetyan +3
Massachusetts Institute of Technology and Woods Hole Oceanographic Institution · Massachusetts Institute of Technology · Woods Hole Oceanographic Institution +1
Underwater environments impose severe constraints on conventional imaging systems and demand solutions that balance high-quality sensing with strict resource efficiency. While emerging event cameras offer a promising alternative, their potential in aquatic scenarios remains largely unexplored. Through the lens of neuromorphic vision, this work pioneers the investigation of motion fields that serve as key media for agile underwater perception. Built upon spiking neural networks, we introduce a self-supervised framework to estimate per-pixel optical flow from asynchronous event streams, elegantly bypassing the long-standing bottleneck of underwater data scarcity. Extensive evaluations demonstrate that our method achieves competitive visual and quantitative results against leading techniques while operating with superior computational efficiency. By bridging neuromorphic sensing and aquatic intelligence, this work opens new frontiers for lightweight, real-time, and low-cost perception on resource-constrained underwater edge platforms.
Pei Zhang, Yunkai Liang, Kaiqiang Wang
School of Electrical Engineering, Guangxi University · Baise Artificial Intelligence Innovation and Development Center · School of Physical Science and Technology, Northwestern Polytechnical University
Oyster reefs are critical ecosystem species that sustain biodiversity, filter water, and protect coastlines, yet they continue to decline globally. Restoring these ecosystems requires regular underwater monitoring to assess reef health, a task that remains costly, hazardous, and limited when performed by human divers. Autonomous underwater vehicles (AUVs) offer a promising alternative, but existing AUVs rely on geometry-based navigation that cannot interpret scene semantics. Recent vision-language models (VLMs) enable semantic reasoning for intelligent exploration, but existing VLM-driven systems adopt an end-to-end paradigm, introducing three key limitations. First, these systems require the VLM to generate every navigation decision, forcing frequent waits for inference. Second, VLMs cannot model robot dynamics, causing collisions in cluttered environments. Third, limited self-correction allows small deviations to accumulate into large path errors. To address these limitations, we propose CORAL, a framework that decouples high-level semantic reasoning from low-level reactive control. The VLM provides high-level exploration guidance by selecting waypoints, while a dynamics-based planner handles low-level collision-free execution. A geometric verification module validates waypoints and triggers replanning when needed. Compared with the previous state-of-the-art, CORAL improves coverage by 14.28% percentage points, or 17.85% relatively, reduces collisions by 100%, and requires 57% fewer VLM calls.
Zhenqi Wu, Yuanjie Lu, Xuesu Xiao +1
University of South Florida · George Mason University