DINO-Explorer: Active Underwater Discovery via Ego-Motion Compensated Semantic Predictive Coding

Yuhan Jin; Nayari Marie Lessa; Mariela De Lucas Alvarez; Melvin Laux; Lucas Amparo Barbosa; Frank Kirchner; Rebecca Adam

cs.ROApr 14, 2026

DINO-Explorer: Active Underwater Discovery via Ego-Motion Compensated Semantic Predictive Coding

Authors: Yuhan Jin, Nayari Marie Lessa, Mariela De Lucas Alvarez, Melvin Laux, Lucas Amparo Barbosa, Frank Kirchner, Rebecca Adam

Organizations: Robotics Innovation Center, German Research Center for Artificial Intelligence, Bremen, Germany · Robotics Research Group, University of Bremen, Bremen, Germany · Computing Department, SENAI CIMATEC, Salvador, Brazil

Abstract

Marine ecosystem degradation necessitates continuous, scientifically selective underwater monitoring. However, most autonomous underwater vehicles (AUVs) operate as passive data loggers, capturing exhaustive video for offline review and frequently missing transient events of high scientific value. Transitioning to active perception requires a causal, online signal that highlights significant phenomena while suppressing maneuver-induced visual changes. We propose DINO-Explorer, a novelty-aware perception framework driven by a continuous semantic surprise signal. Operating within the latent space of a frozen DINOv3 foundation model, it leverages a lightweight, action-conditioned recurrent predictor to anticipate short-horizon semantic evolution. An efference-copy-inspired module utilizes globally pooled optical flow to discount self-induced visual changes without suppressing genuine environmental novelty. We evaluate this signal on the downstream task of asynchronous event triage under variant telemetry constraints. Results demonstrate that DINO-Explorer provides a robust, bandwidth-efficient attention mechanism. At a fixed operating point, the system retains 78.8% of post-discovery human-reviewer consensus events with a 56.8% trigger confirmation rate, effectively surfacing mission-relevant phenomena. Crucially, ego-motion conditioning suppresses 45.5% of false positives relative to an uncompensated surprise signal baseline. In a replay-side Pareto ablation study, DINO-Explorer robustly dominates the validated peak F1 versus telemetry bandwidth frontier, reducing telemetry bandwidth by 48.2% at the selected operating point while maintaining a 62.2% peak F1 score, successfully concentrating data transmission around human-verified novelty events.

Figures & tables

Fig. 1: Conceptual overview of DINO-Explorer: Inspired by predictive coding, the framework generates an intrinsic surprise signal that acts as an indicator of semantic novelty to serve active discovery. Downstream tasks, such as detecting scientifically relevant events (e.g., biological anomalies or habitat transitions), can then be driven by this signal. Furthermore, to mitigate false positives caused by the AUV’s own motion, the system isolates genuine environmental changes by implementing an efference copy-inspired ego-motion compensation module.

Fig. 2: System architecture of DINO-Explorer: The pipeline comprises Feature Extraction (left/yellow), which produces actual semantic state $\mathbf{z}_{t}$ , history semantic states $\mathbf{Z}_{t-L:t-1}$ and RAFT-based motion estimates $\mathbf{M}_{t-L:t-1}$ , and Surprise Signal Generation (right/pink). Within a predictive coding framework, the GRU-based ego motion compensated semantic predictor ( $\mathcal{P}_{comp,\boldsymbol{\theta}}$ ) generates an internal expectation $\hat{\mathbf{z}}_{t}$ conditioned on the motion-aware temporal context $\mathbf{C}_{t-L:t-1}$ , which acts as an efference copy. The resulting surprise signal $\mathcal{S}_{t}$ , quantifying the mismatch between sensory reality $\mathbf{z}_{t}$ and internal expectation $\hat{\mathbf{z}}_{t}$ , is used both to drive continuous online adaptation via self-supervised backpropagation and to provide a real-time novelty indicator for downstream active discovery tasks.

Fig. 3: Qualitative analysis of the semantic surprise signal across a continuous sequence. The plot illustrates the efficacy of our predictive coding framework in isolating genuine novelty: (left/red) DINO-Explorer detects significant deviations from the predicted semantic path caused by a fish rapidly changing its trajectory, generating a valid surprise peak that is correctly preserved after ego-motion compensation; and (right/blue) the efference copy-inspired module effectively filters out predictable reafferent noise induced by an AUV maneuver, suppressing a “naive” false alarm to ensure the autonomous trigger remains robustly focused on scientifically relevant phenomena.

Panel of Figure 4 — Fig. 4: Ablation benchmark for the proposed DINO-Explorer under matched telemetry budgets. Sweeping $\alpha$ avoids tying the comparison to a single downstream event-extraction hyperparameter and shows the ablation benchmark across extraction sensitivities. Each panel is a budget-controlled frontier: at the same bandwidth saving, a higher curve indicates stronger proposal quality or better preservation of the corresponding diagnostic. The sweep compares DINO-Explorer with the uncompensated surprise signal, Direct $\Delta\mathbf{z}_{t}$ , and uniform replay, showing that the ego-motion-compensated surprise signal improves human-reviewer-facing proposal quality while preserving the Phase 1 recall, proposal-confirmation, and DINO-latent semantic-change diagnostics needed for underwater triage.

Setting	SPCR (%)		TCR (%)		FPSR (%)	DR (%)
	P1	P1+P2	P1	P1+P2
$\alpha=2.5$	47.5	78.8	11.8	56.8	45.5	147.8

TABLE I: Operating-point consensus summary. Phase 1 (P1) uses blind same-category consensus intervals; P1+P2 additionally counts reviewer-confirmed model-guided discoveries. FPSR compares against the uncompensated predictive variant, and DR reports additional event coverage with model discovering.

Method	Setting	BSR (%)	Peak F1 (%)	SPCR P1 (%)	TCR P1 (%)	LER (%)
$\mathcal{S}_{comp}$	$\alpha=2.5$	48.2	62.2	47.5	11.8	70.8
$\mathcal{S}_{uncomp}$	$\alpha=2.5$	47.6	46.4	46.3	12.0	68.2
Direct diff ( $\Delta\mathbf{z}_{t}$ )	$\alpha=2.5$	45.9	59.5	48.2	11.3	71.2
Uniform periodic	$\Delta t=12$ s	48.0	19.5	26.5	8.6	53.0

TABLE II: Operating-point replay comparison. Peak F1 is measured against reviewer-confirmed proposal peaks; BSR , SPCR / TCR , and LER report bandwidth, recall/precision diagnostics, and latent-energy retention. Bold values mark the primary DINO-Explorer operating-point readouts.

Metric summary	Value
Avg / median / p95 latency (ms)	227.2 / 228.7 / 272.9
Average FPS	4.40

TABLE III: Runtime context for asynchronous mission-time triage on an RTX A6000.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Fig. 5: Qualitative surprise-event examples. Rows show the compensated surprise trace for a habitat transition, turbidity plume, and illumination shift, illustrating low response to predictable underwater noise and peaks on event-level semantic changes.

Active Visual Perception

Active visual perception studies how agents choose where, when, and how to look to gather task-relevant evidence efficiently.

See all 48 papers

Autonomous Search for Sparsely Distributed Visual Phenomena through Environmental Context Modeling
Mar 10, 2026Eric Chen, Travis Manderson, Nare Karapetyan +3Underwater Robotics Active Visual Perception
Massachusetts Institute of Technology and Woods Hole Oceanographic Institution · Massachusetts Institute of Technology · Woods Hole Oceanographic Institution +1
Aquatic Neuromorphic Optical Flow
May 8, 2026Pei Zhang, Yunkai Liang, Kaiqiang WangOptical Flow Event-Based Vision
School of Electrical Engineering, Guangxi University · Baise Artificial Intelligence Innovation and Development Center · School of Physical Science and Technology, Northwestern Polytechnical University
CORAL: COntextual Reasoning And Local Planning in A Hierarchical VLM Framework for Underwater Monitoring
Mar 16, 2026Zhenqi Wu, Yuanjie Lu, Xuesu Xiao +1Autonomous Underwater Navigation Vision-Language Navigation
University of South Florida · George Mason University

Mar 10, 2026J/K move · Enter open · S save

Autonomous Search for Sparsely Distributed Visual Phenomena through Environmental Context Modeling

Eric Chen, Travis Manderson, Nare Karapetyan, Peter Edmunds+2

Massachusetts Institute of Technology and Woods Hole Oceanographic Institution · Massachusetts Institute of Technology · Woods Hole Oceanographic Institution+1

Underwater Robotics Active Visual Perception

Open paper AskPDF arXiv

Autonomous underwater vehicles (AUVs) are increasingly used to survey coral reefs, yet efficiently locating specific coral species of interest remains difficult: target species are often sparsely distributed across the reef, and an AUV with limited battery life cannot afford to search everywhere. When detections of the target itself are too sparse to provide directional guidance, the robot benefits from an additional signal to decide where to look next. We propose using the visual environmental context -- the habitat features that tend to co-occur with a target species -- as that signal. Because context features are spatially denser and often vary more smoothly than target detections, we hypothesize that a reward function targeted at broader environmental context will enable adaptive planners to make better decisions on where to go next, even in regions where no target has yet been observed. Starting from a single labeled image, our method uses patch-level DINOv2 embeddings to perform one-shot detections of both the target species and its surrounding context online. We validate our approach using real imagery collected by an AUV at two reef sites in St. John, U.S. Virgin Islands, simulating the robot's motion offline. Our results demonstrate that one-shot detection combined with adaptive context modeling enables efficient autonomous surveying, sampling up to 75

\%

of the target in roughly half the time required by exhaustive coverage when the target is sparsely distributed, and outperforming search strategies that only use target detections.

Abstract

Active Visual Perception

Explore similar work

Autonomous Search for Sparsely Distributed Visual Phenomena through Environmental Context Modeling

Aquatic Neuromorphic Optical Flow

CORAL: COntextual Reasoning And Local Planning in A Hierarchical VLM Framework for Underwater Monitoring