OWL: A Novel Approach to Machine Perception During Motion

Daniel Raviv; Juan D. Yepes

cs.CVMar 5, 2026

OWL: A Novel Approach to Machine Perception During Motion

Authors: Daniel Raviv, Juan D. Yepes

Organizations: Electrical Engineering and Computer Science Department Florida Atlantic University Boca Raton, Florida, USA

Abstract

We introduce a perception-related function, OWL, designed to address the complex challenges of 3D perception during motion. It derives its values directly from two fundamental visual motion cues, with one set of cue values per point per time instant. During motion, two visual motion cues relative to a fixation point emerge: 1) perceived local visual looming of points near the fixation point, and 2) perceived rotation of the rigid object relative to the fixation point. It also expresses the relation between two well-known physical quantities, the relative instantaneous directional range and directional translation in 3D between the camera and any visible 3D point, without explicitly requiring their measurement or prior knowledge of their individual values. OWL offers a unified, analytical time-based approach that enhances and simplifies key perception capabilities, including scaled 3D mapping and camera heading. Simulations demonstrate that OWL achieves geometric constancy of 3D objects over time and enables scaled 3D scene reconstruction from visual motion cues alone. By leveraging direct measurements from raw visual motion image sequences, OWL values can be obtained without prior knowledge of stationary environments, moving objects, or camera motion. This approach employs minimalistic, pixel-based, parallel computations, providing an alternative real-time representation for 3D points in relative motion. OWL bridges the gap between theoretical concepts and practical applications in robotics and autonomous navigation and may unlock new possibilities for real-time decision-making and interaction, potentially serving as a building block for next-generation autonomous systems. This paper offers an alternative perspective on machine perception, with implications that may extend to natural perception and contribute to a better understanding of behavioral psychology and neural functionality.

Figures & tables

Fig. 1: Motion relative to a 3D fixation point located on a rigid object

Fig. 2: Fixating on a point $F$ of a moving rigid body at two time instants

Fig. 3: Relative translation decomposed into two orthogonal components

Fig. 4: Complex number representation of $\tilde{t}$ , $\tilde{r}$ and $\tilde{t}/\tilde{r}$

Fig. 6: Effect of speed on the complex value $\tilde{t}/\tilde{r}$ . Increasing $|\tilde{t}|$ while keeping the motion direction constant shifts the point radially in the $\tilde{t}/\tilde{r}$ plane, scaling both $L$ and $\omega$ .

Fig. 7: Equal time-to-contact ( TTC ) planes and equal looming ( $L$ ) spheres

Fig. 8: From $\tilde{t}$ / $\tilde{r}$ lines to $OWL=\tilde{r}/\tilde{t}$ circles

Fig. 9: Rotating OWL domain for better intuition

Fig. 10: Visualizing an equal looming sphere and an equal $\omega$ torus

Fig. 11: Multiple points producing identical OWL values

Fig. 16: Reconstruction of a rigid object from OWL (RoT) values over time as the camera moves

Fig. 17: OWL-based 3D reconstruction. (a) RGB image. (b) Looming ( $L$ ). (c)-(d) Rotation components ( $\omega$ ). (e) Scaled 3D point cloud (RoT).

Fig. 18: Coordinate system and directional unit vectors

3D Scene Representation

3D scene representation studies how to encode geometry, appearance, semantics, and motion for reconstruction, rendering, generation, and interaction.

See all 242 papers

Oct 24, 2026

Segmentation of Moving Objects During Camera Motion Using the OWL Function

We present a principled, analytical framework for segmenting multiple independently moving rigid objects during single camera motion, built directly upon the OWL perception function. OWL encodes two fundamental visual motion cues, perceived looming

L

and perceived rotation

ω

, into a single closed-form representation of relative 3D structure. A key consequence of this structure is that the unit direction of relative translation

\hat{t}

is analytically recoverable per image point from

L

and

ω

alone. All points belonging to a non-rotating rigid body share the same

\hat{t}

, whereas points on different rigid bodies generally yield distinct values. This furnishes a natural segmentation criterion: clustering per-point

\hat{t}

estimates partitions the visible scene into independently moving rigid bodies. Building on the 2D complex form of OWL, we derive the framework in 3D using quaternions, for a single moving camera. The approach is parallel, camera rotation-invariant, and requires no training data, no depth information, and no semantic priors.

Daniel Raviv, Juan D. Yepes

Electrical Engineering and Computer Science Department, Florida Atlantic University, Boca Raton, Florida, USA

Motion Segmentation

Ask PDF

Apr 24, 2026

GenMatter: Perceiving Physical Objects with Generative Matter Models

Human visual perception offers valuable insights for understanding computational principles of motion-based scene interpretation. Humans robustly detect and segment moving entities that constitute independently moveable chunks of matter, whether observing sparse moving dots, textured surfaces, or naturalistic scenes. In contrast, existing computer vision systems lack a unified approach that works across these diverse settings. Inspired by principles of human perception, we propose a generative model that hierarchically groups low-level motion cues and high-level appearance features into particles (small Gaussians representing local matter), and groups particles into clusters capturing coherently and independently moveable physical entities. We develop a hardware-accelerated inference algorithm based on parallelized block Gibbs sampling to recover stable particle motion and groupings. Our model operates on different kinds of inputs (random dots, stylized textures, or naturalistic RGB video), enabling it to work across settings where biological vision succeeds but existing computer vision approaches do not. We validate this unified framework across three domains: on 2D random dot kinematograms, our approach captures human object perception including graded uncertainty across ambiguous conditions; on a Gestalt-inspired dataset of camouflaged rotating objects, our approach recovers correct 3D structure from motion and thereby accurate 2D object segmentation; and on naturalistic RGB videos, our model tracks the moving 3D matter that makes up deforming objects, enabling robust object-level scene understanding. This work thus establishes a general framework for motion-based perception grounded in principles of human vision.

Eric Li, Arijit Dasgupta, Yoni Friedman +5

1MIT CSAIL · 2MIT BCS

Video Object Segmentation Motion Segmentation Object-Centric Representation Learning Structure from Motion 3D Scene Understanding

Ask PDF

Jul 17, 2026

Vision-Language-Motion Maps: An Open-Vocabulary, Uncertainty-Aware, Queryable Motion Attribute for 3D Scene Maps

Open-vocabulary 3D maps let robots answer language queries about what and where, but they assume a static world and cannot answer queries about how scene elements behave. We introduce Vision-Language-Motion Maps (VLMM), an open-vocabulary, language-queryable 3D map - queried through a rule-based intent router over open-vocabulary object nouns, not a general natural-language interface - in which each element carries a fused motion attribute: a VLM/LLM semantic movability prior combined with geometrically observed cross-frame motion, together with a per-element uncertainty. Queries reduce to attribute filters that distinguish what has been seen to move, what could move but has not, and what stays still. On a controlled simulator benchmark with exact ground truth (AI2-THOR, three scene types) we show through ablation that the schema fields are non-substitutable: a semantic-only baseline fails motion queries even with strong features, and neither motion field substitutes for the other (the prior cannot answer "what is moving," observed motion cannot answer "what could move"). On real dynamic RGB-D (TUM and Bonn, six sequences) we show the uncertainty channel - our key difference from prior fused-motion work - consistently improves moving-vs-static average precision and reduces false motion flags, and that it is robust to estimated (noisy) poses. The raw confidence is not calibrated, but post-hoc isotonic calibration reaches an expected calibration error of 0.10. VLMM is a representation contribution: the closest prior maps each lack at least one of the four properties - open-vocabulary, language-queryable, fused prior-and-observed motion, and per-element uncertainty - that our combination provides.

Dibyendu Ghosh, Ayushi Shakya

Rekise Marine, Bangalore, India

Uncertainty Quantification 3D Scene Representation Robotic Perception 3D Mapping 3D Semantic Mapping

Ask PDF

Abstract

3D Scene Representation

Explore similar work

Segmentation of Moving Objects During Camera Motion Using the OWL Function

GenMatter: Perceiving Physical Objects with Generative Matter Models

Vision-Language-Motion Maps: An Open-Vocabulary, Uncertainty-Aware, Queryable Motion Attribute for 3D Scene Maps