We present a principled, analytical framework for segmenting multiple independently moving rigid objects during single camera motion, built directly upon the OWL perception function. OWL encodes two fundamental visual motion cues, perceived looming L and perceived rotation ω, into a single closed-form representation of relative 3D structure. A key consequence of this structure is that the unit direction of relative translation t^ is analytically recoverable per image point from L and ω alone. All points belonging to a non-rotating rigid body share the same t^, whereas points on different rigid bodies generally yield distinct values. This furnishes a natural segmentation criterion: clustering per-point t^ estimates partitions the visible scene into independently moving rigid bodies. Building on the 2D complex form of OWL, we derive the framework in 3D using quaternions, for a single moving camera. The approach is parallel, camera rotation-invariant, and requires no training data, no depth information, and no semantic priors.
Figures & tables
Known per point
Constraints
Points needed
L and ω (vector)
2 (angle + plane)
1
L and ω (scalars)
1 (angle only)
3
TABLE I: Points Needed to Recover t^
Fig. 1: Camera translating through a static environment (with no camera rotation). Top row: a top-down schematic of the camera motion and a representative rendered frame. Middle row, left to right: the ground-truth cues L , ωθ , and ωϕ . Bottom row: the recovered t^ field visualized as RGB color. Since the scene is a single rigid body undergoing pure camera-relative translation, t^ collapses to a single color (red) everywhere, as predicted in Section III-C .
Fig. 2: Three viewpoints of the RoT point cloud reconstructed for the stationary scene of Fig. 1 , confirming that the recovered scaled 3D geometry is consistent with the underlying structure of the 3D environment.
Fig. 3: Case of translation-only motion. Top row: a representative frame and a top-down schematic showing two independently translating objects. Middle row: the corresponding L , ωϕ , and ωθ cues, in which each object is visible as a localized deviation from the background. Bottom row: the recovered t^ field, in which each object forms its own uniformly colored cluster distinct from the background, and the corresponding binary object masks. With no self-rotation on either object, each yields a single well-defined t^ , cleanly separable by clustering.
Fig. 4: Two independently moving objects in a degenerate configuration. Top: a top-down schematic of the camera and object motions. Middle: a representative frame, with the robot moving toward the camera boxed. Bottom: the recovered t^ field. The boxed robot moves exactly opposite to the camera, so its t^ coincides with that of the background and it does not appear as a separate cluster (Assumption 3). The second robot forms a distinct cluster.
Fig. 5: Self-rotating object alongside a purely translating object. Top: a top-down schematic marking the self-rotating object and a representative frame. Bottom, from left to right: the recovered t^ field and binary masks. The purely translating object still yields one uniform t^ color, while the self-rotating object’s cluster shows internal color variation, reflecting the local violation of the pure-translation assumption on that body.
Fig. 6: Camera rotating, while translating forward, over a stationary scene. Top: a top-down schematic of the camera’s rotation. Bottom: two frames (Frame 4 and Frame 19) sampled during the rotation, each with its recovered t^ field shown below it. In both frames the stationary background still collapses to a single, uniform t^ color; the color differs between frames because t^ is expressed in the camera’s own frame, which has rotated. Camera rotation does not break the single-cluster structure of the stationary background.
Fig. 7: Camera rotating and translating, with moving objects translating only. Top: a top-down schematic of the camera and object motions. Bottom: two frames (Frame 610 and Frame 640) with their recovered t^ fields. The two moving objects remain visible as distinct, uniformly colored clusters against the background in both frames, indicating that simultaneous camera rotation and translation does not disrupt the recovery of per-object t^ .
Fig. 8: Camera rotating and translating, with two moving objects, the one on the right is translating only and the one on the left rotates only. Top: a top-down schematic of the combined motions at an earlier frame than Frame 322. Bottom: two frames (Frame 322 and Frame 335) with their recovered t^ fields. As in Fig. 5 , the self-rotation of the objects introduces internal color variation within their clusters, now superimposed on the effects of camera rotation and translation.
Fig. 9: A more general case: multiple independently moving objects (two cars, a walking humanoid robot, a delivery robot, and a pedestrian) together with camera motion. Top: a representative frame. Middle: the recovered t^ field, in which each moving object stands out from the uniform background. Bottom: the corresponding binary masks of moving versus static regions. The rigid vehicles produce nearly uniform t^ , while the articulated humanoid and pedestrian, which violate the rigid-body assumption, show internal t^ variation but remain clearly distinct from the background.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Fig. 10: Geometric decomposition of t/∣r∣ in the plane containing t and e^r . The radial component Le^r (red) is read directly from the looming. The tangential component −e^r×ω (orange) is recovered by a second cross product. Their vector sum (blue) gives t/∣r∣ ; normalizing by L2+ω2 yields t^ .
We introduce a perception-related function, OWL, designed to address the complex challenges of 3D perception during motion. It derives its values directly from two fundamental visual motion cues, with one set of cue values per point per time instant. During motion, two visual motion cues relative to a fixation point emerge: 1) perceived local visual looming of points near the fixation point, and 2) perceived rotation of the rigid object relative to the fixation point. It also expresses the relation between two well-known physical quantities, the relative instantaneous directional range and directional translation in 3D between the camera and any visible 3D point, without explicitly requiring their measurement or prior knowledge of their individual values. OWL offers a unified, analytical time-based approach that enhances and simplifies key perception capabilities, including scaled 3D mapping and camera heading. Simulations demonstrate that OWL achieves geometric constancy of 3D objects over time and enables scaled 3D scene reconstruction from visual motion cues alone. By leveraging direct measurements from raw visual motion image sequences, OWL values can be obtained without prior knowledge of stationary environments, moving objects, or camera motion. This approach employs minimalistic, pixel-based, parallel computations, providing an alternative real-time representation for 3D points in relative motion. OWL bridges the gap between theoretical concepts and practical applications in robotics and autonomous navigation and may unlock new possibilities for real-time decision-making and interaction, potentially serving as a building block for next-generation autonomous systems. This paper offers an alternative perspective on machine perception, with implications that may extend to natural perception and contribute to a better understanding of behavioral psychology and neural functionality.
Daniel Raviv, Juan D. Yepes
Electrical Engineering and Computer Science Department Florida Atlantic University Boca Raton, Florida, USA
Moving object segmentation is conventionally formulated at the sequence and object level, marking an entire instance as moving once it moves anywhere in a video. Recent instantaneous formulations distinguish moving from stationary intervals, but still generate masks at the object level through frozen segmentation foundation models. Because the mask is derived from an instance-level proposal, motion is typically attributed to the entire instance rather than localized to the regions that actually move. Moreover, such instance-level predictions can produce masks even when no part of the instance is actually moving, revealing a more fundamental issue: motion is not an intrinsic property of an instance; it can only be determined by comparing at least two frames. Building on this premise, we introduce MoRSe, which jointly interprets two frames to predict a moving-region mask, distinguishing actual scene motion from apparent motion induced by the camera. A single RGB encoder replaces multi-stage foundation stacks, so no optical flow, point tracks, or separate mask generator is needed at inference. We release MoRSe-5K, a benchmark of 42 sequences with Static, Dynamic-Static, and Dynamic settings designed to evaluate the model's ability to distinguish and localize moving and stationary regions under different camera and scene motion conditions. MoRSe outperforms both specialized moving object segmentation methods and dense 4D reconstruction methods with motion-thresholded predictions, while producing the fewest false positives on sequences where nothing moves.
Wenhua Jin, Tiancheng Chen, Sheng Tang +5
Kosmo Research · Automotive Engineering Department, Jilin University · Xiamen University +1
Rigid bodies constitute the smallest manipulable elements in the real world, and understanding how they physically interact is fundamental to embodied reasoning and robotic manipulation. Thus, accurate detection, segmentation, and tracking of moving rigid bodies is essential for enabling reasoning modules to interpret and act in diverse environments. However, current segmentation models trained on semantic grouping are limited in their ability to provide meaningful interaction-level cues for completing embodied tasks. To address this gap, we introduce MotionBit, a novel concept that, unlike prior formulations, defines the smallest unit in motion-based segmentation through kinematic spatial twist equivalence, independent of semantics. In this paper, we contribute (1) the MotionBit concept and definition, (2) a hand-labeled benchmark, called MoRiBo, for evaluating moving rigid-body segmentation across robotic manipulation and human-in-the-wild videos, and (3) a learning-free graph-based MotionBits segmentation method that outperforms state-of-the-art embodied perception methods by 37.3% in macro-averaged mIoU on the MoRiBo benchmark. Finally, we demonstrate the effectiveness of MotionBits segmentation for downstream embodied reasoning and manipulation tasks, highlighting its importance as a fundamental primitive for understanding physical interactions.
Howard H. Qian, Kejia Ren, Yu Xiang +2
Department of Computer Science Rice University · Department of Computer Science University of Texas at Dallas