GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry

Xiankang He; Peile Lin; Ying Cui; Dongyan Guo; Chunhua Shen; Xiaoqin Zhang

cs.CVFeb 25, 2026

GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry

Authors: Xiankang He, Peile Lin, Ying Cui, Dongyan Guo, Chunhua Shen, Xiaoqin Zhang

Organizations: College of Computer Science and Technology, Zhejiang University of Technology · Zhejiang Key Laboratory of Visual Information Intelligent Processing · State Key Lab of CAD & CG, Zhejiang University

Abstract

Motion segmentation in dynamic scenes is highly challenging, as conventional methods heavily rely on estimating camera poses and point correspondences from inherently noisy motion cues. Existing statistical inference or iterative optimization techniques that struggle to mitigate the cumulative errors in multi-stage pipelines often lead to limited performance or high computational cost. In contrast, we propose a fully learning-based approach that directly infers moving objects from latent feature representations via attention mechanisms, thus enabling end-to-end feed-forward motion segmentation. Our key insight is to bypass explicit correspondence estimation and instead let the model learn to implicitly disentangle object and camera motion. Supported by recent advances in 4D scene geometry reconstruction (e.g., $π^3$ ), the proposed method leverages reliable camera poses and rich spatial-temporal priors, which ensure stable training and robust inference for the model. Extensive experiments demonstrate that by eliminating complex pre-processing and iterative refinement, our approach achieves state-of-the-art motion segmentation performance with high efficiency. The code is available at:https://github.com/zjutcvg/GeoMotion.

Figures & tables

Figure 1: Overview of GeoMotion. Given an input video, our framework integrates 4D geometric priors from a pretrained reconstruction model ( $\pi^{3}$ ) and local pixel-level motion from optical flow to infer dynamic object masks. By leveraging 4D geometric priors, the proposed GeoMotion disentangles object motion from camera motion in a single feed-forward manner.

Figure 2: Architecture of the proposed GeoMotion framework. The model comprises a feature aggregation module and a motion decoder. The former fuses latent 4D features, optical flow features, and camera pose embeddings, while the latter employs multi-head self-attention to decode motion masks. The design enables end-to-end feed-forward motion segmentation without iterative refinement.

Figure 3: Visualization of $\pi^{3}$ features across alternating attention layers. Shallow layers preserve semantic object-level features, whereas deeper layers encode high-level global geometry. Their fusion yields robust latent 4D representations that support accurate motion segmentation.

Methods	Iterative optimization	DAVIS2016-M			DAVIS2017	FBMS-59		DAVIS2016			SegTrackV2	Run Time per frame
Methods	Iterative optimization	$\mathcal{J\&F}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$	$\mathcal{J\&F}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$	$\mathcal{J}\uparrow$	Run Time per frame
OCLR-TTA [ 44 ]	Yes	78.5	80.2	76.9	76.0	69.9	68.3	78.8	80.8	76.8	72.3	1.25s
RoMo [ 7 ]	Yes	-	-	-	-	75.5	-	-	77.3	-	67.7	8.34s
SegAnyMotion [ 11 ]	Yes	89.5	89.2	89.7	90.0	78.3	82.8	90.9	90.6	91.0	76.3	6.44s
CIS [ 50 ]	No	66.2	67.6	64.8	-	63.6	-	68.6	70.3	66.8	62.0	-
EM [ 25 ]	No	75.2	76.2	74.3	55.5	57.9	56.0	70.0	69.3	70.7	55.5	-

Table 1: Quantitative comparison with motion segmentation methods on popular benchmarks. The proposed model obtains state-of-the-art performance. It achieves an excellent trade-off between segmentation quality and computational efficiency.

Figure 4: Qualitative comparison on multiple benchmarks. Visual comparison with state-of-the-art methods including OCLR-Flow [ 44 ] , SegAnyMotion [ 11 ] , and RoMo [ 7 ] . The proposed method produces geometrically complete and visually coherent motion masks, preserving fine object details and boundaries under complex scenes.

Method	Flow	DAVIS2016		DAVIS2017		DAVIS-All
Method	Flow	$\mathcal{J}_{M}\uparrow$	$\mathcal{J}_{R}\uparrow$	$\mathcal{J}_{M}\uparrow$	$\mathcal{J}_{R}\uparrow$	$\mathcal{J}_{M}\uparrow$	$\mathcal{J}_{R}\uparrow$
DUSt3R [ 40 ]	✓	58.5	63.4	48.7	50.2	47.6	48.7
MonST3R [ 52 ]	✓	64.3	73.3	56.4	59.6	51.9	54.1
DAS3R [ 47 ]	✗	54.2	55.8	57.4	61.3	53.9	54.8
Easi3R ${}_{\text{dust3r}}$ [ 3 ]	✗	67.9	71.4	60.1	65.3	54.7	60.6
Easi3R ${}_{\text{monst3r}}$ [ 3 ]	✗	70.7	79.9	67.9	76.1	63.1	72.6

Table 2: Comparison with 3D/4D reconstruction-based methods. The best and second-best results are labeled in bold and underlined , respectively.

Methods	Cam	Flow	Sh.	DAVIS2017
Methods	Cam	Flow	Sh.	$\mathcal{J}\!\&\!\mathcal{F}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$
Baseline	✗	✗	✗	67.9	68.3	67.6
+ Cam	✓	✗	✗	74.2	74.2	74.3
+ Flow	✗	✓	✗	74.7	74.8	74.7
+ Shallow	✗	✗	✓	72.4	71.9	72.9
+ Cam + Shallow	✓	✗	✓	76.0	76.1	75.9

Table 3: Ablation study on feature aggregation. Adding the camera pose, optical flow, and shallow-layer features progressively enhances performance on DAVIS2017. The full model, which combines all three modalities, achieves the best overall accuracy, validating the effectiveness of spatio-temporal feature fusion.

Training Datasets	#D.	DAVIS2016-M		DAVIS2017
Training Datasets	#D.	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$	$\mathcal{J}\uparrow$	$\mathcal{F}\uparrow$
+ HOI4D [ 22 ]	1	67.9	68.8	65.4	66.5
+ Dynamic Replica [ 14 ]	2	75.1	76.2	73.8	74.3
+ OmniWorld-motion	3	77.8	77.6	76.1	75.6
+ YTVOS18-m [ 46 ]	4	80.6	80.8	79.3	79.8
+ GOT-Motion	5	83.5	84.3	81.1	81.8

Table 4: Ablation study on dataset scale. Training on progressively larger and more diverse datasets consistently improves segmentation performance, demonstrating the strong scalability and generalization ability of the proposed method.

Figure 5: Initialization comparison for the motion decoder. Initializing with

\pi^{3}

pretrained parameters yields faster convergence and higher IoU compared to random initialization, demonstrating the benefit of large scale geometry pretraining.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	JM $\uparrow$		JR $\uparrow$		FM $\uparrow$
	w/o SAM	w/ SAM	w/o SAM	w/ SAM	w/o SAM	w/ SAM
Easi3R ${}_{\text{dust3r}}$ [ 3 ]	46.86	60.1	50.54	65.3	39.06	–
Easi3R ${}_{\text{monst3r}}$ [ 3 ]	54.75	67.9	66.16	76.1	44.09	–
MonST3R [ 52 ]	38.07	56.4	36.05	59.6	48.24	–
DAS3R [ 47 ]	44.51	57.4	43.95	61.3	46.71	–
VGGT4D [ 9 ]	56.45	–	65.62	–	51.09	–

Appendix

Table 5: Ablation for SAM2 on DAVIS-2017.

Figure 6: Qualitative ablation results for SAM2.

Figure 7: Compare with Easi3R and VGGT4D without SAM2.

Figure 8: More visual examples of dynamic masks predicted by GeoMotion on the DAVIS benchmark. Odd rows show the RGB input frames, while even rows present the corresponding predicted dynamic masks.

Motion Segmentation

See all 13 papers

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors
Apr 10, 2026Ying Zang, Yidong Han, Chaotao Ding +8UQ for 3D Reconstruction Dynamic Scene Reconstruction
Huzhou University · KOKONI 3D, Moxin Technology · Univeristy of Science and Technology of China +3
4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation
May 12, 2026Ying Zang, Xuanyi Liu, Yidong Han +94D Reconstruction Depth Estimation
4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4
DynEoMT: Learning Object Dynamicity from Online Segmentation Queries
Sep 13, 2026Calvin Galagain, Martyna Poreba, François GouletteVideo Object Segmentation Video Instance Segmentation
Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · U2IS, ENSTA Paris, Institut Polytechnique de Paris, 91120, Palaiseau, France

Apr 10, 2026J/K move · Enter open · S save

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors

Ying Zang, Yidong Han, Chaotao Ding, Yuanqi Hu+7

Huzhou University · KOKONI 3D, Moxin Technology · Univeristy of Science and Technology of China+3

UQ for 3D Reconstruction Dynamic Scene Reconstruction 4D Reconstruction 3D Reconstruction Multi-View 3D Reconstruction 3D ViTs+1

Open paper AskPDF arXiv

Reconstructing dynamic 4D scenes is an important yet challenging task. While 3D foundation models like VGGT excel in static settings, they often struggle with dynamic sequences where motion causes significant geometric ambiguity. To address this, we present a framework designed to disentangle dynamic and static components by modeling uncertainty across different stages of the reconstruction process. Our approach introduces three synergistic mechanisms: (1) Entropy-Guided Subspace Projection, which leverages information-theoretic weighting to adaptively aggregate multi-head attention distributions, effectively isolating dynamic motion cues from semantic noise; (2) Local-Consistency Driven Geometry Purification, which enforces spatial continuity via radius-based neighborhood constraints to eliminate structural outliers; and (3) Uncertainty-Aware Cross-View Consistency, which formulates multi-view projection refinement as a heteroscedastic maximum likelihood estimation problem, utilizing depth confidence as a probabilistic weight. Experiments on dynamic benchmarks show that our approach outperforms current state-of-the-art methods, reducing Mean Accuracy error by 13.43% and improving segmentation F-measure by 10.49%. Our framework maintains the efficiency of feed-forward inference and requires no task-specific fine-tuning or per-scene optimization.

Abstract

Motion Segmentation

Explore similar work

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation

DynEoMT: Learning Object Dynamicity from Online Segmentation Queries