Moving Object Detection from Moving Camera Using Focus of Expansion Likelihood and Segmentation

Masahiro Ogawa; Qi An; Atsushi Yamashita

cs.CVJul 18, 2025

Moving Object Detection from Moving Camera Using Focus of Expansion Likelihood and Segmentation

Authors: Masahiro Ogawa, Qi An, Atsushi Yamashita

Organizations: Department of Precision Engineering, Graduate School of Engineering, The University of Tokyo, Japan · Department of Human and Engineered Environmental Studies, Graduate School of Frontier Sciences, The University of Tokyo, Japan

Abstract

Separating moving and static objects from a moving camera viewpoint is essential for 3D reconstruction, autonomous navigation, and scene understanding in robotics. Existing approaches often rely primarily on optical flow, which struggle to detect moving objects in complex, structured scenes involving camera motion. To address this limitation, we propose Focus of Expansion Likelihood and Segmentation (FoELS), a method based on the core idea of integrating both optical flow and texture information. FoELS computes the focus of expansion (FoE) from optical flow and derives an initial motion likelihood from the outliers of the FoE computation. This likelihood is then fused with a segmentation-based prior to estimate the final moving probability. The method effectively handles challenges including complex structured scenes, rotational camera motion, and parallel motion. Comprehensive evaluations on the DAVIS 2016 and FBMS-59 datasets, along with real-world traffic videos including parallel, cross-direction, opposite-direction, and crowded scenes, demonstrate its effectiveness and state-of-the-art performance.

Figures & tables

Figure 1 : Sample result of FoELS. It detects moving objects from a moving camera at various distances within the scene.

Figure 2 : Detailed flowchart of the proposed method. The side image illustrates the process outlined in the flowchart.

Method	Stop	Go Forward	Rotate	Go Forward and Rotate	Textureless object	Close object	Close dominant object
Flow Orientation [ 1 ]	✓	$\times$	$\times$	$\times$	$\times$	✓	$\times$
FoE [ 4 ]	✓	✓	$\times$	$\times$	$\times$	✓	$\times$
AdversarialNet [ 3 ]	✓	✓	$\triangle$	$\triangle$	$\times$	$\times$	$\times$
FoELS (Ours)	✓	✓	$\triangle$	$\triangle$	✓	✓	$\times$
	by Orientation	by FoE			by Seg	by FoE

Table 1 : Comparison of tractable scenes.

\times

: Not tractable,

\triangle

: Partially tractable, ✓: Tractable The possible reasons for tractability are listed in the bottom row for FoELS.

Method	DAVIS 2016	FBMS 59
Adversarial Net	0.599	0.369
FoELS (Ours)	0.773	0.695

Table 2 : Quantitative evaluation result. The values represent the average IoU scores over the DAVIS 2016 train-val-movobj sequences and FBMS-59 Testset scenes.

Figure 3 : Example visual results of FoELS on the DAVIS 2016 bear scene. First row (left to right): (a) Input frame, (b) segmentation result, and (c) prior moving probability derived from segmentation. Second row (left to right): (d) Optical flow, (e) optical flow with FoE inlier (green arrows) and outliers (red arrows), and (f) the FoE-based moving likelihood. Third row (left to right): (g) Joint moving pixel probability, (h) refined object-level moving mask, and (i) the final moving object result.

Panel of Figure 4 — Figure 4 : Comparison results with AdversarialNet (left) and FoELS (right) across different scenarios. AdversarialNet exhibits limited generalization to unseen scenes, while FoELS maintains robust performance without scene-specific tuning. The dramatic visual improvement reflects the difference between real-world complexity and standard datasets.

Panel of Figure 5 — Figure 5 : Example visual results of FoELS on various motion types including parallel, opposite-direction, cross-direction, and crowded scenes. See Fig. 3 for the 9-subimage format.

Method	IoU
InternImageT (Semantic)	0.532
Oneformer (Panoptic) with ObjRefine	0.65
+ FoE sign (=FoELS)	0.773

Table 3 : Ablation study results. The values represent the average IoU scores over the DAVIS 2016 train-val-movobj sequences. "OneFormer with ObjRefine" refers to the OneFormer model for panoptic segmentation with object refinement. "+ FoE sign" indicates the addition of the FoE sign, representing the final FoELS configuration.

Optical Flow Method	IoU
MemFlow	0.736
UniMatch (FoELS)	0.773

Table 4 : Comparison of optical flow methods. The values represent the average IoU scores of FoELS over the DAVIS 2016 train-val-movobj sequences using different optical flow methods.

Formula	$F_{l}$	Macro-mean IoU	Std
diff_log (FoELS)	$\|\log_{10}(1+\|\Delta\ell\|)\|$	0.773	0.183
diff_lin	$\|\Delta\ell\|$	0.764	0.195
ratio_log	$\|\log_{10}(d_{l}^{\text{ratio}})\|$	0.765	0.186
ratio_lin	$\|d_{l}^{\text{ratio}}-1\|$	0.769	0.182

Table 5 : Ablation of the length-factor formula

F_{l}

in Eq. ( 4 ) on the DAVIS 2016 train-val-movobj benchmark (47 sequences, 3,508 frames), with

\alpha=0.25

and all other parameters fixed. Let

\Delta\ell=\|\mathbf{v}_{P}\|-\overline{\|\mathbf{v}_{P,\text{static}}\|}

and

d_{l}^{\text{ratio}}=\|\mathbf{v}_{P}\|/\overline{\|\mathbf{v}_{P,\text{static}}\|}

. diff_log is the formula used in the paper (Eq. ( 4 )).

$\alpha$	Macro-mean IoU	Std
0.00	0.746	0.184
0.25 (FoELS)	0.773	0.184
0.50	0.767	0.188

Table 6 : Sensitivity sweep over the length-factor weight

\alpha

in Eq. ( 2 ) on the DAVIS 2016 train-val-movobj benchmark (47 sequences, 3,508 frames). Macro-mean IoU is computed by averaging per-sequence mean IoUs across the 47 sequences; the between-sequence standard deviation is reported alongside.

\alpha=0.25

is the value used in the paper.

Explore similar work

Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos
Mar 12, 2026Shuo Sun, Unal Artan, Malcolm Mielle +2Local Camera Pose Scene Reconstruction
Örebro University, Sweden · Schindler EPFL Lab, Lausanne, Switzerland · Technical University of Munich, Germany
GMOS: Grounding Moving Object Segmentation in 3D Space and Time
May 28, 2026Junyu Xie, Tengda Han, Weidi Xie +1Video Object Segmentation Multi-Object Tracking
Visual Geometry Group, Department of Engineering Science, University of Oxford, UK · SAI, Shanghai Jiao Tong University, China
PaMoSplat: Part-Aware Motion-Guided Gaussian Splatting for Dynamic Scene Reconstruction
May 11, 2026Yinan Deng, Jianyu Dou, Jiahui Wang +3Scene Reconstruction Gaussian Splatting
School of Automation, Beijing Institute of Technology, Beijing 100081, China

Mar 12, 2026, cs.CVJ/K move · Enter open · S save

Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos

Shuo Sun, Unal Artan, Malcolm Mielle, Achim J. Lilienthal+1

Örebro University, Sweden · Schindler EPFL Lab, Lausanne, Switzerland · Technical University of Munich, Germany

Local Camera Pose Scene Reconstruction Camera Simultaneous Localization And Mapping Scene Understanding

Open paper AskPDF arXiv

We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple freely moving cameras -- a setting that arises naturally when multiple observers capture a shared event. Prior approaches either handle only single-camera input or require rigidly mounted, pre-calibrated camera rigs, limiting their practical applicability. We propose a two-stage optimization framework that decouples the task into robust camera tracking and dense depth refinement. In the first stage, we extend single-camera visual SLAM to the multi-camera setting by constructing a spatiotemporal connection graph that exploits both intra-camera temporal continuity and inter-camera spatial overlap, enabling consistent scale and robust tracking. To ensure robustness under limited overlap, we introduce a wide-baseline initialization strategy using feed-forward reconstruction models. In the second stage, we refine depth and camera poses by optimizing dense inter- and intra-camera consistency using wide-baseline optical flow. Additionally, we introduce MultiCamRobolab, a new real-world dataset with ground-truth poses from a motion capture system. Finally, we demonstrate that our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks, while requiring less memory.

Abstract

Explore similar work

Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos

GMOS: Grounding Moving Object Segmentation in 3D Space and Time

PaMoSplat: Part-Aware Motion-Guided Gaussian Splatting for Dynamic Scene Reconstruction