All-St3R SLAM: Real-time RGB Dynamic SLAM with Pixel-wise Motion Masking

Yewon Kim; Paul Hongsuck Seo; Sang Chul Ahn

cs.CVOct 12, 2026

All-St3R SLAM: Real-time RGB Dynamic SLAM with Pixel-wise Motion Masking

Authors: Yewon Kim, Paul Hongsuck Seo, Sang Chul Ahn

Organizations: Center for Artificial Intelligence Research, Korea Institute of Science and Technology, Seoul, Korea (South) · Dept. of Computer Science and Engineering, Korea University, Seoul, Korea (South) · Center for Artificial Intelligence Research, Korea Institute of Science and Technology, and with AI-Robotics, KIST School (UST), Seoul, Korea (South)

Abstract

SLAM (Simultaneous Localization and Mapping) is a core technology in autonomous systems that enables them to estimate their own position and build a map of unknown environments simultaneously. SLAM systems in dynamic environments are of great practical importance. However, existing dynamic SLAM approaches still have several limitations like over-removal and high computational costs. To address these limitations, we propose a real-time RGB Dynamic SLAM framework that precisely removes moving objects at the pixel level without relying on predefined labels or segmentation models. Our system combines pixel motion predicted by ego-motion using DUSt3R with point tracking from All-Tracker, removing only actually moving pixels by analyzing discrepancies between the two motion sources. Unlike object-level masking methods, our approach eliminates only currently moving regions, allowing temporarily static parts to be utilized for tracking. Operating at approximately 10 FPS, the system achieves real-time performance on standard 30 FPS video streams by subsampling every 3 frames. Relying solely on RGB inputs, this significantly improves upon state-of-the-art dense dynamic SLAM approaches (0.5-3 FPS), presenting a highly practical solution.

Figures & tables

Fig. 1: Comparison between Ego-Motion , Point Tracking , and Dynamic Movement . Assuming the KeyFrame point cloud reconstructed by DUSt3R is static, projecting those 3D points into the Frame camera produces the pixel motion predicted by camera Ego-Motion . In contrast, All-Tracker predicts pixel correspondences directly between the KeyFrame and the Frame in the image domain, producing Point Tracking . When the KeyFrame point cloud contains dynamic objects, discrepancies arise between these two motions. Points with large discrepancies between the two motions are regarded as Dynamic Movement .

Fig. 2: Overview of the proposed Monocular RGB Dynamic SLAM system.

Fig. 3: Illustration of the frame-by-frame mask accumulation.

Fig. 4: Object-Level Masking on Bonn Dataset (SAM2). Visualization of dynamic masks generated by SAM2 [ 16 ] on the crowd sequence of the Bonn RGB-D Dataset. Even when motion is limited to a specific part (e.g., moving arms), the entire human body is masked out. This over-removal discards valid static regions such as the torso, significantly reducing the number of valid matching correspondences required for stable tracking.

Panel of Figure 5 — Fig. 5: Tracking Failure with Object-Level Masking. The visualization shows the system status using SAM2 masks.

Panel of Figure 6 — Fig. 7: Geometric Verification under Dynamic Occlusion. (Left: Keyframe, Right: Current Frame). Visualization of valid matches (dots) and epipolar lines. Despite large dynamic obstructions, the system maintains accurate epipolar constraints and secures a sufficient number of reliable matches, confirming stable pose estimation.

Dataset	WildGS (Full) [ 8 ]		WildGS (Fast) [ 8 ]		Ours
Dataset	FPS $\uparrow$	ATE $\downarrow$	FPS $\uparrow$	ATE $\downarrow$	FPS $\uparrow$	ATE $\downarrow$
Wild-SLAM	0.49	0.46	1.96	0.48	11.24	2.17
Bonn	0.50	2.31	2.13	2.47	10.13	2.46

TABLE I: Runtime (FPS

\uparrow

) and ATE RMSE (cm

\downarrow

Method	Balloon	Balloon2	Crowd	Crowd2	Person	Person2	Moving	Moving2	Avg.
DSO [ 17 ]	7.3	21.8	10.1	7.6	30.6	26.5	4.7	11.2	15.0
DROID-SLAM [ 18 ]	7.5	4.1	5.2	6.5	4.3	5.4	2.3	4.0	4.91
MonoGS [ 19 ]	15.3	17.3	11.3	7.3	26.4	35.2	22.2	47.2	22.8
Splat-SLAM [ 20 ]	8.8	3.0	6.8	F	4.9	25.8	1.7	3.0	-
DynaMoN (MS) [ 21 ]	6.8	3.8	6.1	5.6	2.4	3.5	1.4	2.6	4.02
DynaMoN (MS&SS) [ 21 ]	2.8	2.7	3.5	2.8	14.8	2.2	1.3	2.7	4.10

TABLE II: Tracking Performance and Ablation Study on Bonn RGB-D Dynamic Dataset (ATE RMSE [cm]). Best results are highlighted as first , second , and third . ‘F’ indicates tracking failure.

Method	f2/dp	f3/ss	f3/sx	f3/sr	f3/shs	f3/ws	f3/wr	f3/wx	f3/whs	Avg.
DSO [ 17 ]	2.2	1.7	11.5	3.7	12.4	1.5	12.9	13.8	40.7	11.1
DROID-SLAM [ 18 ]	0.6	0.5	0.9	2.2	1.4	1.2	1.6	4.0	2.2	1.62
MonoGS [ 19 ]	112.8	1.2	6.1	5.1	28.3	1.1	21.5	17.4	44.2	26.4
Splat-SLAM [ 20 ]	0.7	0.5	0.9	2.3	1.5	2.3	1.3	3.9	2.2	1.71
DynaMoN (MS) [ 21 ]	0.6	0.5	0.9	2.1	1.9	1.4	1.4	3.9	2.0	1.63
DynaMoN (MS&SS) [ 21 ]	0.7	0.5	0.9	2.4	2.3	0.7	1.4	3.9	1.9	1.63

TABLE III: Tracking Performance and Ablation Study on TUM RGB-D Dynamic Dataset (ATE RMSE [cm]). Best results are highlighted as first , second , and third . Results are marked with ’-’ when the released implementation could not be successfully reproduced.

Fig. 8: Qualitative comparison on a Static Scene. Reconstruction results on freiburg1_room . Even without the heavy matching head of MASt3R [ 25 ] , our method (combining DUSt3R [ 9 ] with All-Tracker [ 10 ] ) exhibits superior geometric alignment in the reconstructed map. This confirms that our lightweight flow integration effectively compensates for the absence of learned matching features.

Fig. 9: Mask Quality Comparison. Middle: Uncertainty map from WildGS-SLAM [ 8 ] , which is blurry due to feature downsampling. Right: Our pixel-wise dynamic mask on the Bonn dataset, showing high-resolution sharpness and better boundary adherence.

Fig. 10: KeyFrame Dynamic Masks. Visualization of dynamic masks on keyframes from freiburg3_walking_xyz . The system effectively identifies the moving person.

Fig. 11: Reconstruction on Bonn Dataset. Left: Final reconstruction of person_tracking . The moving person is completely removed, resulting in clean geometry. Right: Live tracking visualization showing the current frame (rainbow) aligned to the canonical map, with the dynamic person effectively masked out.

Fig. 12: Generalization: Quadruped Robot. Top: Clean reconstruction. Bottom: Dynamic mask. Our method successfully detects and removes the ANYmal robot without prior knowledge of the object class.

Dynamic SLAM

See all 12 papers

CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments
Oct 19, 2026Gaia Borghi, João Carlos Virgolino Soares, Angelo Bratta +2Visual SLAM
Dynamic Legged Systems Lab (DLS), Istituto Italiano di Tecnologia, Genoa, Italy · Section of Automation & Control, Department of Electronic Systems, Aalborg University, Denmark
NGD-SLAM: Towards Real-Time Dynamic SLAM without GPU
May 12, 2024Yuhao Zhang, Mihai Bujanca, Mikel LujánSimultaneous Localization and Mapping Dynamic SLAM
University of Manchester. 2University of Cambridge. · 3Qualcomm Technologies XR Labs, Austria.
DROID-SLAM in the Wild
Mar 19, 2026Moyang Li, Zihan Zhu, Marc Pollefeys +1Dynamic SLAM Visual SLAM
ETH Zurich · Microsoft

Oct 19, 2026J/K move · Enter open · S save

CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments

Gaia Borghi, João Carlos Virgolino Soares, Angelo Bratta, Shahab Heshmati-Alamdari+1

Dynamic Legged Systems Lab (DLS), Istituto Italiano di Tecnologia, Genoa, Italy · Section of Automation & Control, Department of Electronic Systems, Aalborg University, Denmark

Visual SLAM

Open paper AskPDF arXiv

Robust visual Simultaneous Localization and Mapping (vSLAM) in dynamic environments faces challenges due to the presence of moving objects that may introduce erroneous observations, corrupt the map reconstruction and overall degrade localization accuracy. Existing vSLAM approaches often rely on predefined semantic classes or costly pixel-wise classification, limiting their robustness or applicability in real-world scenarios. This paper presents CAOS-SLAM, a dynamic vSLAM framework that extends ORB-SLAM3 with a hybrid perception pipeline to detect, track, and filter both semantically known and previously unseen moving objects in real-time. On-demand panoptic segmentation is performed for object discovery, with candidate unknown objects instantiated through depth-based clustering of unclassified regions. Between panoptic updates, persistent object tracks are updated using lightweight detection when available or temporal mask propagation based on world-frame Kalman prediction, depth consistency, and appearance similarity. The estimated object motion is then used to reject dynamic features from the SLAM pipeline. Experimental results on multiple dynamic RGB-D datasets and real-world experiments demonstrate improved localization accuracy over state-of-the-art dynamic vSLAM approaches, achieving up to 59% lower localization error in scenes characterized by unknown dynamic objects and up to 88% lower error in heavily occluded scenes, while achieving a median processing rate of 30 FPS on a laptop. Furthermore, the tracking module provides estimates of object trajectories and velocities, supporting potential future applications in perception, navigation, and motion-planning tasks in dynamic environments. Code and dataset will be released upon acceptance.

Abstract

Dynamic SLAM

Explore similar work

CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments

NGD-SLAM: Towards Real-Time Dynamic SLAM without GPU

DROID-SLAM in the Wild