Markerless Robot Detection and 6D Pose Estimation for Multi-Agent SLAM

Markus Rueggeberg; Maximilian Ulmer; Maximilian Durner; Wout Boerdijk; Marcus Gerhard Mueller; Rudolph Triebel; Riccardo Giubilato

cs.ROFeb 18, 2026

Markerless Robot Detection and 6D Pose Estimation for Multi-Agent SLAM

Authors: Markus Rueggeberg, Maximilian Ulmer, Maximilian Durner, Wout Boerdijk, Marcus Gerhard Mueller, Rudolph Triebel, Riccardo Giubilato

Organizations: Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Weßling, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Abstract

The capability of multi-robot SLAM approaches to merge localization history and maps from different observers is often challenged by the difficulty in establishing data association. Loop closure detection between perceptual inputs of different robotic agents is easily compromised in the context of perceptual aliasing, or when perspectives differ significantly. For this reason, direct mutual observation among robots is a powerful way to connect partial SLAM graphs, but often relies on the presence of calibrated arrays of fiducial markers (e.g., AprilTag arrays), which severely limits the range of observations and frequently fails under sharp lighting conditions, e.g., reflections or overexposure. In this work, we propose a novel solution to this problem leveraging recent advances in Deep-Learning-based 6D pose estimation. We feature markerless pose estimation as part of a decentralized multi-robot SLAM system and demonstrate the benefit to the relative localization accuracy among the robotic team. The solution is validated experimentally on data recorded in a test field campaign on a planetary analogous environment.

Figures & tables

Panel of Figure 1 — Fig. 1 : ( top: ) Multi-robot detection of the LRU (Lightweight Rover Unit), and the Lander unit, from the perspective of the 2nd LRU unit during the [anonymous] field test campaign on Mount. Etna, Sicily. The figure shows the projection of the robot shapes, known a-priori, to demonstrate the quality of pose estimation. The distance of the LRU rover to the observer, the LRU2, as well as intense light reflections, would make it impossible through conventional fiducial markers to establish a robot detection. ( bottom: ) Members of the multi-robot team on Mt. Etna: LRU, LRU2 and the Lander unit.

Method	Constraint Source	Detects Robots	Use Markers	Detection Type	Modality
Kimera-Multi [ 7 ]	VPR, BoW vectors	✗	-	-	Visual
DOOR-SLAM [ 5 ]	VPR, NetVLAD descriptors	✗	-	-	Visual
CVI-SLAM [ 8 ]	VPR, visual features	✗	-	-	Visual
CCM-SLAM [ 4 ]	VPR, visual features	✗	-	-	Visual
LAMP 2.0 [ 9 ]	LiDAR scan registration	✗	-	-	LiDAR
DiSCo-SLAM [ 10 ]	Scan Context descriptors	✗	-	-	LiDAR

TABLE I : Comparison of Selected Collaborative SLAM, Tracking or Localization Approaches by Inter-Robot Constraint

Fig. 2 : Schematic overview of the employed decentralized SLAM system, with a focus on the multi-robot detection module capabilities. Each robot utilizes visual and inertial inputs to compute state estimation and partition robot states into submaps. Visual inputs are used to compute visual keyframes for place recognition, as well as detecting robots in the image, either through legacy AprilTag detection, or the proposed markerless approach. Results of submapping and robot detections from each robot in the team are embedded into a SLAM graph

Panel of Figure 4 — Fig. 3 : Impressions of training samples. ( left: ) OAISYS images including LRU and Lander in an Etna-like setting. ( right: ) BlenderProc images, with LRU, Lander and random AprilTags for distractions, on a random background.

Fig. 4 : Illustration of the markerless detection pipeline. A 2D object detection first generates an image crop, which is then processed by the pose estimator. The encoder-decoder architecture produces three outputs: 2D–3D correspondences, foreground-background masks, and surface regions as an auxiliary task. The 2D–3D correspondences are subsequently passed to a pose regression network to predict the object’s 6D pose.

Fig. 5 : Examples of heavy augmentations on LRU instances from the synthetic samples used for an effective training of the pose estimation network in Sec. III-B

Test Set	Performance Metric	Observed Robot	Training Set
Test Set	Performance Metric	Observed Robot	BPROC	OAISYS	COMBINED
BPROC	$\rho\ \ \ \ \ \ \ \ (\uparrow)$	Lander	0.160	0.260	0.284
	$\rho\ \ \ \ \ \ \ \ (\uparrow)$	LRU	0.080	0.064	0.324
	$t\ \ \ [m]\ \ (\downarrow)$	Lander	0.310	1.497	0.259
	$t\ \ \ [m]\ \ (\downarrow)$	LRU	0.082	1.621	0.153
	$\theta\ [rad]\ (\downarrow)$	Lander	0.421	2.207	0.514

TABLE II : Synthetic Test Results: Detection rate

\rho

, translation error

t

, rotational error

\theta

for different training and test set combinations. The arrow after a variable indicates whether a higher (

\uparrow

) or lower (

\downarrow

) value is desirable.

Panel of Figure 8 — Fig. 6 : Evaluation of translation ( top-left ) and rotation ( top-right ) errors of the proposed markerless pose estimation approach, and comparison with AprilTag detection, between the real LRU system, with LRU2 as observer, under VICON ground-truth measurements. ( bottom: ) Visual examples of markerless robot detection and re-projection of the CAD model on the image.

Performance Metric	Tag	Tag + Markerl.	Improvement
Mission 1, LRU
# Detections	49	87	+87%
Max. Det. Distance [m]	4.94	16.15	+227%
Max. Open-Loop Duration [s]	2300	1033	-55%
Traj. Error [m]	1.57	1.13	-28%
Mission 1, LRU2

TABLE III : Multi-robot detection metrics during the SLAM tests comparing the baseline approach with AprilTag only, and the proposed addition of Markerless detection and pose estimation

Panel of Figure 10 — Fig. 7 : Multi-robot SLAM results from Mission 1 . ( left: ) Trajectory of the two rovers over the entire mission, localized via visual multi-robot SLAM using full input knowledge. ( top right: ) Localization error of LRU, when observing the Lander and LRU2. Error correction after markerless LRU2 observations at 516s , 874s and markerless Lander detection at 2250s . ( bottom right: ) Localization error of LRU2, when observing the Lander and LRU. Error correction after markerless LRU detections at 636s , 1304s .

Panel of Figure 11 — Fig. 8 : Multi-robot SLAM results from Mission 2 . ( left: ) Trajectory of the two rovers over the entire mission, localized via visual multi-robot SLAM using full input knowledge. ( top right: ) Localization error of LRU, when observing the Lander and LRU2. Error correction after markerless Lander observation at 593s and markerless LRU2 observation at 751s . ( bottom right: ) Localization error of LRU2, when observing the Lander and LRU. No significant differences with or without the markerless modality.

Simultaneous Localization and Mapping

Simultaneous localization and mapping estimates a robot’s motion while building a map of an unknown environment.

See all 197 papers

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM
Jun 23, 2026Yixian Zhao, Yan Huang, Yang Xu +2Simultaneous Localization and Mapping Riemannian Optimization
College of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, China · Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology, Stockholm, SE-100 44, Sweden · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai, 201210, China
Semantic Semi-Incremental Data-Association-Free Object SLAM
Jul 25, 2026Yihao Zhang, Jungseok Hong, John J. LeonardSimultaneous Localization and Mapping Visual SLAM
Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology
A Decentralized LiDAR-SLAM System with Certifiably Optimal Pose Graph Optimization
May 24, 2026Baoshan Song, Feng Huang, Li-Ta HsuSimultaneous Localization and Mapping Pose Graph Optimization
Department of Aeronautical and Aviation Engineering

Jun 23, 2026J/K move · Enter open · S save

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

Yixian Zhao, Yan Huang, Yang Xu, Liang Li+1

College of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, China · Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology, Stockholm, SE-100 44, Sweden · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai, 201210, China

Simultaneous Localization and Mapping Riemannian Optimization Pose Graph Optimization Decentralized Optimization

Open paper AskPDF arXiv

Pose graph optimization (PGO) is a key back-end component for state estimation in networked multi-robot simultaneous localization and mapping (SLAM). In object-based multi-robot SLAM, the problem becomes more tightly coupled because robots must jointly estimate both their trajectories and the poses of persistent objects observed by multiple agents. Existing decentralized solutions often assume that the communication graph closely matches the physical interaction topology, which is restrictive in realistic deployments where communication is sparse, intermittent, or time-varying. This paper presents a fully decentralized Riemannian optimization framework for object-based multi-robot PGO that decouples the coupled estimation problem via a consensus mechanism, enabling flexible communication topologies. To improve convergence under limited communication budgets, we further develop a distributed approximate-Newton scheme that exploits local second-order information while operating directly on the SE(d) manifold to preserve geometric consistency, and we establish the convergence to Riemannian first-order stationary points and provide a local condition-number analysis explaining the benefit of approximate second-order information over first-order Riemannian descent. The resulting method reduces iteration count and communication overhead without sacrificing estimation accuracy. Extensive evaluations on public benchmarks, large-scale simulations, and real-world multi-robot experiments demonstrate improved accuracy, runtime efficiency, scalability across network topologies, and robustness to communication failures.

Abstract

Simultaneous Localization and Mapping

Explore similar work

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

Semantic Semi-Incremental Data-Association-Free Object SLAM

A Decentralized LiDAR-SLAM System with Certifiably Optimal Pose Graph Optimization