Polygraph: Large-Scale Multi-Session Trajectory Optimization using Pose Graphs
Authors: Petar Lukovic, Tian Yi Lim, Marc Pollefeys, Pablo Speciale, Hermann Blum, Zuria Bauer
Organizations: ETH Zurich, Switzerland. · Microsoft Spatial AI Lab, Zurich, Switzerland. · University of Bonn and Lamarr Institute for Machine Learning and Artificial Intelligence, Germany.
Robots, vehicles, and wearable devices often traverse the same environment many times. Each pass produces its own trajectory from onboard visual or visual-inertial odometry. These trajectories are locally accurate but drift over time and live in unrelated coordinate frames. Turning many such independent trajectories into one globally consistent set is the large-scale multi-session trajectory optimization problem. Existing solutions largely work offline. All sessions must be available up front, one session is fixed as the reference frame, and a single fused map is then optimized. This paper presents Polygraph, a multi-session trajectory optimization pipeline that processes sessions online, one keyframe and one spatial cluster at a time, and never builds a 3D map. Keyframes are grouped into local clusters. When a cluster closes, it is matched against every other session through learned image retrieval. Candidate pairs are verified with sparse feature matching, monocular metric depth, and robust pose estimation. A two-tier consensus gate admits a cross-session edge into the shared pose graph only once independent evidence corroborates it. A cheap incremental solve keeps the graph current, and a periodic robust batch solve prunes outliers. We evaluate Polygraph on stereo outdoor driving, HoloLens and iPad indoor traversals, legged indoor locomotion with a Boston Dynamics Spot, and large-scale outdoor vehicle and handheld routes. Against the closest prior work on the same GPU model, using its reported timings, Polygraph runs faster, commits fewer loop-closure edges, and matches or improves on accuracy.
Figures & tables
Figure 1: Loop-closure visualisation . Five independent HYDRO sessions recorded with a Boston Dynamics Spot, each in its own color, placed into one shared frame by Polygraph and overlaid on the site’s 3D scan. Purple lines mark confirmed cross-session loop-closure edges. No two of these trajectories start out knowing that they share the same building.
Figure 2: Polygraph pipeline . Each session’s odometry is keyframed and grouped online into spatial clusters. A closed cluster is matched against every other session by image retrieval, candidate pairs are verified by joint pose estimation into one confirmed match, which enters the shared pose graph once the consensus gate admits it. A fast incremental solve runs after every closed cluster, and a periodic full solve prunes outliers.
Figure 3: Joint pose estimation . Turns one candidate cluster pair into one confirmed match. A spanning tree over descriptor similarity selects a few keyframe pairs (purple, one wrong pair in red). Each pair yields its own metric pose, which is lifted through the odometry into the common cluster frame. Estimates that agree within τagree form the consensus set, and the outlier is dropped. The chordal mean of the set is the measurement the gate decides on, and the agreeing pairs enter the graph as factors once the match is admitted.
Figure 4: The consensus gate . Same-session matches commit directly. A cross-session edge whose query session has no live cross-session edge yet must find an agreeing, independent peer in the corroboration pool. Once a session is connected, each edge is checked against its live estimate and queued for later re-checks if it fails.
ORB-SLAM3 odometry
VINS-Fusion odometry
Sequence
Length
Duration
odom
ORB-
Maplab
Maplab
2GO
Ours
Ours
odom
VINS-
Maplab
Maplab
2GO
Ours
Ours
only
SLAM3
BIN
SP
single
multi
only
PGO
BIN
SP
single
multi
KITTI-00
3.72 km
470 s
4.20
4.85
37.51
12.69
1.82
1.28
1.27
13.09
4.20
11.13
11.39
1.58
1.34
1.33
KITTI-02
5.07 km
483 s
8.05
5.64
18.01
44.78
8.04
5.30
5.30
21.02
13.03
23.96
18.29
10.58
7.96
7.96
KITTI-05
2.21 km
288 s
2.05
0.95
7.44
7.98
0.92
0.89
0.89
6.39
3.68
11.37
11.21
1.70
1.88
1.88
KITTI-06
1.23 km
114 s
2.05
0.79
7.09
10.74
0.62
0.72
0.72
3.54
1.68
13.27
14.76
1.48
1.36
1.36
Table I: RMSE ATE on KITTI’s loop sequences and on VBR . Refining ORB-SLAM3 (left) and VINS-Fusion (right) odometry [ 8 ] . Best per odometry source shaded and bold, second best underlined.
Dataset
Setting
Devices
Sess.
Multi-device
KITTI
outdoor, driving
car, stereo
7
no
VBR
driving, handheld
car / handheld
8
no
HYDRO
indoor
Spot, iPad
10
yes
HGE
indoor
HoloLens, iPad
9
yes
EuRoC
indoor, MAV
MAV, stereo(-IMU)
11
yes
Table II: Datasets used throughout Experiments . Multi-device marks a site captured by more than one platform. VBR’s multi-session sites include two campus sessions.
Figure 5: Single-session VBR visualisation . Two handheld and two vehicle. Top shows VINS-Fusion odometry; bottom shows Polygraph, both against ground truth in gray. Shades mark spatial clusters; dots mark clusters joined by a surviving loop closure.
Method
Split (robots)
Joint ATE (m)
RTF
Swarm-SLAM [ 6 ]
2
2.17
–
Ours, w/ RTAB-Map odom.
2
1.16
0.81
Ours, w/ ORB-SLAM3 odom.
2
1.22
0.74
Ours, w/ VINS-Fusion odom.
2
1.47
0.73
DOOR-SLAM [ 5 ]
3
8.00
–
Ours, w/ RTAB-Map odom.
3
1.26
0.95
Table III: KITTI multi-robot experiment . KITTI-00 cut into 2 or 3 equal parts, one robot each, against DOOR-SLAM and Swarm-SLAM on their own protocol, RTAB-Map stereo odometry for both. Best shaded and bold, second underlined, per split. RTF is conservative, run on a shared cluster.
ATE (m)
Anchoring, mixed
Device
odom only
multi-session
mixed
trans (m)
trans (%)
rot (deg)
HYDRO
Spot
Mean
0.299
0.149
0.148
0.181
0.20
0.39
L2
0.727
0.336
0.336
0.362
0.41
0.79
iOS
Mean
0.408
0.312
0.309
0.418
0.47
0.62
L2
0.990
0.722
0.721
0.963
1.08
1.42
HGE
HoloLens
Mean
0.658
0.280
0.281
0.956
0.57
0.84
Table IV: RMSE ATE on HYDRO [ 11 ] (5 Spot and 5 iOS sessions, footprint diagonal 89 m) and HGE [ 10 ] (5 HoloLens and 4 iOS sessions, footprint diagonal 167 m) . Each device on its own odometry, and reference-anchoring error with every session of a site placed into one shared frame. Pooled per device as mean and L2 norm. Best of multi-session and mixed shaded and bold, second best underlined.
Figure 6: Reference anchoring on HGE (top) and HYDRO (bottom) . One fixed transform per connected component applied to every session past its reference. Ground truth in green, Polygraph’s estimate on top. On HGE, (a) is HoloLens alone with two reference groups, (b) is iOS alone, (c) is HoloLens and iOS anchored together. On HYDRO, (a) is Spot alone, (b) is Spot and iOS together, all ten sessions in one component.
Dataset
Cameras
Single (m)
Multi (m)
Cross edges
HGE
1, baseline
0.652
0.280
252
4, best
0.652
0.278
506
HYDRO
1, baseline
0.236
0.149
1484
4, best
0.197
0.146
3672
KITTI
1, baseline
1.946
1.930
18
2, best
1.946
1.930
18
Table V: Camera-pairing ablation . Baseline is single-camera production; best is the best combination available. The better ATE per pair is shaded and bold.
Figure 7: Pose-estimation ablation . Mean wall time against mean ATE across KITTI, HGE, HGE iOS, and HYDRO, both raw values, so KITTI’s larger gaps keep their real weight instead of being normalized down to HGE’s. XLG = XFeat+LighterGlue (production), M3R = MASt3R, M3D = Metric3D depth, M3Rd = MASt3R’s own depth. XLG+M3D ties M3R+M3D on accuracy (2.20 m each) at 21% less wall time. XLG+M3Rd is faster still but 0.07 m worse, mostly one 0.3 m KITTI gap. MASt3R-native is off this plot at 4.26 m, nearly twice everyone else’s.
Method
Front-end
Edges
Trans. (m)
Rot. (deg)
COVINS-G PnP
VINS-Mono
852
0.69 ± 0.60
3.44 ± 3.5
COVINS-G 17-point
VINS-Mono
1823
0.10 ± 0.17
1.09 ± 1.25
COVINS-G PnP
ORB-SLAM3
505
0.23 ± 0.32
1.56 ± 3.7
COVINS-G 17-point
ORB-SLAM3
406
0.07 ± 0.12
1.01 ± 0.9
Ours, stereo-inertial
ORB-SLAM3
43
0.48 ± 0.29
1.61 ± 0.96
Ours, stereo
ORB-SLAM3
62
0.57 ± 0.41
2.31 ± 1.71
Table VI: EuRoC [ 13 ] . Accuracy of committed cross-session edges on MH01–05 before graph optimization, mean and standard deviation against ground truth, and COVINS-G values from their paper [ 33 ] .
Figure 8: Runtime and committed edges against 2GO . Ran on the same RTX 4090, VINS-Fusion odometry, log scale. 2GO values are from their paper; ours is the mean of five runs. ciampino’s speed config trims top_k_clusters / min_kf_pairs vs. production.
Submitted
Trans err (m)
Rot err (deg)
Dataset
On
Off
On
Off
On
Off
Exec Δ
HGE
250
373
0.399
2.574
0.87
12.41
+7.0%
HYDRO
775
839
0.726
1.223
0.90
1.78
+8.6%
Total
1025
1212
0.646
1.639
0.89
5.05
+8.2%
Table VII: Two-tier consensus gate ablation . Six HGE/HYDRO multi-session conditions pooled by dataset. Submitted are distinct cross-session candidates the gate admitted; trans/rot err is per committed edge before graph optimization. Exec Δ is gate-off vs. on wall time, run on an RTX 2080, not comparable in absolute terms to the pinned-4090 numbers elsewhere. Gate-on shaded and bold.
Trajectory optimization computes motion or control sequences that minimize costs while satisfying system dynamics, safety limits, and task constraints.