Dense Endoluminal Mapping and Localisation from Monocular Bronchoscopy

Salvatore Esposito; Matias Mattamala; Daniel Rebain; Liang Zhao; Kev Dhaliwal; Mohsen Khadem; Subramanian Ramamoorthy

cs.CVOct 12, 2026

Dense Endoluminal Mapping and Localisation from Monocular Bronchoscopy

Authors: Salvatore Esposito, Matias Mattamala, Daniel Rebain, Liang Zhao, Kev Dhaliwal, Mohsen Khadem, Subramanian Ramamoorthy

Organizations: University of Edinburgh · University of British Columbia

Abstract

Bronchoscopy inserts a camera through the airways to diagnose endoluminal disease. Locating the camera along the airway usually requires additional equipment, such as an electromagnetic tracker and a preoperative CT scan. In this paper, we investigate the use of the monocular camera stream to localise the bronchoscope without requiring additional sensing. We introduce a monocular SLAM system that localises the tip of a bronchoscope and reconstructs the airway surface around it from RGB video alone. A Depth Anything 3 backbone, fine-tuned on over a million photorealistic airway renderings, returns metric per-pixel depth and is the system's only source of scale. DINOv3 features routed through a custom cross-attention matcher supply sub-pixel correspondences on mucosa where classical descriptors fail. Both enter a factor graph alongside a dense constraint from projective iterative closest point (ICP) registration against the system's own running reconstruction, so surface evidence stabilises the directions the visual factors leave weakly observable. On six airways rendered from patient CT scans, the system reaches 26.1 mm mean absolute trajectory error and 77.5% mean surface F-score. Its mean surface accuracy exceeds that of five recent monocular SLAM and feed-forward baselines. MASt3R-SLAM, which fits a free scale to the ground truth, has a lower mean trajectory error (22.0 mm). ORB-SLAM3 completes only two of the six airways, whereas our system completes all six and also reconstructs two ex vivo specimens.

Figures & tables

Fig. 1: From bronchoscope video to an endoluminal map. A robotic bronchoscope (left) streams RGB video of the airway interior. This video feds our system, which estimates the camera pose and reconstructs the airway wall, using learnt modules trained on synthetic bronchoscopy data (top right). The robot image is taken from the Ion system page of Intuitive [ 6 ] .

Fig. 2: System overview. For every frame, the front end (Sec. IV ) predicts metric depth and dense descriptors. Camera pose tracking (Sec. VI ) matches descriptors across keyframes and solves PnP for an initial pose. The back end (Sec. VII ) refines that pose by pose-graph optimisation, registers the depth against the running map by ICP, adds the resulting constraint to the pose graph, and integrates the fused pose into a dense TSDF reconstruction. Map management (Sec. V ) maintains the resulting landmark and TSDF maps.

Symbol	Description
$\mathbf{I}_{t}$ , $\mathbf{K}$	RGB frame at time $t$ ; camera intrinsics
$\mathbf{D}_{t}$ , $\mathbf{S}_{t}$	predicted metric depth and its standard deviation
$\mathbf{Q}_{t}$	per-pixel depth reliability, $\in[0,1]$
$\mathbf{T}_{\mathtt{W}\mathtt{C}}^{t}$ , $\mathbf{T}_{\mathtt{W}\mathtt{C}}^{t,\star}$	camera pose; best camera pose estimate (after pass two)
$\boldsymbol{\ell}_{\mathtt{W}}^{j}$	position of landmark $j$ in the world frame $\mathtt{W}$
$\mathbf{u}_{tj}$ , $z_{tj}$	measured patch coordinate and optical-axis depth

TABLE I: Notation used across the article.

Fig. 3: Keypoint correspondences on textureless mucosa using DINO features. The rows show two ex vivo sequences and the columns the same keypoints tracked over four forward displacements. The second row overlays the cosine similarity map for the red keypoint. The same regions are tracked despite the displacement. The front end therefore uses the pre-trained DINOv3 features without fine-tuning.

Fig. 4: Two-pass factor graph. White circles are keyframe poses $\mathbf{T}_{\mathtt{W}\mathtt{C}}^{t}$ and landmarks $\boldsymbol{\ell}_{\mathtt{W}}^{j}$ ; coloured circles are factors, one colour per type (legend, bottom). (1) The first pass optimises an anchor factor on the first pose, a motion factor between consecutive poses, a reprojection factor for each observation of a landmark, and one depth factor between a landmark and the keyframe that created it. It returns the first-pass pose $\mathbf{T}_{\mathtt{W}\mathtt{C}}^{t}$ . (2) That pose initialises the point-to-plane ICP registration of Sec. VII-B against the TSDF submap. (3) The accepted correspondences add one ICP factor on the current pose, alongside the ICP factors of earlier poses, and re-optimising propagates the correction to earlier poses and landmarks, returning the fused pose $\mathbf{T}_{\mathtt{W}\mathtt{C}}^{t,\star}$ .

Fig. 5: Camera paths on the six CT-derived airways. Each panel is a synthetic airway rendered from the CT scan of one patient (all six are ATM’22 cases [ 17 ] ; Sec. IX-A ). Open circles mark the start; filled diamonds mark the end.

Anatomy	AbsRel $\downarrow$	RMSE (mm) $\downarrow$	$\boldsymbol{\delta_{1}}$ (%) $\uparrow$	$\boldsymbol{s^{\star}}$ ( $\to 1$ )
Patient 1	$0.174$	$5.22$	$73.5$	$0.883$
Patient 2	$0.186$	$4.80$	$75.3$	$0.882$
Patient 3	$0.159$	$5.13$	$79.3$	$0.954$
Patient 4	$0.116$	$3.65$	$88.3$	$1.058$
Patient 5	$0.171$	$4.98$	$75.4$	$0.931$
Patient 6	$0.148$	$4.50$	$85.7$	$0.920$

TABLE II: Metric depth accuracy of the bronchoscopy-adapted DA 3 predictor on the six CT-derived airways (Patients 1 to 6, Sec. IX-A ), per valid pixel against simulator ground truth. The scale factor

s^{\star}=\mathrm{median}(z/\hat{z})

tests absolute scale. Lower AbsRel/RMSE, higher

\delta_{1}

, and

s^{\star}

nearer

1.00

are better.

Fig. 6: Estimated trajectories on the six CT-derived airways (rows: synthetic airways rendered from the CT scans of Patients 1 to 6, all ATM’22 cases [ 17 ] ; Sec. IX-A ) for ours, Endo3R, MASt3R-SLAM, VGGT-SLAM 2.0 and ViSTA-SLAM (columns). Dashed grey is ground truth. Colour shows per-frame ATE after alignment (blue $\to$ red, common scale capped at $100$ mm). Each trajectory is aligned with Sim(3) on its first $40\,\%$ of poses and then left unaltered; Table III uses SE(3) for ours. Each row uses the projection plane that the airway’s ground truth moves in; a baseline whose estimate leaves the airway also leaves the panel. Numerical ATE values are in Table III .

	Method	Patient 1	Patient 2	Patient 3	Patient 4	Patient 5	Patient 6	Mean †
ATE (mm) $\downarrow$	Ours	26.42	34.19	28.26	34.53	22.57	10.83	26.13
	ORB-SLAM3 [ 18 ]	$\times$	$\times$	$\times$	20.84	$\times$	6.98	13.91
	Endo3R [ 33 ]	52.36	52.10	163.42	107.05	62.00	84.39	86.89
	MASt3R-SLAM [ 30 ]	15.17	12.35	23.62	23.07	29.12	28.77	22.02
	VGGT-SLAM 2.0 [ 31 ]	321.39	46.73	111.57	53.46	72.66	158.75	127.43
	ViSTA-SLAM [ 32 ]	33.41	17.64	47.60	40.81	18.28	30.27	31.34

TABLE III: Tracking and reconstruction metrics on the six CT-derived airways (Patients 1 to 6, Sec. IX-A ). Our method is aligned with SE(3) and keeps its metric scale; every other method is aligned with Sim(3). The alignments are computed using the first

40\,\%

of poses. F-score at

10

mm. “

\times

” marks tracking failure or under

80\,\%

temporal coverage; “–” marks a sparse map, which has no surface to evaluate. † The mean is over the entries of the row that are not “

\times

”.

Fig. 7: Qualitative comparison of airway-surface reconstruction. Ground truth (GT) surface points and reconstructions from our method and four dense SLAM baselines, on the six CT-derived airways (rows: synthetic airways rendered from the CT scans of Patients 1 to 6, all ATM’22 cases [ 17 ] ; Sec. IX-A ). We register predicted clouds to GT rigidly, and every method keeps the same projection and scale. We crop each panel to the ground truth depth range and mark a method with nothing left inside it accordingly. We omit ORB-SLAM3 for the reconstruction evaluation because of its sparse map.

Configuration	ATE $\downarrow$	F-score $\uparrow$	CD $\downarrow$
Ours (full)	6.12	95.3	4.10
ORB instead of DINO matcher	$\times$	$\times$	$\times$
w/o TSDF map and ICP factor	16.34	64.9	13.97
single-pass fusion	18.88	70.2	15.95
w/o GNC (initial scale)	27.47	55.3	23.55

TABLE IV: Ablation study on the six synthetic phantom sequences (means over B00, B46, B71, B96, B97 and B118; Sec. IX-A ; not the CT-derived airways of Table III ), each row removing or replacing one component. ATE and CD in mm, F-score in %; best per column in bold . “

\times

” indicates tracking failure.

Fig. 8: Reconstruction comparison on ex vivo tissue. Each panel is framed on its own extent because real sequences carry no ground truth pose and monocular scale is free. The comparison therefore judges structural coherence. An empty panel is itself the result.

Fig. 9: From bronchoscope frame to endoluminal surface. Left: input RGB frame from an ex vivo bronchoscopy sequence. Centre: dense point cloud obtained by back-projecting the DA 3 depth at the SLAM-estimated pose. Right: corresponding region of the fused TSDF surface after marching-cubes extraction and light smoothing.

Fig. 10: Estimated trajectory against electromagnetic-tracking ground truth, evaluated as odometry. Five acquisition windows, each projected onto its dominant plane $(u,v)$ . Dashed grey is the EMT reference over its longest continuous valid stretch. We align the estimate with a similarity transform fitted on the start of each window (Sec. IX-E ), and the colour (shared bar, mm) shows the drift accumulated since the start.

Fig. 11: Endoluminal reconstruction on ex vivo tissue. Graph-optimised sparse landmark maps for four of the five EMT windows of Fig. 10 , metrically scaled and rendered as depth-ordered surfels; colour is the observed mucosal appearance and the scale bar is $10$ mm. The maps recover a continuous lumen with visible cartilage ridging and bifurcation openings.

Fig. 12: Novel-view synthesis from the estimated trajectory. For each of the four sequences, an observed frame above the rendering of that same held-out view; the yellow box marks a detailed region, magnified in the inset. See the supplementary video for a closer look.

Sequence	Views	Segments	PSNR train	PSNR held out
1	$647$	$21$	$35.18$	$33.86$
2	$270$	$9$	$35.33$	$34.70$
3	$270$	$9$	$35.43$	$34.53$
4	$337$	$11$	$35.55$	$33.98$
Mean			$\mathbf{35.37}$	$\mathbf{34.27}$

TABLE V: Novel-view synthesis, per sequence. A

3

D Gaussian Splatting model trained from the estimated poses of each EMT sequence. Held-out views are excluded from training on a global index.

Explore similar work

Sep 23, 2026

BronchoTop: Bronchoscopy Navigation via RGB-Only Topological Localization

Accurate localization of the bronchoscope within the bronchial tree is essential for clinicians to be able to reach target lesions, perform biopsies and avoid misidentification of airway segments during diagnostic and therapeutic procedures. However, existing navigation systems typically rely on patient-specific CT scans or additional external sensors, increasing cost, setup time and patient radiation exposure. This work presents BronchoTop, a real-time, RGB-only framework for topological bronchoscopy localization that eliminates the need for patient-specific data. BronchoTop estimates scope location relative to a generic airway model through four modules: lumen detection and tracking, lumen-branch label association, probabilistic scope location estimation, and switch verification. By using only standard bronchoscopy video input, BronchoTop provides practical, real-time navigational assistance to physicians. Evaluation on phantom, simulated and real data demonstrates state-of-the-art accuracy, improving existing approaches performance by over 20% on real bronchoscopy sequences. BronchoTop is the first published framework including both the localization algorithms as well as all the real data used, together with code to generate additional simulations, encouraging and facilitating further developments and benchmarking. The results highlight BronchoTop's potential to enhance procedural safety, efficiency and accessibility in clinical and robotic bronchoscopy.

Clara Tomasini, Ana Cristina Murillo, Luis Riazuelo

DIIS - I3A - Universidad de Zaragoza, Spain

Endoscopy Map-Based Localization Topological Mapping

Ask PDF

Aug 7, 2026

Geometry-Aware Camera Localization for Bronchoscopy

Camera localization in bronchoscopy remains a challenging problem due to stringent accuracy requirements, real-time constraints, and limited training data. Compared to natural scenes, the confined anatomical structures demand millimeter-level precision, while intraoperative guidance necessitates low-latency inference. However, existing methods often fail to effectively exploit preoperative geometric priors, limiting their robustness and accuracy. To address these limitations, we propose a unified geometry-aware bronchoscope localization framework (GABL) that effectively fuses preoperative structural priors with paired intraoperative video to estimate 6-DoF camera poses. Specifically, to address visual ambiguity in complex airways, we propose a graph-guided coarse-to-fine localization scheme that effectively leverages structural priors for precise pose estimation. Furthermore, to mitigate pose jitter and bridge the visual-structural gap, we integrate a Transformer-based tracking model with a novel RGB-depth matching objective, jointly enforcing spatio-temporal and geometric consistency. Extensive experiments demonstrate that our method yields remarkable reductions of 8.37% and 31.76% in translation and rotation errors over the prior state-of-the-art, alongside 4 times inference speedup (33.6 FPS) for robust real-time bronchoscope localization. Project website: https://paulili08.github.io/GABL/.

Lumin Chen, Qingyao Tian, Jinpeng Li +5

Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · The First Affiliated Hospital, Sun Yat-sen University

Medical Imaging Camera Pose Estimation Map-Based Localization

Ask PDF

Jul 6, 2026

Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion

Navigational bronchoscopy is critical for pulmonary interventions, yet current platforms depend heavily on pre-operative CT or external sensors, limiting their use in critical care and resource-constrained settings. Vision-only navigation offers a scalable alternative, but conventional visual odometry (VO) struggles with texture-poor airway images, specularities, and the vanishing-point singularities of tubular anatomy, leading to frequent tracking failures and drift. We present a geometry-aware VO framework that explicitly leverages vanishing-point cues from airway lumens. Detected lumens are back-projected to 3D rays, whose weighted fusion yields a stable forward heading even when parallax cues are absent. This heading, together with looming-based velocity estimates, is fused with noisy VO outputs using a bespoke high-gain observer that enforces airway-following priors and rejects drift. We validate the method on ex-vivo mechanically ventilated human lungs with electromagnetic tracking ground truth. Compared to state-of-the-art pipelines (ORB-SLAM2, LoFTR-VO, DPVO), our approach reduces absolute trajectory error by more than 50% and achieves the lowest relative pose error across all test sequences.

Mohammadreza Kasaei, Francis Xiatian Zhang, Feng Li +3

School of Informatics, University of Edinburgh, UK · Baillie Gifford Pandemic Science Hub, Institute for Regeneration and Repair, University of Edinburgh, UK · Walker Department of Mechanical Engineering and Texas Robotics, The University of Texas at Austin, USA

Visual Odometry

Ask PDF