cs.CVOct 6, 2026
SaveLARK: A Low-Cost, Accurate, Occlusion-Resilient, Kalman Filter-Assisted Tracking System for Image-Guided Surgery
Organizations: Department of Mechanical Engineering, McGill University, Montreal, Canada · Montreal Neurological Institute, McGill University, Montreal, Canada
Abstract
Image-guided surgery (IGS) depends on accurate tracking of surgical instruments to provide real-time navigation relative to anatomical structures. Commercial stereo infrared trackers are accurate but prone to occlusion and cost-prohibitive for many settings. This work presents LARK, a multi-camera optical tracking system using commodity RGB hardware and multi-view redundancy and fusion. We develop and evaluate two complete tracking methods: multi-view monocular pose fusion and multi-view triangulation. Both methods are assessed under varying occlusion levels using a precision-machined grid and an anatomical head phantom, and compared against a gold-standard stereo infrared system. With five cameras and adaptive Kalman filtering, LARK achieves median target registration errors of 0.64 mm for point localization with triangulation and 0.73 mm for trajectory tracking with pose fusion on the machined grid. Camera-subset experiments show graceful degradation in adaptive pose-fusion accuracy as fewer views remain available. With tracking hardware costing under $1,000 USD, LARK provides a low-cost platform for image-guided surgery research. Hardware designs and software are publicly available at https://nist.mni.mcgill.ca/software/ , and datasets at https://nist.mni.mcgill.ca/data/ .
Figures & tables
Figure 1 : Annotated schematic of the proposed system. The system is composed of a multi-camera fixture (top left), a surgical pointer and a static reference (bottom left). The fixture is attached to a surgical light to track the pointer relative to the reference (right). Typical surgical light geometries consist of an articulated arm whose base is mounted to the OR ceiling. The position of the light can be freely manipulated to illuminate the surgical area, and by doing so, ensure that the cameras maintain line of sight with the anatomy of interest.
Figure 2 : Real-time surgical navigation interface using the IBIS platform ( Drouin et al., 2017 ) . The tracked pointer position (red dot) is displayed on axial, coronal, and sagittal MRI views, with a 3D rendering showing the pointer, reference dodecahedron, and patient anatomy.
Figure 3 : Instrument calibration schematic. The pointer is pivoted and rotated about its axis on the ChArUco board origin (bottom left) while video frames are recorded. After initialisation the dodecahedron marker corner coordinates are refined.
Figure 4 : Multi-view monocular pose fusion schematic. For each camera that views the surgical pointer and static reference, an independent monocular pose measurement is computed through the PnP algorithm. At each frame, an adaptive Kalman filter is used to fuse all available measurements.
Figure 5 : Experimental validation setups. Left column shows the experiment with the precision-machined grid. Right column shows the experiment with the SLA-printed head phantom. Both setups include the multi-camera fixture mounted on the surgical light stand-in, the surgical pointer, and the static reference. The fusionTrack 500 optical tracker (visible on the side) provides simultaneous measurements.
Figure 6 : Experimental setups for system evaluation. (a) Machined grid for baseline accuracy measurements. (b) SLA-printed head phantom for neurosurgery-representative validation.
Figure 7 : Head phantom experiment occlusion scenarios and severity scale.
| # of Cameras | Monocular Pose Fusion | Triangulation | fusionTrack 500 | |||||
| Basic | Kalman | Kalman Adaptive | Basic | Kalman | Kalman Adaptive | |||
| Grid Experiment | ||||||||
| Fiducial Point Median Registration RMSE | 1 | 1.2224 (±0.2577) | 1.1693 (±0.2532) | 1.1732 (±0.2531) | N/A | N/A | N/A | 0.5410 (±0.0420) |
| 2 | 0.9697 (±0.1757) | 0.9704 (±0.1734) | 1.0193 (±0.1728) | 0.9942 (±0.3259) | 0.9871 (±0.3241) | 0.9648 (±0.3158) | ||
| 3 | 0.8419 (±0.1454) | 0.8576 (±0.1445) | 0.9174 (±0.1448) | 1.0433 (±0.4894) | 1.0427 (±0.4755) | 1.0250 (±0.4076) | ||
| 4 | 0.7683 (±0.1275) | 0.7820 (±0.1272) | 0.8517 (±0.1307) | 0.9461 (±0.2039) | 0.9487 (±0.2021) | 0.9249 (±0.1890) | ||
Table 1 : Registration RMSE in mm (median ± standard deviation) across 20 trials for Grid and Head Phantom experiments. Total trajectory RMSE pools valid samples across the measured curves for both systems.
Figure 8 : TRE analysis on the machined grid showing filtering method comparison across camera counts. All rows show individual point TRE values (left column) and per-trajectory RMSE values (right column). Top row: Monocular pose fusion. Bottom row: Triangulation. Boxes = IQR, red dots = median, violins = kernel density. Statistical comparisons use 20 paired trial summaries.
Figure 9 : Five-camera performance comparison with adaptive Kalman filtering on the machined grid. Both LARK pose estimation methods (monocular pose fusion and triangulation) are compared against the gold-standard fusionTrack 500 for point localization (left) and trajectory tracking (right). Boxes = IQR, red dots = median, violins = kernel density.
Figure 10 : TRE analysis on the SLA-printed head phantom showing filtering method comparison across camera counts. All rows show individual point TRE values (left column) and per-trajectory RMSE values (right column). Top row: Monocular pose fusion. Bottom row: Triangulation. Boxes = IQR, red dots = median, violins = kernel density. Statistical comparisons use 20 paired trial summaries.
Figure 11 : Five-camera performance comparison with adaptive Kalman filtering on the anatomical head phantom. Under challenging viewing conditions with partial marker occlusion, triangulation outperforms monocular pose fusion for both point localization (1.13 mm versus 1.35 mm) and trajectory tracking (1.06 mm versus 1.53 mm). Boxes = IQR, red dots = median, violins = kernel density.
Figure 12 : Camera views during trajectory tracking at the moment of maximum error in the head phantom experiment. All five camera views are shown with detected markers highlighted in green boxes. Camera 3 (top right) demonstrates a poor view condition where the pointer partially occludes the reference, leaving only a single marker visible at an oblique angle to the image plane. The bottom right panel shows a magnified view of camera 3’s compromised perspective.
Figure 13 : Comparison of tracked trajectories versus ground truth for the left occlusion scenario in the head phantom experiment trial which produced the largest overall outlier. Top two rows show monocular pose fusion for 3-camera and 5-camera configurations respectively. Bottom two rows show triangulation for 3-camera and 5-camera configurations respectively. Columns from left to right show Basic, Kalman, and Adaptive Kalman filtering. Red lines represent ground truth, blue connected dots represent tracked trajectories. The cropped, shaded head provides qualitative spatial context; the shared XYZ triad indicates orientation only. Panel scales differ, so apparent trajectory sizes should not be compared across panels.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1 : Spatial distribution of the norm of the measurement noise covariance matrices for each camera across all measured positions.
| # of Cameras | Monocular Pose Fusion | Triangulation | |||||
| Basic | Kalman | Kalman Adaptive | Basic | Kalman | Kalman Adaptive | ||
| Top Occlusion Scenario | |||||||
| Fiducial Point Median Registration RMSE | 1 | 1.7044 (±0.3059) | 1.6380 (±0.2716) | 1.6414 (±0.2710) | N/A | N/A | N/A |
| 2 | 1.3203 (±0.2003) | 1.2829 (±0.1911) | 1.2201 (±0.1589) | 1.1210 (±0.1340) | 1.1065 (±0.1322) | 1.0839 (±0.1245) | |
| 3 | 1.1006 (±0.1276) | 1.1261 (±0.1387) | 1.1227 (±0.1088) | 1.0796 (±0.1323) | 1.0707 (±0.1310) | 1.0701 (±0.1260) | |
| 4 | 1.0754 (±0.1302) | 1.0981 (±0.1330) | 0.9978 (±0.0973) | 1.0196 (±0.1273) | 1.0106 (±0.1186) | 1.0093 (±0.0947) | |
Table C.1 : Registration RMSE in mm (median ± standard deviation) across 20 Head Phantom Experiment trials for Top, Right, and Left Occlusion Scenarios.
Figure C.1 : Error-distribution violins for FRE across the two tracking pipelines. Top row: Grid experiment; Bottom row: Head Phantom experiment. Left column = Monocular Pose Fusion, Right column = Triangulation. Boxes = IQR, red dots / curved boxes = median, violins = kernel density. Median accuracy value reported in each box below the violin plot. Statistical comparisons use 20 paired trial summaries per phantom.
| Camera counts | Grid | Head phantom | ||||
| FRE | Point TRE | Trajectory TRE | FRE | Point TRE | Trajectory TRE | |
| 1 versus 2 | 1.91e-05 | 1.91e-05 | 3.15e-04 | 1.91e-05 | 1.91e-05 | 1.91e-05 |
| 1 versus 3 | 2.67e-05 | 1.91e-05 | 0.0408 † | 2.86e-05 | 1.91e-05 | 0.8182 |
| 1 versus 4 | 1.91e-05 | 1.91e-05 | 0.1392 | 1.91e-05 | 1.91e-05 | 0.0099 † |
| 1 versus 5 | 1.91e-05 | 1.91e-05 | 1.14e-04 | 1.91e-05 | 1.91e-05 | 0.0029 |
| 2 versus 3 | 0.0136 | 0.0073 | 1.91e-05 | 0.0032 | 1.34e-04 | 0.0099 † |
Table C.2 : Two-sided paired Wilcoxon signed-rank -values for pooled camera-count comparisons, with Holm correction over the ten count pairs within each experiment and metric. Trial mean squares are averaged equally across available tracking methods and filters before taking the square root (pose fusion only at one camera; both tracking methods at two to five cameras). All tests use 20 paired trial summaries. marks a comparison whose significance decision at differs under the Holm-adjusted paired sign-test sensitivity analysis.
| Camera count | Tracking method | Metric | Basic versus Kalman | Basic versus Kalman Adaptive | Kalman versus Kalman Adaptive |
| 1 | Monocular Pose Fusion | FRE | 1.89e-04 | 1.89e-04 | 0.8124 |
| Point TRE | 5.04e-04 | 9.65e-04 | 0.7562 | ||
| Trajectory TRE | 5.72e-06 | 5.72e-06 | 1.05e-04 | ||
| 2 | Monocular Pose Fusion | FRE | 0.5459 | 0.0097 † | 0.0060 † |
| Point TRE | 0.2858 | 0.2858 | 0.1595 | ||
| Trajectory TRE | 5.72e-06 | 1.14e-05 | 0.0192 |
Table C.3 : Two-sided paired Wilcoxon signed-rank -values for filtering methods (Grid), with Holm correction over the three filter pairs within each camera count, tracking method, and metric. All tests use 20 paired trial summaries. marks a comparison whose significance decision at differs under the Holm-adjusted paired sign-test sensitivity analysis.
| Camera count | Tracking method | Metric | Basic versus Kalman | Basic versus Kalman Adaptive | Kalman versus Kalman Adaptive |
| 1 | Monocular Pose Fusion | FRE | 7.63e-06 | 5.72e-06 | 8.51e-04 |
| Point TRE | 0.4049 | 0.4049 | 0.0983 | ||
| Trajectory TRE | 0.0592 | 0.0577 | 0.0592 | ||
| 2 | Monocular Pose Fusion | FRE | 0.6976 | 0.6976 | 0.4949 |
| Point TRE | 0.1794 | 0.2611 | 0.0719 | ||
| Trajectory TRE | 0.8124 | 0.0645 | 0.1165 |
Table C.4 : Two-sided paired Wilcoxon signed-rank -values for filtering methods (Head phantom), with Holm correction over the three filter pairs within each camera count, tracking method, and metric. All tests use 20 paired trial summaries. marks a comparison whose significance decision at differs under the Holm-adjusted paired sign-test sensitivity analysis.