CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation
Authors: Mohammad Eslami, Dhanvinkumar Ganeshkumar, Saber Kazeminasab, Michael G. Morley, Michael V. Boland, Michael M. Lin, John B. Miller, David S. Friedman, Nazlee Zebardast, Lucia Sobrin, Tobias Elze
Organizations: Harvard Ophthalmology AI Lab, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA · Thomas Jefferson High School for Science and Technology, Chantilly, Virginia, USA · Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
We present CataractSAM-2, a domain-adapted extension of Meta's Segment Anything Model 2, designed for real-time semantic segmentation of cataract ophthalmic surgery videos with high accuracy. Positioned at the intersection of computer vision and medical robotics, CataractSAM-2 enables precise intraoperative perception crucial for robotic-assisted and computer-guided surgical systems. Furthermore, to alleviate the burden of manual labeling, we introduce an interactive annotation framework that combines sparse prompts with video-based mask propagation. This tool significantly reduces annotation time and facilitates the scalable creation of high-quality ground-truth masks, accelerating dataset development for ocular anterior segment surgeries. We also demonstrate the model's strong zero-shot generalization to glaucoma trabeculectomy procedures, confirming its cross-procedural utility and potential for broader surgical applications. The trained model and annotation toolkit are released as open-source resources, establishing CataractSAM-2 as a foundation for expanding anterior ophthalmic surgical datasets and advancing real-time AI-driven solutions in medical robotics, as well as surgical video understanding.
TABLE I : Annotated Classes in the Cataract-1K Dataset
Figure 1: Random examples of binary segmentation predictions from SAM-2 models using SAM2ImagePredictor, shown alongside test frames and ground truth masks; performance differences are expected as only our model was trained on cataract surgery.
Figure 2: Performance Comparison for IoU, and PAC Across Six Test Videos.
Figure 3: CataractSAM-2 generalization to trabeculectomy surgery using SAM2VideoPredictor. From top to bottom: (1) initial conjunctival incision, (2) scleral flap dissection, and (3) scleral flap creation during trabeculectomy. Left: original video frame. Right: predicted segmentation by CataractSAM-2. More frames are shown in Appendix.
Figure 4: CataractSAM-2 performance on the CaDIS dataset across 25 cataract videos. Metrics are reported as mean ± standard deviation.
Figure 5: The interactive prompt-based annotation pipeline, user interface, and the resulting masks for a four-class instrument video clip. A video demonstration is available at https: (removed for anonymity, but available in supplementary material)
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure: Qualitative montage (random 100 frames from several videos) illustrating the generalization of CataractSAM-2 to trabeculectomy (glaucoma filtering surgery), serving as a baseline for a prompt-based annotation tool. Segmentation masks were generated using the SAM2 Automatic Mask Generator (CataractSAM-2 model) without manual refinement. The results demonstrate the promising ability of CataractSAM-2 to generalize to other anterior segment surgeries and serve as an initialization step for prompt-based annotation tools.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Center for Imaging Science, Rochester Institute of Technology, Rochester, NY 14623, USA · Biomedical Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA
Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore · School of Computer Science, The University of Sheffield, Sheffield, UK +1