Surgical Scene Understanding

Surgical scene understanding aims to automatically interpret the visual and auditory information within an operating room to improve surgical procedures and training. Current research focuses on developing robust models, often employing vision transformers and large language models, to perform tasks such as semantic segmentation of instruments and tissues, 3D reconstruction of the surgical scene, and understanding surgeon intent from audio and visual cues. These advancements hold significant promise for improving surgical safety, assisting surgeons during procedures, and creating more effective surgical training tools.