Surgical Video Understanding

Overview

The field studies how to turn surgical video into useful descriptions of what is happening, where, and when, including procedural phases, tool-tissue interactions, anatomy, and surgical skill. Common approaches combine video encoders and temporal models such as temporal convolutional networks or state-space models with segmentation, tracking, and vision-language methods; newer systems also use structured scene representations and evidence-grounded reasoning. Evaluation spans phase and action recognition on datasets such as Cholec80 and AutoLaparo, instrument and interaction benchmarks such as EndoVis, and skill assessment settings such as JIGSAWS, alongside growing long-video question-answering tests. A central challenge is making models reliable across procedures, hospitals, and changing visual conditions while ensuring that their temporal conclusions and explanations rely on clinically relevant evidence.

Latest papers

CardsList

132 papers17 papers in the last four weeks, up 31% on the four weeks before. 0.1% of all new papers.

  1. SurgAtt-Tracker: Online Surgical Attention Tracking via Temporal Proposal Reranking and Motion-Aware Refinement

    Feb 24, 2026Rulin Zhou, Guankun Wang, An Wang +12Visual AttentionSurgical Video Understanding

  2. YOLOv10-Based Multi-Task Framework for Hand Localization and Laterality Classification in Surgical Videos

    Feb 21, 2026Kedi Sun, Le ZhangSurgical Video UnderstandingVideo Object Detection

  3. SurgFusion-Net: Diversified Adaptive Multimodal Fusion Network for Surgical Skill Assessment

    Feb 18, 2026Runlong He, Freweini M. Tesfai, Matthew W. E. Boal +13Surgical Video UnderstandingMultimodal Fusion

  4. Automated Assessment of Kidney Ureteroscopy Exploration for Training

    Feb 17, 2026Fangjie Li, Nicholas Kavoussi, Charan Mohan +2Surgical Video UnderstandingEndoscopy

  5. SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos

    Feb 5, 2026Jinlin Wu, Felix Holm, Chuxi Chen +18Joint-Embedding Predictive ArchitectureSurgical Video Understanding

  6. Beyond Trajectory Similarity: Evaluating Surgical Planning by Protocol Adherence

    Jan 15, 2026Ruochen Li, Kun Yuan, Yufei Xia +5Trajectory-Based EvaluationLLM Planning

  7. State-Change Learning for Prediction of Future Events in Endoscopic Videos

    Oct 14, 2025Saurav Sharma, Chinedu Innocent Nwoye, Didier Mutter +1Event Time PredictionSurgical Video Understanding

  8. SurgXBench: Explainable Vision-Language Model Benchmark for Surgery

    May 16, 2025Jiajun Cheng, Xianwu Zhao, Sainan Liu +5VLM EvaluationSurgical Video Understanding

  9. SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis

    Mar 13, 2025Chang Han Low, Ziyue Wang, Tianyi Zhang +4Surgical Video UnderstandingMultimodal CoT Reasoning