cs.CVNov 3, 2026

SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model

Authors: Saurabh Koju, Love Panta, Deepak Thapa, Rudra P. K. Poudel, Prajwal Ghimire, Angus Watson, Yash Raj Shrestha, Prashnna Kumar Gyawali, 

Organizations: NAAMII · University of Aberdeen · Toshiba Europe · King’s College London · 

Abstract

Surgical world models have the potential to provide intra-operative guidance, accelerate risk-free training and facilitate transfer of expert surgical maneuvers. However, the use of world models for transferring surgical expertise has been largely unexplored. Moreover, recent approaches in this domain have mainly relied on ground truth kinematics actions or expert annotations for specific tasks, limiting their applicability beyond the narrow range of tasks with ground-truth annotations. In contrast, existing works in the natural domain have shown promising results in learning latent actions solely from unlabeled videos and transferring the learned actions across contexts. We observe that this paradigm is short-sighted for the surgical domain, failing to capture the hierarchical and goal-directed structure of surgical procedures. Thus, we propose a novel multi-step action learning framework to learn context-invariant latent actions that are easily transferable across various surgical contexts. We perform extensive experiments on the publicly available datasets, SurgToolLoc-2022 and PitVis-2023, verifying our approach with both quantitative metrics and qualitative evaluation by surgical experts. We have published our code and the implementation details at https://github.com/bhattarailab/Surgical-Vision-World-Model.

Figures & tables

Explore similar work

CardsList
  1. Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

    Aug 11, 2026Wenrui Bao, Tianyun Jiang, Zhiben Chen +3World Models for RoboticsBimanual Robotic Manipulation

  2. SurgWorld: Learning Surgical Representations Beyond Observation via Latent Prediction

    Nov 8, 2026Jingkun Yue, Pengxiang Wang, Siqi Zhang +7Surgical Video UnderstandingVideo Representation Learning

  3. SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics

    Jun 18, 2026Wentao Pan, Wuyang Li, Shengyuan Liu +3World ModelsVideo Prediction