SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video

Guanyi Qin; Xiaozhen Wang; Zhu Zhuo; Chang Han Low; Yuancan Xiao; Yibing Fu; Haofeng Liu; Kai Wang

cs.CVFeb 25, 2026

SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video

Authors: Guanyi Qin, Xiaozhen Wang, Zhu Zhuo, Chang Han Low, Yuancan Xiao, Yibing Fu, Haofeng Liu, Kai Wang, +2 more

Organizations: National University of Singapore, Singapore · Guangzhou Research Translation and Innovation Institute, National University of Singapore, China · Southern Medical University, China

Abstract

Minimally invasive surgery has dramatically improved patient operative outcomes, yet identifying safe operative zones remains challenging in critical phases, requiring surgeons to integrate visual cues, procedural phase, and anatomical context under high cognitive load. Existing AI systems offer binary safety verification or static detection, ignoring the phase-dependent nature of intraoperative reasoning. We introduce ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder. We introduce evaluation metrics that treat correct grounding under incorrect phase as failures, revealing that most vision-language models cannot handle such tasks and perform poorly. We then present SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context. On unseen procedures, SurGo-R1 achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hardcore accuracy, a 6.6 $\times$ improvement over the mainstream generalist VLMs. Code, model and benchmark will be available at https://github.com/jinlab-imvr/SurGo-R1

Figures & tables

Figure 1: Demo of the proposed ResGo dataset, a novel multimodal benchmark that covers phase recognition, Go Zone grounding, and safety reasoning. Built on ResGo , a GRPO-optimized VLM with context-aware multi-turn reasoning, named SurGo-R1 , is further introduced, enabling safety-aware reasoning and Go Zone grounding across scenarios.

Figure 2: The left panels detail patient demographics and clinical profiles, including distributions for gender, age, ASA classification, BMI, and relevant medical history. The right panels present the processed surgical phase statistics, illustrating duration distributions, occurrence frequencies, and the phase transition matrix for the four defined phases.

Data Acquisition
Attribute	Specification
Equipment	Laparoscope (Olympus/Storz/H.)
Video Resolution	$1920\times 1080$ (Downsampled)
FPS	0.2 fps
Total Videos	21 (Cholecystectomy)
Total Raw Frames	6,138

Table 1: Annotation Statistics and Properties. 2,686 annotated samples in total, where each sample consists of a structured instruction spanning four dimensions. desc. for description.

Figure 3: Illustration of the annotation and review pipeline

Figure 4: Overview of SurGo-R1 and phase-then-go pipeline.

Method	Phase	Grounding				Conditioned				Hardcore
Method	Acc	Acc@ 0.25	mA@ 0.25:0.5	$\Delta_{cen}\downarrow$	mIoU	CA 0.25	CA 0.25:0.5	C $\Delta_{cen}\downarrow$	CmIoU	HA 0.25	HmIoU
Generalist VLMs
Qwen3-VL-4B-Ins	36.5	14.7	7.61	11.7	12.3	21.5	11.5	8.84	15.8	7.88	5.79
Qwen3-VL-8B-Ins	39.3	16.9	7.83	10.9	13.2	14.8	6.24	9.83	12.5	5.86	4.94
Qwen3-VL-32B-Ins	53.1	16.3	8.01	9.29	14.2	15.5	7.67	9.26	13.5	8.28	7.20
Qwen3-VL-30B-A3B-Ins	51.9	11.9	5.35	12.6	10.7	12.8	5.51	11.7	11.4	6.67	5.95

Table 2: Quantitative results on the ResGo dataset. The best results are highlighted in bold .

Figure 5: Qualitative results of reasoning and grounding. Models trained without $\mathbf{R}_{\text{reason}}$ are denoted as w/o Rea or w/o Reason. Yellow and green bounding boxes represent model predictions and ground truths, respectively.

$\mathbf{R}_{\text{dist}}$	Def-Tool	Grounding			Hardcore
$\mathbf{R}_{\text{dist}}$	Def-Tool	Acc@ 0.25	$\Delta_{cen}\downarrow$	mIoU	HA 0.25	HmIoU
✗	✗	56.3	5.77	23.2	43.5	16.9
✓	✗	58.4	5.23	23.6	44.2	17.5
✗	✓	67.8	4.35	32.1	53.1	25.0
✓	w/o-rectify	64.7	4.58	30.3	48.7	21.4
✓	✓	68.3	4.11	32.7	54.8	25.9

Table 3: Ablations on the ResGo dataset. ✓indicates inclusion in the training process, while ✗ denotes exclusion.

Method	Phase	Grounding			Hardcore
Method	Acc	Acc@ 0.25	$\Delta_{cen}\downarrow$	mIoU	HA 0.25	HmIoU
Single-turn	69.5	66.9	4.24	32.1	49.6	22.7
Multi-turn (Ours)	76.6	68.3	4.11	32.7	54.8	25.9

Table 4: Comparison between Single- and Multi-turn performance.

Setting	Selected Ratio (%) $\uparrow$	Acc. Rating $\uparrow$
Qwen3-VL-8B-Ins	0.27	10.7
w/o $\mathbf{R}_{\text{reason}}$	17.3	47.2
w/ $\mathbf{R}_{\text{reason}}$ (Ours)	79.9	52.5

Table 5: Ablation study on the reasoning reward (

\mathbf{R}_{\text{reason}}

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Procedural Phase	Go Zone Definition & Reasoning Instruction
Preparation of the Go Zone	Target the Calot’s triangle area. The Go Zone is defined as the peritoneum overlying the cystic duct and artery junction, where the initial dissection must commence to expose the underlying structures.
Dissection of Calot’s Triangle	Target the “Safety Window.” The Go Zone is strictly limited to the fibro-fatty tissue within the triangle. It explicitly excludes the liver bed (cystic plate) and the common bile duct (CBD) to prevent bile duct injury.
Clip and Divide	Target the skeletonized cystic duct and artery. The Go Zone is the specific segment of these structures that is free of surrounding tissue and distinct from the CBD, rendering it suitable for safe clip application.
Gallbladder Dissection	Target the connective tissue plane (areolar tissue). The Go Zone is identified as the white or translucent line of adhesion separating the gallbladder from the liver bed.

Appendix

Table 6: Anatomical Go Zone Definitions for Phase-Guided Reasoning. The content utilized by the Phase-Definition Mapping Tool is presented in this table, supplying specific operative cues due to the deficiency in surgical domain knowledge common to current VLMs.

Explore similar work

Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation
Mar 12, 2026Jian Jiang, Chenxi Lin, Yiming Gu +24Medical VLMs Surgical Video Understanding
Global College, Shanghai Jiao Tong University, Shanghai, China · Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Biomedical Engineering and Department of Electrical and Computer Engineering, National University of Singapore, Singapore +11
SUREON: A Benchmark and Vision-Language-Model for Surgical Reasoning
Mar 6, 2026Alejandra Perez, Anita Rau, Lee White +4Surgical Video Understanding Medical VQA
Intuitive Surgical Inc., Sunnyvale, United States
SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis
Mar 13, 2025Chang Han Low, Ziyue Wang, Tianyi Zhang +4Surgical Video Understanding Multimodal CoT Reasoning
National University of Singapore, Singapore · Bioinformatics Institute (BII), Agency for Science, Technology and Research (A*STAR), Singapore · Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) and the Department of Medical Physics and Biomedical Engineering, University College London, UK

Mar 12, 2026J/K move · Enter open · S save

Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation

Jian Jiang, Chenxi Lin, Yiming Gu, Zengyi Qin+23

Global College, Shanghai Jiao Tong University, Shanghai, China · Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Biomedical Engineering and Department of Electrical and Computer Engineering, National University of Singapore, Singapore+11

Medical VLMs Surgical Video Understanding VLM Reasoning

Open paper AskPDF arXiv

Surgical scene understanding demands not only accurate predictions but also interpretable reasoning that surgeons can verify against clinical expertise. However, existing surgical vision-language models generate predictions without reasoning chains, and general-purpose reasoning models fail on compositional surgical tasks without domain-specific knowledge. We present Surg-R1, a surgical Vision-Language Model that addresses this gap through hierarchical reasoning trained via a four-stage pipeline. Our approach introduces three key contributions: (1) a three-level reasoning hierarchy decomposing surgical interpretation into perceptual grounding, relational understanding, and contextual reasoning; (2) the largest surgical chain-of-thought dataset with 320,000 reasoning pairs; and (3) a four-stage training pipeline progressing from supervised fine-tuning to group relative policy optimization and iterative self-improvement. Evaluation on SurgBench, comprising six public benchmarks and six multi-center external validation datasets from five institutions, demonstrates that Surg-R1 achieves the highest Arena Score (64.9%) on public benchmarks versus Gemini 3.0 Pro (46.1%) and GPT-5.1 (37.9%), outperforming both proprietary reasoning models and specialized surgical VLMs on the majority of tasks spanning instrument localization, triplet recognition, phase recognition, action recognition, and critical view of safety assessment, with a 15.2 percentage point improvement over the strongest surgical baseline on external validation.

Abstract

Explore similar work

Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation

SUREON: A Benchmark and Vision-Language-Model for Surgical Reasoning

SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis