cs.CVSep 25, 2026

Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment

Authors: Qing Xu, Yuxiang Luo, Zhen Chen

Organizations: School of Computer Science, University of Nottingham, UK · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Surgical scene understanding is critical for computer-assisted intervention, yet laparoscopic cholecystectomy remains challenged by the complex anatomy of the hepatocystic triangle and the risk of bile duct injury. Existing methods for Critical View of Safety (CVS) assessment typically treat it as a holistic prediction task, mapping visual features directly to criterion-level labels. This black-box paradigm lacks explicit reasoning about anatomical relationships, limiting both interpretability and compositional generalization. To address this, we propose ReasonCVS, a structured reasoning agentic framework empowered by Vision-Language Models (VLMs) that decomposes CVS assessment into explicit, fine-grained anatomical verification. Specifically, we devise an Anatomical Scene Graph Abstraction (ASGA) that organizes anatomical entities and their spatial relationships into a structured representation. To operationalize this, we introduce a Rationale-Aware Reasoning Agent, powered by a Large Language Model (LLM) fine-tuned via rationale distillation. Functioning as a strict central decision-maker, it invokes VLM-driven Sub-criterion Verifier as a specialized perceptual tool to parse the graph and independently evaluate individual sub-criteria. Through calibrated soft reasoning, this agent synthesizes the tool-gathered distributed observations, yielding a final verdict alongside a traceable clinical rationale. Extensive experiments on the Endoscapes-CVS201 benchmark demonstrate that ReasonCVS achieves superior performance (68.1% mAP) over state-of-the-art while providing interpretable, criterion-level explanations for reliable surgical assessment.

Figures & tables

Figure 1: Comparison of existing methods and our ReasonCVS for CVS assessment. (a) Existing methods map visual features directly to criterion-level labels for holistic prediction. (b) Our ReasonCVS abstracts the surgical scene into an anatomical scene graph and invokes sub-criterion Verification for interpretable assessment.
Figure 2: The overview of our ReasonCVS framework for interpretable CVS assessment, consisting of Anatomical Scene Graph Abstraction (ASGA), VLM-driven Sub-criterion Verification, and Rationale-Aware Reasoning Agent. ReasonCVS decomposes CVS assessment into explicit anatomical verification with traceable clinical rationale.
Table 1: Comparison with state-of-the-art methods on Endoscapes-CVS201 for CVS assessment. †\dagger denotes spatiotemporal methods. “-” indicates that the result is not reported in the original paper. Best results are in bold , second best underlined .
Figure 3: Qualitative comparison of CVS assessment results. Our ReasonCVS provides correct, criterion-level rationales grounded in specific anatomical evidence, enabling interpretable and traceable clinical reasoning.
Table 2: Ablation study of our CVSAgent on Endoscapes-CVS201 for CVS assessment.
Figure 4: Distribution of expert evaluation scores for the Vanilla Agent and ReasonCVS across four dimensions. ReasonCVS consistently achieves higher and more concentrated scores, indicating superior and more stable reasoning quality.
More in this topic

Surgical Video Understanding

Surgical video understanding uses computer vision and multimodal AI to interpret procedures, instruments, actions, and events over time.

See all 118 papers

Explore similar work

CardsList
Apr 24, 2026

Sum-of-Checks: Structured Reasoning for Surgical Safety with Large Vision-Language Models

Purpose: Accurate assessment of the Critical View of Safety (CVS) during laparoscopic cholecystectomy is essential to prevent bile duct injury, a complication associated with significant morbidity and mortality. While large vision-language models (LVLMs) offer flexible reasoning, their predictions remain difficult to audit and unreliable on safety-critical surgical tasks. Methods: We introduce Sum-of-Checks, a framework that decomposes each CVS criterion into expert-defined reasoning checks reflecting clinically relevant visual evidence. Given a laparoscopic frame, an LVLM evaluates each check, producing a binary judgment and justification. Criterion-level scores are computed via fixed, weighted aggregation of check outcomes. We evaluate on the Endoscapes2023 benchmark using three frontier LVLMs, comparing against direct prompting, chain-of-thought, and sub-question decomposition, each with and without few-shot examples. Results: Sum-of-Checks improves average frame-level mean average precision by 12--14% relative to the best baseline across all three models and criteria. Analysis of individual checks reveals that LVLMs are reliable on observational checks (e.g., visibility, tool obstruction) but show substantial variability on decision-critical anatomical evidence. Conclusion: Structuring surgical reasoning into expert-aligned verification checks improves both accuracy and transparency of LVLM-based CVS assessment, demonstrating that explicitly separating evidence elicitation from decision-making is critical for reliable and auditable surgical AI systems. Code is available at https://github.com/BrachioLab/SumOfChecks.

Weiqiu You, Cassandra Goldberg, Amin Madani +2

Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA. · Department of Surgery, University of Toronto, Toronto, ON, Canada. · Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada. +1

EndoscopySurgical Video UnderstandingStructured ReasoningMedical VLMsLaparoscopic Surgery
AskPDF
Sep 23, 2026

CasCVS-Net: A Staged Multi-Task Cascade for Critical View of Safety Assessment

Automated assessment of the Critical View of Safety (CVS) in laparoscopic cholecystectomy requires both recognition of the three CVS criteria and anatomical grounding in small, rare, and often occluded hepatocystic structures. Learning-based methods differ in the anatomical information they use, from image-level classification to detection, segmentation, or graph-based reasoning, yet grounding the safety-critical anatomy remains the main bottleneck. We propose CasCVS-Net, a staged multi-task cascade that jointly performs object detection, semantic segmentation, and CVS assessment, trained on the Endoscapes dataset. The model couples the tasks through predicted anatomy: predicted boxes guide segmentation, and predicted masks provide region-level features for CVS classification, so CVS assessment at inference uses only model predictions rather than ground-truth annotations. To reduce optimisation instability in this coupled setting, training progresses from detection to detection-segmentation and then to the full three-task cascade, followed by task-wise fine-tuning. Evaluation on the public unseen test set shows that CasCVS-Net improves over matched single-task baselines on all three tasks, achieving 32.0 detection mAP, 46.8 semantic mIoU, 15.3 rare-anatomy mIoU, and 67.2 CVS mAP. It outperforms the state-of-the-art LG-CVS and SV2LSTG by 6.3% and 4.5% relative CVS mAP, respectively, corresponding to 4.0 and 2.9 mAP points. These results show that staged task coupling through predicted boxes and masks improves anatomical grounding for CVS assessment, particularly for rare hepatocystic structures.

Bock-Zien Toh, Yuanchuan Ren, Tay Aw Yu +3

Department of Computer Science, University College London, London, UK · UCL Hawkes Institute, University College London, UK

Multi-Task LearningSurgical Video UnderstandingMedical Image SegmentationObject Detection
AskPDF
Mar 12, 2026

Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation

Surgical scene understanding demands not only accurate predictions but also interpretable reasoning that surgeons can verify against clinical expertise. However, existing surgical vision-language models generate predictions without reasoning chains, and general-purpose reasoning models fail on compositional surgical tasks without domain-specific knowledge. We present Surg-R1, a surgical Vision-Language Model that addresses this gap through hierarchical reasoning trained via a four-stage pipeline. Our approach introduces three key contributions: (1) a three-level reasoning hierarchy decomposing surgical interpretation into perceptual grounding, relational understanding, and contextual reasoning; (2) the largest surgical chain-of-thought dataset with 320,000 reasoning pairs; and (3) a four-stage training pipeline progressing from supervised fine-tuning to group relative policy optimization and iterative self-improvement. Evaluation on SurgBench, comprising six public benchmarks and six multi-center external validation datasets from five institutions, demonstrates that Surg-R1 achieves the highest Arena Score (64.9%) on public benchmarks versus Gemini 3.0 Pro (46.1%) and GPT-5.1 (37.9%), outperforming both proprietary reasoning models and specialized surgical VLMs on the majority of tasks spanning instrument localization, triplet recognition, phase recognition, action recognition, and critical view of safety assessment, with a 15.2 percentage point improvement over the strongest surgical baseline on external validation.

Jian Jiang, Chenxi Lin, Yiming Gu +24

Global College, Shanghai Jiao Tong University, Shanghai, China · Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Biomedical Engineering and Department of Electrical and Computer Engineering, National University of Singapore, Singapore +11

Medical VLMsSurgical Video UnderstandingVLM Reasoning
AskPDF