Attachment Anchors: A Novel Framework for Laparoscopic Grasping Point Prediction in Colorectal Surgery

Dennis N. Schneider; Lars Wagner; Daniel Rueckert; Dirk Wilhelm

cs.CVFeb 19, 2026

Attachment Anchors: A Novel Framework for Laparoscopic Grasping Point Prediction in Colorectal Surgery

Authors: Dennis N. Schneider, Lars Wagner, Daniel Rueckert, Dirk Wilhelm

Organizations: Technical University of Munich, TUM School of Medicine and Health, TUM University Hospital rechts der Isar, Department of Surgery, Research Group MITI, Munich, Germany · Technical University of Munich, TUM School of Medicine and Health, TUM University Hospital rechts der Isar, Chair for AI in Healthcare and Medicine Munich, Munich, Germany · Department of Computing, Imperial College London, London, UK

Abstract

Accurate grasping point prediction is a key challenge for autonomous tissue manipulation in minimally invasive surgery, particularly in complex and variable procedures such as colorectal interventions. Due to their complexity and prolonged duration, colorectal procedures have been underrepresented in current research. At the same time, they pose a particularly interesting learning environment due to repetitive tissue manipulation, making them a promising entry point for autonomous, machine learning-driven support. Therefore, in this work, we introduce attachment anchors, a structured representation that encodes the local geometric and mechanical relationships between tissue and its anatomical attachments in colorectal surgery. This representation reduces uncertainty in grasping point prediction by normalizing surgical scenes into a consistent local reference frame. We demonstrate that attachment anchors can be predicted from laparoscopic images and incorporated into a grasping framework based on machine learning. Experiments on a dataset of 90 colorectal surgeries demonstrate that attachment anchors improve grasping point prediction compared to image-only baselines. There are particularly strong gains in out-of-distribution settings, including unseen procedures and operating surgeons. These results suggest that attachment anchors are an effective intermediate representation for learning-based tissue manipulation in colorectal surgery.

Figures & tables

Fig. 1: Pipeline of the proposed grasping point prediction framework. The model first detects and classifies the attachment anchor representation from the input image and then predicts a grasping point relative to the anchor origin.

Panel of Figure 2 — Fig. 2: Example surgical input images illustrating the three attachment anchor cases. Each image shows the dissection point $D$ (top) and the corresponding attachment anchor representation (bottom), with the soft tissue of interest highlighted in blue.

Fig. 3: Architecture of the proposed model. The attachment anchor encoder $\phi_{A}$ (blue) extracts the anchor representation, which is then used by the grasping point decoder $\phi_{G}$ (cyan).

Fig. 4: Dataset composition showing colorectal surgical procedures stratified by anatomical region along the colon.

Panel of Figure 5 — Fig. 5: Statistical distribution of grasping points before and after normalizing with attachment anchors, per case.

	Grasp		Dissect
Type	std(x) (%)	std(y) (%)	std(x) (%)	std(y) (%)
Absolute	17.81	17.09	16.45	17.22
Case 1	18.29	16.91	17.54	16.19
Case 2	16.87	17.11	15.32	17.01
Case 3	18.05	17.41	17.27	16.24
Relative	13.53	15.74	-	-

TABLE I: Standard deviations of grasping and dissection point distributions across absolute coordinates, relative-to-dissection coordinates, and attachment anchor-based normalized representations.

Model	Precision@6% $\uparrow$	RMSE $\downarrow$	Type Precision $\uparrow$
KP-YOLOv8	$78.24\pm 2.62$	$12.91\pm 4.81$	$64.50\pm 5.64$
+ Contrastive	$\textbf{87.84}\pm\textbf{3.40}^{\dagger}$	$\textbf{9.75}\pm\textbf{1.66}$	$\textbf{67.17}\pm\textbf{7.84}$
${}^{\dagger}\;\textrm{p}<0.01$
$\textrm{*}\;\textrm{p}<0.05$

TABLE II: Attachment anchor prediction performance (mean

\pm

standard deviation

\%

) under 5-fold cross-validation. Results compare a purely image-based keypoint detector (KP-YOLOv8) with the same architecture augmented by contrastive learning.

Model	RMSE (rad) $\downarrow$
	$\mathbf{e}_{adh}$	$\mathbf{e}_{mnt,1}$	$\mathbf{e}_{mnt,2}$
KP-YOLOv8	$0.55\pm 0.04$	$0.63\pm 0.04$	$0.65\pm 0.09$
+ Contrastive	$\textbf{0.48}\pm\textbf{0.03}\textrm{*}$	$\textbf{0.57}\pm\textbf{0.04}\textrm{*}$	$\textbf{0.60}\pm\textbf{0.04}$
${}^{\dagger}\;\textrm{p}<0.01$
$\textrm{*}\;\textrm{p}<0.05$

TABLE III: Attachment anchor direction prediction performance (mean RMSE

\pm

standard deviation

radian

) under 5-fold cross-validation. Results compare a purely image-based keypoint detector (KP-YOLOv8) with the same architecture augmented by contrastive learning.

Model	Precision@6% $\uparrow$	RMSE $\downarrow$
Vision only
KP-YOLOv8	$37.80\pm 2.44$	$18.63\pm 0.84$
Attachment Anchors
Abs-YOLOv8	$49.90\pm 4.60$	$\textbf{14.77}\pm\textbf{1.12}^{\dagger}$
Rad-YOLOv8	$\textbf{51.20}\pm\textbf{3.70}^{\dagger}$	$15.00\pm 1.44$
${}^{\dagger}\;\textrm{p}<0.01$

TABLE IV: Grasping point prediction performance (mean

\pm

standard deviation

\%

) under 5-fold cross-validation. Results compare vision-only prediction with models incorporating attachment anchor representations.

Model	Precision@6% on unseen surgery type $\uparrow$				Mean (%)
Model	Sigmoid	Rectum	Hemicol. left	Hemicol. right	Mean (%)
KP-YOLOv8	$30.66\pm 2.77$	$41.60\pm 3.01$	$27.04\pm 4.32$	$33.89\pm 10.26$	33.30
Abs-YOLOv8	$43.39\pm 1.04$	$52.06\pm 1.86$	$\textbf{36.30}\pm\textbf{1.48}^{\dagger}$	$\textbf{50.33}\pm\textbf{4.90}\textrm{*}$	45.52
Rad-YOLOv8	$\textbf{44.10}\pm\textbf{1.54}^{\dagger}$	$\textbf{52.22}\pm\textbf{1.65}^{\dagger}$	$35.40\pm 2.44$	$48.85\pm 4.45$	45.14
${}^{\dagger}\;\textrm{p}<0.01$
$\textrm{*}\;\textrm{p}<0.05$

TABLE V: Grasping point prediction performance (mean

\pm

standard deviation

\%

) on unseen surgeries across 5 evaluation runs.

Model	Precision@6% on unseen surgeon identifier $\uparrow$						Mean (%)
Model	1	2	3	4, 5, 6	7, 8, 9	10, 11, 12, 13, 14	Mean (%)
KP-YOLOv8	$43.05\pm 2.76$	$34.71\pm 2.00$	$43.56\pm 5.35$	$33.25\pm 5.22$	$33.72\pm 2.65$	$38.28\pm 6.67$	37.76
Abs-YOLOv8	$\textbf{56.27}\pm\textbf{3.62}^{\dagger}$	$38.82\pm 2.73$	$\textbf{53.97}\pm\textbf{1.49}^{\dagger}$	$53.18\pm 3.97$	$\textbf{48.57}\pm\textbf{1.43}^{\dagger}$	$\textbf{50.00}\pm\textbf{3.45}^{\dagger}$	50.14
Rad-YOLOv8	$55.17\pm 4.74$	$\textbf{40.68}\pm\textbf{3.50}^{\textrm{*}}$	$51.77\pm 1.32$	$\textbf{55.82}\pm\textbf{4.59}^{\dagger}$	$48.20\pm 3.35$	$48.05\pm 2.71$	49.95
${}^{\dagger}\;\textrm{p}<0.01$
$\textrm{*}\;\textrm{p}<0.05$

TABLE VI: Grasping point prediction performance (mean

\pm

standard deviation

\%

) on unseen surgeons across 5 evaluation runs. Surgeons are grouped to ensure sufficient sample sizes per group.

Model	Precision@6% $\uparrow$
Model	W/o augmentation	With augmentation
Abs-YOLOv8	$49.90\pm 4.60$	$\textbf{52.27}\pm\textbf{3.93}$
Rad-YOLOv8	$51.20\pm 3.70$	$\textbf{52.55}\pm\textbf{3.05}$

TABLE VII: Grasping point prediction performance (mean

\pm

standard deviation

\%

) with and without attachment anchor–based warp transformations, evaluated over 5 independent runs.

Panel of Figure 13 — Fig. 6: Example of a realistic surgical scene transformation obtained by warping only the adhesion vector $e_{adh}.$

Panel of Figure 14 — Fig. 7: Statistical distribution of grasping and dissection points in the dataset.

Laparoscopic Surgery

See all 21 papers

AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction
Apr 1, 2026Aiza Maksutova, Lalithkumar Seenivasan, Hao Ding +5Robotic Affordance Inference Surgical Robotics
Johns Hopkins University, Baltimore MD, USA
Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment
Aug 18, 2026Marko Haralović, Zhiqi Miao, Alexander Machiel Bont +3Surgical Video Understanding Laparoscopic Surgery
University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia · University of Twente, Enschede, The Netherlands · University of Groningen, Bernoulli Institute, Groningen, The Netherlands +2
S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks
Sep 13, 2026Jiaming Zhang, Zijian Wu, Mehran Armand +1Point Tracking Surgical Video Understanding
Department of Computer Science, Johns Hopkins University · Robotics and Control Laboratory, The University of British Columbia · The Institute for Integrative and Innovative Research, University of Arkansas

Apr 1, 2026J/K move · Enter open · S save

AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction

Aiza Maksutova, Lalithkumar Seenivasan, Hao Ding, Jiru Xu+4

Johns Hopkins University, Baltimore MD, USA

Robotic Affordance Inference Surgical Robotics

Open paper AskPDF arXiv

Surgical action automation has progressed rapidly toward achieving surgeon-like dexterous control, driven primarily by advances in learning from demonstration and vision-language-action models. While these have demonstrated success in table-top experiments, translating them to clinical deployment remains challenging: current methods offer limited predictability on where instruments will interact on tissue surfaces and lack explicit conditioning inputs to enforce tool-action-specific safe interaction regions. Addressing this gap, we introduce AffordTissue, a multimodal framework for predicting tool-action specific tissue affordance regions as dense heatmaps during cholecystectomy. Our approach combines a temporal vision encoder capturing tool motion and tissue dynamics across multiple viewpoints, language conditioning enabling generalization across diverse instrument-action pairs, and a DiT-style decoder for dense affordance prediction. We establish the first tissue affordance benchmark by curating and annotating 15,638 video clips across 103 cholecystectomy procedures, covering six unique tool-action pairs involving four instruments (hook, grasper, scissors, clipper) and their associated tasks: dissection, grasping, clipping, and cutting. Experiments demonstrate substantial improvement over vision-language model baselines (20.6 px ASSD vs. 60.2 px for Molmo-VLM), showing that our task-specific architecture outperforms large-scale foundation models for dense surgical affordance prediction. By predicting tool-action specific tissue affordance regions, AffordTissue provides explicit spatial reasoning for safe surgical automation, potentially unlocking explicit policy guidance toward appropriate tissue regions and early safe stop when instruments deviate outside predicted safe zones.

Abstract

Laparoscopic Surgery

Explore similar work

AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction

Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment

S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks