Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment

Marko Haralović; Zhiqi Miao; Alexander Machiel Bont; Jiapan Guo; Frans van Workum; Estefanía Talavera

cs.CVAug 18, 2026

Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment

Authors: Marko Haralović, Zhiqi Miao, Alexander Machiel Bont, Jiapan Guo, Frans van Workum, Estefanía Talavera

Organizations: University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia · University of Twente, Enschede, The Netherlands · University of Groningen, Bernoulli Institute, Groningen, The Netherlands · Radboud University Medical Center, Nijmegen, The Netherlands ·

Abstract

Surgical performance assessment in minimally invasive surgery largely relies on manual expert review, making it time-consuming, subjective, and difficult to scale. While existing surgical video understanding methods address tasks such as instrument segmentation, surgical phase recognition, and action recognition, they do not explicitly capture fine-grained tissue handling, a key indicator of surgical quality. To address this gap, we introduce tissue tension recognition, a new clinically motivated video understanding task for laparoscopic and robot-assisted rectal cancer surgery. To support this task, we construct SurgTension, the first expert-annotated tissue tension dataset, providing a benchmark for objective tissue tension recognition. We further propose TensionTRAC, a lightweight trajectory-based framework that models tissue tension from sparse point trajectories. Using a compact trajectory encoder, TensionTRAC achieves competitive performance against strong pretrained video backbones.

Figures & tables

Figure 1: Motivation for trajectory-based tissue tension recognition. If we uniformly sample points on a frame and employ a point tracker to track their movement over time (generating trajectories), we can observe that the tension-present clip (a) exhibits spatially structured local tissue deformation, whereas the no-tension clip (b) contains limited or tension-unrelated motion (e.g., instrument movement). This contrast motivates the use of sparse trajectory modeling for tissue tension recognition.

Category	Clinical description	#	%
0 Non-tension ( $\checkmark$ )	No visible traction-induced tissue tension or deformation.	874	22.6
1 Low ( $\checkmark$ )	Minimal traction or tissue deformation, with no apparent risk of tissue injury.	545	14.1
2 Moderate ( $\checkmark$ )	Visible traction and tissue deformation sufficient to facilitate exposure or dissection.	1349	34.9
3 Adequate ( $\checkmark$ )	Pronounced but controlled traction and tissue deformation within the clinically acceptable range.	749	19.4
4 Excessive ( $\checkmark$ )	Marked or uncontrolled tissue stretching beyond the clinically acceptable range, with increased risk of tissue injury or loss of the dissection plane.	76	2.0
Reverse ( $\times$ )	Reduction, release, or reversal of previously applied tissue tension.	208	5.4

Table 1: SurgTension tissue tension dataset categories and category distribution.

Metric	Non-Tension	Tension	$\Delta$	$p$
Visible pts.	87.11	77.33	-9.78	$\mathbf{1.06{\times}10^{-8}}$
Velocity mean	1.78	2.29	+0.51	$\mathbf{1.11{\times}10^{-5}}$
Velocity max	27.84	30.90	+3.06	$\mathbf{7.60{\times}10^{-3}}$
Accel. mean	0.66	0.69	+0.03	$\mathbf{2.40{\times}10^{-2}}$
Accel. max	15.31	11.76	-3.55	$0.633$
Dist. change mean	0.23	-0.30	-0.53	$0.209$

Table 2: Trajectory-derived motion and deformation statistics for Non-Tension and Tension clips. For each metric, Non-Tension and Tension report the average value over clips, and

\Delta=\text{Tension}-\text{Non-Tension}

indicates the magnitude of the difference. The

p

value is obtained from a two-sided statistical test comparing the two groups. Significant differences at the nominal level of

\alpha=0.05

are bolded. The results show that tension clips exhibit stronger motion and deformation patterns, supporting the use of sparse trajectories for tissue tension recognition.

Figure 2: Overview of TensionTRAC . Frozen DINOv2 and CoTrackerV3 models are used offline for semantic-feature extraction and trajectory generation, respectively. The intra-trajectory and inter-trajectory motion branches, feature fusion, and spatio-temporal transformer form the pretrained trajectory encoder, which is frozen to extract clip-level representations for downstream $k$ -NN and MLP classification on SurgTension.

Method	Trainable Params	Frames	Encoder GFLOPs	Pretraining
Method	Trainable Params	Frames	Encoder GFLOPs	Dataset	Scale
SlowFast [ 9 ] R50 $4{\times}16$	34M	$32{\times}1$	36	K400 [ 18 ]	240K videos
Video Swin-3D [ 26 ]	88M	$32{\times}1$	282	IN-21K $\rightarrow$ K400 [ 18 ]	14M img. + 240K vid.
TimeSformer [ 5 ] DivST	122M	$16{\times}1$	362	IN-21K $\rightarrow$ K600 [ 7 ]	14M img. + 392K vid.
SurgMotion [ 48 ] ViT-L	304M	$64{\times}1$	5,785	SurgMotion-15M [ 48 ]	15M images
TensionTRAC	18M	$8{\times}1$	35	K400-10K [ 18 , 49 ]	10K videos

Table 3: Comparison of representation-encoder complexity and pretraining scale. Trainable parameters and pretraining refers to the encoders optimized during their representation-learning stages. For TensionTRAC , DINOv2 and CoTrackerV3 remain frozen and performed offline; the reported parameters, GFLOPs, and pretraining scale cover only the trajectory encoder after offline feature and trajectory extraction.

Task	Method	Classifier	Stratified			Video-grouped
			Acc.	BalAcc.	M-F1	Acc.	BalAcc.	M-F1
Binary	SlowFast [ 9 ]	KNN	87.6	80.4	82.2	81.1	72.7	73.6
	SlowFast [ 9 ]	MLP	87.7	84.3	83.7	79.7	75.9	74.3
	TimeSformer [ 5 ]	KNN	89.7	83.6	85.2	81.9	72.4	73.9
	TimeSformer [ 5 ]	MLP	88.6	85.1	84.8	83.0	76.2	76.6
	Video Swin-3D [ 26 ]	KNN	89.7	83.6	85.3	81.5	73.1	74.0

Table 4: Comparison of KNN (

k=3

) and MLP classifiers on frozen features for binary tension classification and end-to-end cascade tension recognition. Stratified evaluation uses clip-level stratified splits, whereas video-grouped evaluation assigns all clips from the same surgical video exclusively to the same fold, ensuring strict source-video separation. All metrics are reported in %.

Figure 3: Qualitative results of TensionTRAC for tissue tension recognition. The examples present predictions for binary tension classification (left) and cascade two-stage evaluation (right). Sparse trajectories visualize local tissue and instrument motion in tension-present clips, while failure cases arise from camera, instrument, or other non-tension-related motion, as well as confusion between neighboring tension levels.

Surgical Video Understanding

Surgical video understanding uses computer vision and multimodal AI to interpret procedures, instruments, actions, and events over time.

See all 113 papers

TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Models
Mar 7, 2026Jiajun Cheng, Xiaofan Yu, Subarna Tripathi +2Medical VLMs
S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks
Sep 13, 2026Jiaming Zhang, Zijian Wu, Mehran Armand +1Point Tracking Surgical Video Understanding
Department of Computer Science, Johns Hopkins University · Robotics and Control Laboratory, The University of British Columbia · The Institute for Integrative and Innovative Research, University of Arkansas
AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction
Apr 1, 2026Aiza Maksutova, Lalithkumar Seenivasan, Hao Ding +5Robotic Affordance Inference Surgical Robotics
Johns Hopkins University, Baltimore MD, USA

Mar 7, 2026J/K move · Enter open · S save

TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Models

Jiajun Cheng, Xiaofan Yu, Subarna Tripathi, Sainan Liu+1

Medical VLMs

Open paper AskPDF arXiv

Recognizing instruments' interactions with tissues is essential for building context-aware AI assistants in robotic surgery. Vision-language models (VLMs) have opened a new avenue for surgical perception and achieved better generalization on a wide range of tasks compared to conventional task-specific deep learning approaches. However, their performance on instrument--tissue interaction recognition remains limited, largely due to two challenges: (1) many models do not effectively leverage temporal information, and (2) alignment between vision and text often misses fine-grained action details. To address these issues, we propose TrajPred, a framework that encodes instrument trajectories to incorporate temporal motion cues and, conditioned on these trajectories, introduces a predictor module to generate visual semantic embeddings that better capture fine-grained action details. We further incorporate prompt tuning and a verb-rephrasing technique to enable smooth adaptation to the instrument--tissue interaction recognition task. Extensive experiments on the public laparoscopic benchmark, CholecT50, show that our method improves both Average Precision and Top-K accuracy. We also investigate whether visual embeddings of instrument--tissue interaction regions align better with the corresponding text by visualizing the cosine similarity between visual and textual embeddings. The visualization results indicate that the proposed method improves alignment between relevant visual and textual representations.

Abstract

Surgical Video Understanding

Explore similar work

TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Models

S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks

AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction