Spatiotemporal Recognition of Abdominal Wall Layers During Optical Trocar Entry

Shreeram Athreya; Andrew Melehy; Carter Prentice; Michael Robinson; Corey Arnold; James Wu; William Speier

cs.CVOct 10, 2026

Spatiotemporal Recognition of Abdominal Wall Layers During Optical Trocar Entry

Authors: Shreeram Athreya, Andrew Melehy, Carter Prentice, Michael Robinson, Corey Arnold, James Wu, William Speier

Organizations: Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, California, USA. · Department of Surgery, University of California, Los Angeles, Los Angeles, California, USA. · Department of Radiological Sciences, University of California, Los Angeles, Los Angeles, California, USA. · Department of Bioengineering, University of California, Los Angeles, Los Angeles, California, USA. · Department of Pathology and Laboratory Medicine, University of California, Los Angeles, Los Angeles, California, USA.

Abstract

Objective. To evaluate whether spatial context and temporal modeling improve tissue recognition during optical trocar entry, where multiple abdominal-wall tissues may be visible simultaneously. Methods. We retrospectively analyzed 71 recordings with 1,516 annotated keyframes and 29,151 clinician-annotated sectors. Each frame was divided into 28 polar-grid sectors assigned to nine tissue classes. Color, neighboring-sector, and positional features were used by a gradient-boosted emission classifier. The full pipeline added learned temporal smoothing, spatial regularization, an anatomically ordered hidden Markov model, and posterior blending. Five-fold cross-validation was grouped by recording, with cross-fitting to prevent within-recording leakage. Primary evaluation used clinician-derived targets; ablations assessed classification, probability quality, spatial consistency, and peritoneum-or-deeper onset timing. Results. The complete pipeline achieved accuracy 0.646, macro-F1 0.507, and macro-AUROC 0.882, versus 0.600, 0.440, and 0.861 for emissions alone. Macro-F1 improved by 0.066 across all five folds. Label changes decreased from 66.2 to 19.5 per sector-minute, and median onset error decreased from 15.58 to 2.45 seconds. Frames with a tissue-depth-code span of at least three decreased from 53% to 26%, versus 22% for clinician annotations. Performance remained limited for preperitoneal fat and bowel, and the full model retained a 44% premature-onset proportion. Conclusions. Combining appearance, spatial context, and temporal constraints improved retrospective tissue recognition and onset localization. Tissue-specific limitations, residual timing errors, and reliance on future observations warrant external validation and prospective evaluation of causal models before clinical use.

Figures & tables

Figure 1: Spatial and temporal posterior refinement. Learned temporal smoothing combines emission probabilities across a temporal neighborhood, shown schematically around time $t$ . MRF regularization couples neighboring sectors within each frame. The banded HMM permits persistence and transitions between adjacent tissue states $k-1$ , $k$ , and $k+1$ . Posterior blending combines the regularized output with the original emission probabilities, represented by the dashed bypass, to obtain final tissue-class probabilities. The illustrated temporal neighborhood and colors are schematic; the temporal neighborhoods and nine-class vocabulary are specified in the text.

Dataset component	Count or specification
Analyzed videos	71 (21 dense; 50 transferred)
Approximately 60-fps recordings	62 (59.99–60.02 fps)
Approximately 30-fps recordings	9 (29.97–30.01 fps)
Annotated keyframes with eligible sectors	1,516
All-label sectors	36,886
Clinician-only sectors	29,151

Table 1: Dataset Assembly and Evaluation Populations

Channel	Lower endpoint	Upper endpoint
$L^{*}$	25	95
$a^{*}$	$-15$	40
$b^{*}$	$-15$	55
Rotated hue	0	0.45
Saturation	0	0.7
Value	0.3	1.0

Table 2: Color Normalization and Sector Descriptor Composition

Model	Accuracy	Macro-F1	Macro AUROC	$\Delta$ F1 (positive folds)
Class-prior reference	$0.084\pm 0.024$	$0.017\pm 0.005$	$0.500\pm 0.000$	$-0.423\pm 0.046(0/5)$
Emission model	$0.600\pm 0.033$	$0.440\pm 0.045$	$0.861\pm 0.024$	—
Learned temporal smoothing	$0.615\pm 0.031$	$0.469\pm 0.036$	$0.879\pm 0.018$	$+0.029\pm 0.020(4/5)$
MRF	$0.603\pm 0.036$	$0.462\pm 0.044$	$0.854\pm 0.028$	$+0.021\pm 0.015(4/5)$
Strict left-to-right HMM	$0.622\pm 0.045$	$0.429\pm 0.041$	$0.816\pm 0.014$	$-0.011\pm 0.023(1/5)$
Banded HMM	$0.625\pm 0.044$	$0.443\pm 0.048$	$0.838\pm 0.045$	$+0.003\pm 0.019(2/5)$

Table 3: Primary Performance on Clinician-Annotated Sectors

Model	Accuracy	Macro-F1	Macro AUROC	$\Delta$ F1 (positive folds)
Class-prior reference	$0.298\pm 0.044$	$0.051\pm 0.006$	$0.500\pm 0.000$	$-0.399\pm 0.047(0/5)$
Emission model	$0.656\pm 0.035$	$0.450\pm 0.045$	$0.867\pm 0.031$	—
Learned temporal smoothing	$0.665\pm 0.034$	$0.468\pm 0.039$	$0.883\pm 0.022$	$+0.017\pm 0.013(4/5)$
MRF	$0.641\pm 0.040$	$0.457\pm 0.046$	$0.859\pm 0.035$	$+0.007\pm 0.014(4/5)$
Strict left-to-right HMM	$0.700\pm 0.030$	$0.450\pm 0.034$	$0.827\pm 0.015$	$+0.000\pm 0.020(2/5)$
Banded HMM	$0.691\pm 0.036$	$0.461\pm 0.046$	$0.852\pm 0.048$	$+0.010\pm 0.014(3/5)$

Table 4: Secondary Performance on All Labeled Sectors

Figure 2: Qualitative examples of spatially resolved tissue recognition. Columns show the optical-entry frame, clinician annotations, and model predictions on the polar grid. The upper row contains subcutaneous fat (yellow), anterior rectus sheath (cyan), and rectus muscle (red); the lower row contains peritoneum (purple) and intraperitoneal fat (green). Clinician-drawn boundaries vary within sectors, whereas predictions assign one tissue class to each sector. These examples illustrate the spatial representation and do not establish pixel-level segmentation accuracy.

Model	Changes/sector-min	Backward share
Clinician keyframes	$5.9\pm 1.1$	$0.148\pm 0.039$
Emission model	$66.2\pm 3.7$	$0.477\pm 0.006$
Learned temporal smoothing	$43.0\pm 4.5$	$0.452\pm 0.010$
MRF	$61.2\pm 3.0$	$0.472\pm 0.007$
Strict left-to-right HMM	$6.1\pm 1.5$	$0.002\pm 0.001$
Banded HMM	$10.4\pm 1.5$	$0.248\pm 0.037$

Table 5: Temporal Coherence and Peritoneum-or-Deeper Onset Recognition

Figure 3: Class-specific recognition for seven of the nine tissue classes. Gray bars show the emission baseline. Red bars, labeled “Hidden markov model,” represent the complete pipeline: learned temporal smoothing, MRF regularization, banded HMM inference, and blending. Their values correspond to the full-pipeline column of Table 6 . Percentages give each class’s share of the 29,151 clinician-annotated sectors. F1 values pool held-out predictions across folds. Scarpa’s fascia and preperitoneal fat are omitted from this plot and retained in the table.

Tissue	Sectors	Emission	Temporal smoother	MRF–HMM–blend	Full pipeline
Subcutaneous fat	2,373	0.590	0.646	0.637	0.676
Scarpa’s fascia	1,082	0.229	0.381	0.313	0.455
Anterior sheath	2,406	0.515	0.519	0.543	0.572
Rectus muscle	7,876	0.768	0.756	0.778	0.778
Posterior sheath	4,517	0.481	0.501	0.525	0.555
Preperitoneal fat	537	0.018	0.015	0.025	0.024

Table 6: Pooled Class-Specific F1 on Clinician-Annotated Sectors

Features	Dimensions	Accuracy	Macro-F1	Macro AUROC
No context	180	$0.514\pm 0.033$	$0.402\pm 0.032$	$0.833\pm 0.022$
PCA-15 context	255	$0.578\pm 0.027$	$0.428\pm 0.038$	$0.847\pm 0.029$
Moments + glare	245	$0.600\pm 0.033$	$0.440\pm 0.045$	$0.861\pm 0.024$

Table 7: Complete Feature-Context Ablation Results

Alternative	Accuracy	Macro-F1	Macro AUROC
Hierarchical frame-depth prior ( $\alpha=1$ )	$0.576\pm 0.039$	$0.439\pm 0.045$	$0.854\pm 0.020$
Learned spatial stacking	$0.571\pm 0.040$	$0.415\pm 0.036$	$0.838\pm 0.017$
Spatial stacking + banded HMM	$0.613\pm 0.041$	$0.431\pm 0.036$	$0.836\pm 0.038$
MRF + per-class decision bias	$0.602\pm 0.040$	$0.471\pm 0.044$	$0.859\pm 0.028$
HMM emission exponent $\eta<1$	—	—	—

Table 8: Alternative Spatial Models, Decision Biases, and Emission Weighting

Experiment	Setting or comparison	Reported outcome
MRF coupling	$\beta\in[0.25,1]$	Macro-F1 gain over emissions retained
Geometric blend without temporal stacking	$w=0.25,0.5,0.75$	All tested weights improved accuracy, macro-F1, and AUROC over emissions
HMM emission discounting	$\eta=0.1,0.25,0.5$	No tested value improved macro-F1
Posterior vs path decoding	Forward–backward vs Viterbi	Differences within $\pm 0.01$ on label metrics
Hierarchical evidence weighting	$\alpha\leq 0.5$	Collapsed toward rare classes
PCA context vs adopted context	255 vs 245 dimensions	Macro-F1 change $-0.012$ ; 2/5 folds improved

Table 9: Additional Ablations, Parameter Sweeps, and Calibration Diagnostics

Explore similar work

Training-free Temporal Object Tracking in Surgical Videos
Mar 8, 2026Subhadeep Koley, Abdolrahim Kadkhodamohammadi, Santiago Barbarisi +2Visual Object Tracking Surgical Video Understanding
Medtronic plc., London, UK. · UCL Hawkes Institute, University College London, London, UK.
TRUST: Efficient Abdominal Trauma Recognition via Image-to-Ultrasound-Video Transfer Learning
Jun 26, 2026Enguang Wang, Hao Zhou, Shuo Gao +2Medical Diagnosis Ultrasound Imaging
School of Biological Science and Medical Engineering, Southeast University, Nanjing, China · Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University, Nanjing, China · State Key Laboratory of Digital Medical Engineering, Southeast University, Nanjing, China +1
Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition
May 11, 2026Yang Liu, Ning Zhu, Jingjing Peng +4Surgical Video Understanding Temporal Consistency
King’s College London, London, UK · University of Electronic Science and Technology of China, Chengdu, China · Mach Drive

Mar 8, 2026J/K move · Enter open · S save

Training-free Temporal Object Tracking in Surgical Videos

Subhadeep Koley, Abdolrahim Kadkhodamohammadi, Santiago Barbarisi, Danail Stoyanov+1

Medtronic plc., London, UK. · UCL Hawkes Institute, University College London, London, UK.

Visual Object Tracking Surgical Video Understanding Video Object Segmentation

Open paper AskPDF arXiv

Purpose: In this paper, we present a novel approach for online object tracking in laparoscopic cholecystectomy (LC) surgical videos, targeting localisation and tracking of critical anatomical structures and instruments. Our method addresses the challenges of costly pixel-level annotations and label inconsistencies inherent in existing datasets. Methods: Leveraging the inherent object localisation capabilities of pre-trained text-to-image diffusion models, we extract representative features from surgical frames without any training or fine-tuning. Our tracking framework uses these features, along with cross-frame interactions via an affinity matrix inspired by query-key-value attention, to ensure temporal continuity in the tracking process. Results: Through a pilot study, we first demonstrate that diffusion features exhibit superior object localisation and consistent semantics across different decoder levels and temporal frames. Later, we perform extensive experiments to validate the effectiveness of our approach, showcasing its superiority over competitors for the task of temporal object tracking. Specifically, we achieve a per-pixel classification accuracy of 79.19%, mean Jaccard Score of 56.20%, and mean F-Score of 79.48% on the publicly available CholeSeg8K dataset. Conclusion: Our work not only introduces a novel application of text-to-image diffusion models but also contributes to advancing the field of surgical video analysis, offering a promising avenue for accurate and cost-effective temporal object tracking in minimally invasive surgery videos.

Abstract

Explore similar work

Training-free Temporal Object Tracking in Surgical Videos

TRUST: Efficient Abdominal Trauma Recognition via Image-to-Ultrasound-Video Transfer Learning

Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition