cs.CVMar 7, 2026

Virtual Intraoperative CT (viCT): Sequential Anatomic Updates for Modeling Tissue Resection Throughout Endoscopic Sinus Surgery

Authors: Nicole M. Gunderson, Graham J. Harris, Jeremy S. Ruthberg, Pengcheng Chen, Di Mao, Randall A. Bly, Waleed M. Abuzeid, Eric J. Seibel

Organizations: Department of Mechanical Engineering, University of Washington, Seattle, WA, USA. · Department of Otolaryngology–Head and Neck Surgery, University of Washington Medical Center, University of Washington, Seattle, WA, USA.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Purpose: Incomplete dissection is a common cause of persistent disease and revision endoscopic sinus surgery (ESS) in chronic rhinosinusitis. Current image-guided surgery systems typically reference static preoperative CT (pCT), and do not model evolving resection boundaries. We present Virtual Intraoperative CT (viCT), a method for sequentially updating pCT throughout ESS using intraoperative 3D reconstructions from monocular endoscopic video to enable visualization of evolving anatomy in CT format. Methods: Monocular endoscopic video is processed using a depth-supervised NeRF framework with virtual stereo synthesis to generate metrically scaled 3D reconstructions at multiple surgical intervals. Reconstructions undergo rigid, landmark-based registration in 3D Slicer guided by anatomical correspondences, and are then voxelized into the pCT grid. viCT volumes were generated using a ray-based occupancy comparison between pCT and reconstruction to delete outdated voxels and remap preserved anatomy and updated boundaries. Performance is evaluated in a cadaveric feasibility study of four specimens across four ESS stages using volumetric overlap (DSC, Jaccard) and surface metrics (HD95, Chamfer, MSD, RMSD), and qualitative comparisons to ground-truth CT. Results: viCT updates show agreement with ground-truth anatomy across surgical stages, with submillimeter mean surface errors. Dice Similarity Coefficient (DSC) = 0.88 +/- 0.05 and Jaccard Index = 0.79 +/- 0.07, and Hausdorff Distance 95% (HD95) = 0.69 +/- 0.28 mm, Chamfer Distance = 0.09 +/- 0.05 mm, Mean Surface Distance (MSD) = 0.11 +/- 0.05 mm, and Root Mean Square Distance (RMSD) = 0.32 +/- 0.10 mm. Conclusion: viCT enables CT-format anatomic updating in an ESS setting without ancillary hardware. Future work will focus on fully automating registration, validation in live cases, and optimizing runtime for real-time deployment.

Figures & tables

Figure 1: Flowchart of simulated stereoscopy NeRF-based reconstruction workflow.
Figure 2: Ray-based viCT updating workflow and example of tissue resection modeling.
(a) Post Maxillary surgical interval
(a) Post Ethmoid surgical interval
Figure 5: Qualitative comparison of viCT updates with ground-truth postoperative CT across surgical intervals. (A) Post Maxillary, (B) Post Ethmoid, and (C) Complete. Axial, coronal, and sagittal views demonstrate close correspondence between updated viCT volumes and postoperative ground-truth anatomy.
Figure 6: Summarized quantitative data across the four cadaveric specimens.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Sample monocular endoscopic views from videos used as input to the 3D reconstruction and viCT algorithms.
(a) Post Maxillary surgical interval
(a) Post Ethmoid surgical interval
Figure 10: Qualitative comparison of viCT updates with ground-truth postoperative CT across surgical intervals. (A) Post Maxillary, (B) Post Ethmoid, and (C) Complete. Axial, coronal, and sagittal views demonstrate close correspondence between updated viCT volumes and postoperative ground-truth anatomy.
(a) Post Maxillary surgical interval
(a) Post Ethmoid surgical interval
Figure 13: Qualitative comparison of viCT updates with ground-truth postoperative CT across surgical intervals. (A) Post Maxillary, (B) Post Ethmoid, and (C) Complete. Axial, coronal, and sagittal views demonstrate close correspondence between updated viCT volumes and postoperative ground-truth anatomy.
(a) Post Maxillary surgical interval
(a) Post Ethmoid surgical interval
Figure 16: Qualitative comparison of viCT updates with ground-truth postoperative CT across surgical intervals. (A) Post Maxillary, (B) Post Ethmoid, and (C) Complete. Axial, coronal, and sagittal views demonstrate close correspondence between updated viCT volumes and postoperative ground-truth anatomy.
Figure 17: Full quantitative data across the four cadaveric specimens.
More in this topic

Computed Tomography

Computed tomography uses X-ray measurements from multiple angles to reconstruct cross-sectional and volumetric images for diagnosis and scientific imaging.

See all 272 papers

Explore similar work

CardsList
Feb 17, 2026

NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy

Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.

Laura Salort-Benejam, Antonio Agudo

Institut de Rob`otica i Inform`atica Industrial, CSIC-UPC, Spain

Endoscopic 3D Reconstruction3D ReconstructionDynamic Scene ReconstructionMonocular 3D ReconstructionDeformable Shape ReconstructionNeural Radiance Fields
AskPDF
Aug 31, 2026

Multimodal Shared Latent Representation of Narration, Microscope and iOCT Images for Phase Recognition in Vitreoretinal Surgery

Surgical phase recognition is key to context-aware computer-assisted feedback in vitreoretinal procedures, yet the scarcity of synchronized multimodal intraoperative data, particularly microscope views and intraoperative OCT, limits approaches that aim to replicate the multimodal integration surgeons perform naturally. Surgical narration, by contrast, is abundantly available online and offers rich semantic supervision. Prior work has mainly explored pairwise contrastive learning (e.g., intraoperative OCT-microscope or microscope-narration), leaving the joint modeling of all three modalities largely unexplored. We introduce a framework that uses microscope views as a shared anchor to bridge surgical narrations and intraoperative OCT (iOCT) without requiring a fully synchronized tri-modal dataset, leveraging real microscope-narration videos and a synthetic dataset of synchronized microscope video and tool-aligned iOCT pairs. Contrastive alignment transfers structural priors from the synthetic domain to real videos lacking iOCT, and a dual-head MS-TCN++ integrates the resulting embeddings for joint macro- and micro-phase prediction. Evaluated on real vitreoretinal surgeries, our framework improves macro-phase recognition over a zero-shot baseline (mean F1 0.38 to 0.53) and provides an exploratory route to estimating fine-grained instrument-tissue measurements that are not directly observable in real microscope video alone; these micro-phase estimates are validated quantitatively on synthetic data and shown only qualitatively on real surgery. To our knowledge, this is the first work to unify microscope view, iOCT B-scans, and surgical narrations in a shared latent space for surgical phase recognition.

Onur Izmitlioglu, Shervin Dehghani, Tarek Ghannoum +2

Computer Aided Medical Procedures, Technical University of Munich, Germany · SynthesEyes GmbH, Munich, Germany · Department of Ophthalmology, LMU University Hospital, Munich, Germany

Surgical Video UnderstandingMultimodal Contrastive LearningMultimodal FusionOphthalmic Image AnalysisOphthalmology
AskPDF
Jun 8, 2026

Deep Slice Interpolation for Reducing Through-Plane Anisotropy and Noise in Head CT

Head computed tomography (CT) typically uses sub-millimeter in-plane resolution but 2-5 mm through-plane spacing, creating substantial anisotropy that degrades multiplanar reconstructions, volumetric measurements such as hematoma volume estimation, and downstream algorithms that assume near-isotropic voxels. We present a deep learning system that synthesizes intermediate CT slices from pairs of neighboring axial slices, halving the effective through-plane spacing. The system improves three-dimensional visualization while simultaneously producing inherently denoised outputs, yielding two complementary benefits from a single inference pass. To build a reliable system, we systematically evaluate pixel-wise losses, namely mean squared error (MSE) and mean absolute error (L1); structural-similarity losses, namely the structural similarity index (SSIM) and its multi-scale variant (MS-SSIM); and hybrid combinations. On a held-out test set, all converged models outperform classical interpolation baselines and pretrained video frame interpolation methods (RIFE, FILM) on all structural measures, with MS-SSIM+L1 offering the strongest balanced profile. We also document training instability in SSIM-family losses and identify partial remedies: the standard numerical fixes eliminate the dominant failure mode but leave residual divergence at smaller batch sizes. All results are reported with patient-level bootstrap confidence intervals and paired statistical tests. As an illustration, we apply the system to an out-of-distribution head CT series from Hospital Universitario Virgen del Rocío: the model synthesizes intermediate slices and exhibits on the real slices the implicit-denoising signature predicted by our theoretical analysis, supporting in a single external case that interpolation quality and implicit denoising are not confined to the training distribution.

Luis Cortés Ferre, Miguel A. Gutiérrez-Naranjo, Marcin Balcerzyk

Department of Computer Science and Artificial Intelligence, University of Seville, Avda. Reina Mercedes s/n, 41012 Sevilla, Spain · Bioaraba Health Research Institute, New Technologies and Information Systems in Health Research Group, 01009 Vitoria-Gasteiz, Spain · IKERBASQUE, Basque Foundation of Science, Plaza Euskadi 5, 48009 Bilbao, Spain

3D Medical ImagingMedical Image Super-ResolutionComputational ImagingComputed Tomography
AskPDF