Label-Free Deep-Tissue Peripheral Nerve Detection with a Handheld Multimodal OCT Probe and NerveDetNet
Authors: Yihan Wang, Ruilin You, Shaobai Li, Jiabin Chen, Bofan Song, Anh D. Le, Rongguang Liang
Organizations: Wyant College of Optical Sciences, University of Arizona, Tucson, Arizona 85721, USA · Department of Oral & Maxillofacial Surgery, University of Pennsylvania School of Dental Medicine, Philadelphia, PA 19104, USA
Peripheral nerves buried beneath intact tissue are difficult to visualize during surgery and remain inaccessible to white light wide-field imaging and other surface optical imaging methods. Existing OCT nerve studies have largely relied on exposed nerves or polarization contrast with limited depth penetration, restricting their value for subsurface intraoperative guidance. Here, we introduce, to our knowledge, the first label-free framework for detecting peripheral nerves beneath unopened tissue and resolving their depth using intensity-based OCT structural signatures alone. The framework combines a handheld multimodal probe, integrating swept-source OCT with co-registered white light and autofluorescence imaging, with a ``confirm-then-capture'' workflow designed for practical surgical use. To enable efficient analysis of sparsely sampled OCT volumes, we develop NerveDetNet, a lightweight 2.5D segmentation network that recovers weak and spatially displaced nerve signals by incorporating spatial context, frame-order information, and shift-tolerant correlations across frames through a dedicated nerve feature correlation module. In ex vivo tissue experiments, NerveDetNet consistently outperformed six representative 2D baselines across all frame spacings, achieving a Dice score of 0.725 under the sparsest sampling condition while using approximately half the model parameters. End-to-end validation demonstrated localization of nerves invisible at the surface and depth-resolved detection up to 1.3--1.4~mm below the tissue surface, with OCT derived depth maps overlaid directly onto the surgical view. Together, these results establish a practical label-free approach for subsurface nerve visualization that supports intraoperative compatibility, enables efficient sparse-volume analysis, and provides depth-resolved guidance without tissue opening, contrast agents, or nerve exposure.
Figures & tables
Fig. 1: Schematic of the optical system of the handheld multimodal OCT probe.
Fig. 2: Pipeline of multimodal OCT peripheral nerve detection. (a) Distortions are corrected via a calibrated displacement field. (b) NerveDetNet segments peripheral nerves from the corrected volumes or sparse frames. (c) In parallel, the tissue surface is detected. (d) The nerve detected volume with the segmented region overlaid. (e) Combining the nerve region with the tissue surface yields the subsurface nerve depth map. (f) Results for visualization and cross-validation: autofluorescence (left), white light fused with the OCT-derived depth map (center and right).
Fig. 3: OCT signatures and spatial coherence of peripheral nerve. (a) Surface-registered B-scans separated by 5 frames and corresponding normalized difference maps. (b) Long-axis intensity profiles showing a shared spatial-frequency peak within the Bands-of-Fontana period range. (c) CLAHE-enhanced en-face projection showing periodic nerve texture. (d) Autofluorescence of exposed nerve under 365 nm excitation.
Fig. 4: Overview of NerveDetNet for sparse-frame OCT nerve detection. (a) Sparse B-scan stack and frame-wise prediction. (b) NFCM guidance extraction. (c) NFCM-guided decoder fusion. (d) Main and consensus logit fusion. (e) Compact nested encoder–decoder topology with guidance applied along the final high-resolution path.
Fig. 5: Five-fold Dice benchmark for sparse-frame OCT nerve segmentation. Bars show mean ± standard deviation, and points show individual folds. SF denotes the skipped B-scans between frames in the five-frame input stack.
Fig. 6: Representative validation case at SF=23 . (a) B-scan predictions with slice Dice scores. (b) En-face projection after sparse-frame reconstruction. (c) 3D OCT rendering with predicted nerve masks. NerveDetNet shows more continuous nerve recovery.
Fig. 7: Additional shallow and deep nerve examples. (a,b) B-scan and 3D rendering for a shallow nerve. (c,d) Deep-tissue case with baseline under-segmentation. Dice scores are shown below predictions.
Model
SF=0
SF=5
SF=11
SF=23
Time ↓ (s/vol.)
U-Net
0.756 ± 0.057
0.750 ± 0.057
0.727 ± 0.061
0.658 ± 0.080
0.098
U-Net++
0.781 ± 0.044
0.772 ± 0.045
0.739 ± 0.069
0.700 ± 0.079
0.316
SegResNet
0.758 ± 0.069
0.768 ± 0.047
0.736 ± 0.066
0.707 ± 0.073
0.086
DeepLabV3+
0.739 ± 0.065
0.722 ± 0.070
0.708 ± 0.066
0.663 ± 0.078
0.042
Attention U-Net
0.751 ± 0.064
0.735 ± 0.059
0.716 ± 0.068
0.672 ± 0.069
0.246
TransUNet-lite
0.738 ± 0.060
0.730 ± 0.065
0.705 ± 0.070
0.660 ± 0.060
0.238
TABLE I: Five-fold Dice and network inference time.
Model
SF=0 ↓
SF=5 ↓
SF=11 ↓
SF=23 ↓
U-Net
2.151 ± 0.018
0.697 ± 0.012
0.390 ± 0.007
0.256 ± 0.011
U-Net++
4.600 ± 0.018
1.538 ± 0.012
0.817 ± 0.008
0.471 ± 0.007
SegResNet
2.048 ± 0.032
0.643 ± 0.006
0.379 ± 0.009
0.242 ± 0.009
DeepLabV3+
1.508 ± 0.025
0.476 ± 0.015
0.288 ± 0.009
0.214 ± 0.014
Attention U-Net
3.836 ± 0.032
1.273 ± 0.012
0.687 ± 0.010
0.400 ± 0.005
TransUNet-lite
3.733 ± 0.023
1.239 ± 0.011
0.673 ± 0.011
0.397 ± 0.008
TABLE II: Anchor-interpolated inference time by frame spacing.
Model
SF=0
SF=5
SF=11
SF=23
IoU
U-Net
.670±.066(.697)
.672±.063(.699)
.659±.063(.700)
.593±.091(.619)
U-Net++
.697±.055(.707)
.674±.081(.712)
.673±.076(.697)
.643±.075(.693)
SegResNet
.674±.067(.704)
.688±.062(.702)
.668±.067(.698)
.631±.074(.673)
DeepLabV3+
.644±.082(.679)
.634±.083(.653)
.637±.075(.689)
.594±.086(.631)
Attention U-Net
.664±.079(.688)
.655±.075(.681)
.648±.071(.681)
.603±.059(.640)
TABLE III: Volumetric segmentation results after sparse-frame reconstruction. Values are mean ± SD, with median in parentheses.
Fig. 8: End-to-end label-free nerve detection with the handheld multimodal OCT probe. (a) OCT geometric correction and white light registration using tissue and grid targets. (b) B-scan tissue-surface detection, showing gradient response, interpolated profile, and final surface. (c) Depth-resolved subsurface nerve detection in representative B-scans and en-face depth fusion. Color indicates nerve depth below the tissue surface; scale bar, 2 mm.
Optical coherence tomography (OCT) offers compact, contactless, micron-scale imaging suitable for intraoperative guidance, but native OCT volumes are discretely sampled, anisotropic, and currently inefficient for downstream geometric reasoning and robot integration. We present OCTN (pronounced "octane"), an implicit neural representation framework that converts volumetric OCT scans into a continuous, differentiable, and spatially faithful tissue-intensity field. OCTN uses a two-stage hybrid training strategy that combines supervision from acquired voxels with inter-slice interpolations, preserving B-scan fidelity while improving continuity in sparsely sampled regions. For versatility, we first show that OCTN enables fast volumetric reasoning by storing the learned tissue representation natively on the GPU, supporting intensity-based spatial queries with up to 43x speedup over conventional CPU processing. We then demonstrate OCTN-enabled OCT-guided robotic laser surgery where the continuous tissue representation supports implicit surface discovery and surface-constrained path planning via multiple optimization strategies, including Newton- and SGD-based optimization. Next, OCTN enables reconstruction of dense volumetric structure from sparsely acquired B-scans, while reducing acquisition time by 4x and preserving clinically relevant structures. Across the newly generated Duke TissueOCT dataset and public OCT datasets, OCTN achieves robust, high-fidelity reconstruction with PSNR > 30 dB and training time < 10 s, while preserving surface consistency within 10 μm Chamfer distance relative to baseline reconstruction. The TissueOCT dataset and code are publicly available at raprakashvi.github.io/octn
Ravi Prakash, Ryan P. McNabb, Patrick J. Codd +1
Duke University, Durham, NC 27705 USA · Arizona State University, Phoenix, AZ 85287 USA
Purpose: The integration of multimodal imaging into operating rooms paves the way for comprehensive surgical scene understanding. In ophthalmic surgery, by now, two complementary imaging modalities are available: operating microscope (OPMI) imaging and real-time intraoperative optical coherence tomography (iOCT). This first work toward temporal OPMI and iOCT feature fusion demonstrates the potential of multimodal image processing for multi-head prediction through the example of precise instrument tracking in vitreoretinal surgery. Methods: We propose a multimodal, temporal, real-time capable network architecture to perform joint instrument detection, keypoint localization, and tool-tissue distance estimation. Our network design integrates a cross-attention fusion module to merge OPMI and iOCT image features, which are efficiently extracted via a YoloNAS and a CNN encoder, respectively. Furthermore, a region-based recurrent module leverages temporal coherence. Results: Our experiments demonstrate reliable instrument localization and keypoint detection (95.79% mAP50) and show that the incorporation of iOCT significantly improves tool-tissue distance estimation, while achieving real-time processing rates of 22.5 ms per frame. Especially for close distances to the retina (below 1 mm), the distance estimation accuracy improved from 284 μm (OPMI only) to 33 μm (multimodal). Conclusion: Feature fusion of multimodal imaging can enhance multi-task prediction accuracy compared to single-modality processing and real-time processing performance can be achieved through tailored network design. While our results demonstrate the potential of multi-modal processing for image-guided vitreoretinal surgery, they also underline key challenges that motivate future research toward more reliable, consistent, and comprehensive surgical scene understanding.
Nikolo Rohrmoser, Ghazal Ghazaei, Michael Sommersperger +1
Chair for Computer Aided Medical Procedures, Technical University of Munich, Garching, Germany. · Corporate Research and Technology, Carl Zeiss AG, Munich, Germany.
Reliable tool--tissue contact detection can support interaction-aware control and downstream force estimation in robot-assisted surgery. Most existing methods learn a contact classifier from RGB appearance, which is hard to generalize. In this work, we use the depth image generated from a stereo pair to give more information about tool--tissue contact. For each depth frame, we localize a spatially supported minimum-distance patch around the tool boundary and reduce it to a single scalar, −log10∣d∣; this signal rises and falls in step with ground-truth contact. We formalize this observation with a fully supervised two-state hidden Markov model. We fit this model as a six-fold leave-one-session-out (LOSO) ensemble on six palpation sessions against a single silicone cup-like phantom, with the decision threshold selected from the pooled out-of-fold predictions. It is evaluated on four held-out sessions of three categories: 1. same task on same phantom; 2. same task on different phantom; 3. different task on different phantom. This model reaches held-out macro F1 0.927 and AUPRC 0.980. We further compare against a reproduction of an RGB-based contact classifier from prior work. This RGB-based model achieves high performance on the first category (F1 0.965), but substantially lower performance on the other two, resulting in macro F1 0.320 across all four sessions. These results indicate that the tool--tissue distance is a strong, transferable cue for contact detection in robot-assisted surgery.
Mingyeung Wu, Zhonghao Zhang, Hao Yang +2
Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, United States · Department of Computer Science, Vanderbilt University, Nashville, TN, United States