Label-Free Deep-Tissue Peripheral Nerve Detection with a Handheld Multimodal OCT Probe and NerveDetNet

Yihan Wang; Ruilin You; Shaobai Li; Jiabin Chen; Bofan Song; Anh D. Le; Rongguang Liang

physics.opticsAug 13, 2026

Label-Free Deep-Tissue Peripheral Nerve Detection with a Handheld Multimodal OCT Probe and NerveDetNet

Authors: Yihan Wang, Ruilin You, Shaobai Li, Jiabin Chen, Bofan Song, Anh D. Le, Rongguang Liang

Organizations: Wyant College of Optical Sciences, University of Arizona, Tucson, Arizona 85721, USA · Department of Oral & Maxillofacial Surgery, University of Pennsylvania School of Dental Medicine, Philadelphia, PA 19104, USA

Abstract

Peripheral nerves buried beneath intact tissue are difficult to visualize during surgery and remain inaccessible to white light wide-field imaging and other surface optical imaging methods. Existing OCT nerve studies have largely relied on exposed nerves or polarization contrast with limited depth penetration, restricting their value for subsurface intraoperative guidance. Here, we introduce, to our knowledge, the first label-free framework for detecting peripheral nerves beneath unopened tissue and resolving their depth using intensity-based OCT structural signatures alone. The framework combines a handheld multimodal probe, integrating swept-source OCT with co-registered white light and autofluorescence imaging, with a ``confirm-then-capture'' workflow designed for practical surgical use. To enable efficient analysis of sparsely sampled OCT volumes, we develop NerveDetNet, a lightweight 2.5D segmentation network that recovers weak and spatially displaced nerve signals by incorporating spatial context, frame-order information, and shift-tolerant correlations across frames through a dedicated nerve feature correlation module. In ex vivo tissue experiments, NerveDetNet consistently outperformed six representative 2D baselines across all frame spacings, achieving a Dice score of 0.725 under the sparsest sampling condition while using approximately half the model parameters. End-to-end validation demonstrated localization of nerves invisible at the surface and depth-resolved detection up to 1.3--1.4~mm below the tissue surface, with OCT derived depth maps overlaid directly onto the surgical view. Together, these results establish a practical label-free approach for subsurface nerve visualization that supports intraoperative compatibility, enables efficient sparse-volume analysis, and provides depth-resolved guidance without tissue opening, contrast agents, or nerve exposure.

Figures & tables

Fig. 1: Schematic of the optical system of the handheld multimodal OCT probe.

Fig. 2: Pipeline of multimodal OCT peripheral nerve detection. (a) Distortions are corrected via a calibrated displacement field. (b) NerveDetNet segments peripheral nerves from the corrected volumes or sparse frames. (c) In parallel, the tissue surface is detected. (d) The nerve detected volume with the segmented region overlaid. (e) Combining the nerve region with the tissue surface yields the subsurface nerve depth map. (f) Results for visualization and cross-validation: autofluorescence (left), white light fused with the OCT-derived depth map (center and right).

Fig. 3: OCT signatures and spatial coherence of peripheral nerve. (a) Surface-registered B-scans separated by 5 frames and corresponding normalized difference maps. (b) Long-axis intensity profiles showing a shared spatial-frequency peak within the Bands-of-Fontana period range. (c) CLAHE-enhanced en-face projection showing periodic nerve texture. (d) Autofluorescence of exposed nerve under $365$ nm excitation.

Fig. 4: Overview of NerveDetNet for sparse-frame OCT nerve detection. (a) Sparse B-scan stack and frame-wise prediction. (b) NFCM guidance extraction. (c) NFCM-guided decoder fusion. (d) Main and consensus logit fusion. (e) Compact nested encoder–decoder topology with guidance applied along the final high-resolution path.

Fig. 5: Five-fold Dice benchmark for sparse-frame OCT nerve segmentation. Bars show mean $\pm$ standard deviation, and points show individual folds. $SF$ denotes the skipped B-scans between frames in the five-frame input stack.

Fig. 6: Representative validation case at $\mathrm{SF}=23$ . (a) B-scan predictions with slice Dice scores. (b) En-face projection after sparse-frame reconstruction. (c) 3D OCT rendering with predicted nerve masks. NerveDetNet shows more continuous nerve recovery.

Fig. 7: Additional shallow and deep nerve examples. (a,b) B-scan and 3D rendering for a shallow nerve. (c,d) Deep-tissue case with baseline under-segmentation. Dice scores are shown below predictions.

Model	SF=0	SF=5	SF=11	SF=23	Time $\downarrow$ (s/vol.)
U-Net	0.756 $\pm$ 0.057	0.750 $\pm$ 0.057	0.727 $\pm$ 0.061	0.658 $\pm$ 0.080	0.098
U-Net++	0.781 $\pm$ 0.044	0.772 $\pm$ 0.045	0.739 $\pm$ 0.069	0.700 $\pm$ 0.079	0.316
SegResNet	0.758 $\pm$ 0.069	0.768 $\pm$ 0.047	0.736 $\pm$ 0.066	0.707 $\pm$ 0.073	0.086
DeepLabV3+	0.739 $\pm$ 0.065	0.722 $\pm$ 0.070	0.708 $\pm$ 0.066	0.663 $\pm$ 0.078	0.042
Attention U-Net	0.751 $\pm$ 0.064	0.735 $\pm$ 0.059	0.716 $\pm$ 0.068	0.672 $\pm$ 0.069	0.246
TransUNet-lite	0.738 $\pm$ 0.060	0.730 $\pm$ 0.065	0.705 $\pm$ 0.070	0.660 $\pm$ 0.060	0.238

TABLE I: Five-fold Dice and network inference time.

Model	SF=0 $\downarrow$	SF=5 $\downarrow$	SF=11 $\downarrow$	SF=23 $\downarrow$
U-Net	2.151 $\pm$ 0.018	0.697 $\pm$ 0.012	0.390 $\pm$ 0.007	0.256 $\pm$ 0.011
U-Net++	4.600 $\pm$ 0.018	1.538 $\pm$ 0.012	0.817 $\pm$ 0.008	0.471 $\pm$ 0.007
SegResNet	2.048 $\pm$ 0.032	0.643 $\pm$ 0.006	0.379 $\pm$ 0.009	0.242 $\pm$ 0.009
DeepLabV3+	1.508 $\pm$ 0.025	0.476 $\pm$ 0.015	0.288 $\pm$ 0.009	0.214 $\pm$ 0.014
Attention U-Net	3.836 $\pm$ 0.032	1.273 $\pm$ 0.012	0.687 $\pm$ 0.010	0.400 $\pm$ 0.005
TransUNet-lite	3.733 $\pm$ 0.023	1.239 $\pm$ 0.011	0.673 $\pm$ 0.011	0.397 $\pm$ 0.008

TABLE II: Anchor-interpolated inference time by frame spacing.

IoU
Model	SF=0	SF=5	SF=11	SF=23
U-Net	$.670{\pm}.066(.697)$	$.672{\pm}.063(.699)$	$.659{\pm}.063(.700)$	$.593{\pm}.091(.619)$
U-Net++	$.697{\pm}.055(.707)$	$.674{\pm}.081(.712)$	$.673{\pm}.076(.697)$	$.643{\pm}.075(\mathbf{.693})$
SegResNet	$.674{\pm}.067(.704)$	$.688{\pm}.062(.702)$	$.668{\pm}.067(.698)$	$.631{\pm}.074(.673)$
DeepLabV3+	$.644{\pm}.082(.679)$	$.634{\pm}.083(.653)$	$.637{\pm}.075(.689)$	$.594{\pm}.086(.631)$
Attention U-Net	$.664{\pm}.079(.688)$	$.655{\pm}.075(.681)$	$.648{\pm}.071(.681)$	$.603{\pm}.059(.640)$

TABLE III: Volumetric segmentation results after sparse-frame reconstruction. Values are mean

\pm

SD, with median in parentheses.

Fig. 8: End-to-end label-free nerve detection with the handheld multimodal OCT probe. (a) OCT geometric correction and white light registration using tissue and grid targets. (b) B-scan tissue-surface detection, showing gradient response, interpolated profile, and final surface. (c) Depth-resolved subsurface nerve detection in representative B-scans and en-face depth fusion. Color indicates nerve depth below the tissue surface; scale bar,

2

mm.

Explore similar work

OCTN: Neural OCT Representations for Robot-Guided Precision Intervention
Sep 6, 2026Ravi Prakash, Ryan P. McNabb, Patrick J. Codd +1Optical Coherence Tomography Implicit Neural Representations
Duke University, Durham, NC 27705 USA · Arizona State University, Phoenix, AZ 85287 USA
Towards Comprehensive Real-Time Scene Understanding in Ophthalmic Surgery through Multimodal Image Fusion
Mar 26, 2026Nikolo Rohrmoser, Ghazal Ghazaei, Michael Sommersperger +1Medical Image Analysis Surgical Video Understanding
Chair for Computer Aided Medical Procedures, Technical University of Munich, Garching, Germany. · Corporate Research and Technology, Carl Zeiss AG, Munich, Germany.
Transferable Tool-Tissue Contact Detection from Stereo Depth in Robot-Assisted Surgery
Aug 18, 2026Mingyeung Wu, Zhonghao Zhang, Hao Yang +2Surgical Robotics
Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, United States · Department of Computer Science, Vanderbilt University, Nashville, TN, United States

Sep 6, 2026J/K move · Enter open · S save

OCTN: Neural OCT Representations for Robot-Guided Precision Intervention

Ravi Prakash, Ryan P. McNabb, Patrick J. Codd, Shan Lin

Duke University, Durham, NC 27705 USA · Arizona State University, Phoenix, AZ 85287 USA

Optical Coherence Tomography Implicit Neural Representations Surgical Robotics

Open paper AskPDF arXiv

Optical coherence tomography (OCT) offers compact, contactless, micron-scale imaging suitable for intraoperative guidance, but native OCT volumes are discretely sampled, anisotropic, and currently inefficient for downstream geometric reasoning and robot integration. We present OCTN (pronounced "octane"), an implicit neural representation framework that converts volumetric OCT scans into a continuous, differentiable, and spatially faithful tissue-intensity field. OCTN uses a two-stage hybrid training strategy that combines supervision from acquired voxels with inter-slice interpolations, preserving B-scan fidelity while improving continuity in sparsely sampled regions. For versatility, we first show that OCTN enables fast volumetric reasoning by storing the learned tissue representation natively on the GPU, supporting intensity-based spatial queries with up to 43x speedup over conventional CPU processing. We then demonstrate OCTN-enabled OCT-guided robotic laser surgery where the continuous tissue representation supports implicit surface discovery and surface-constrained path planning via multiple optimization strategies, including Newton- and SGD-based optimization. Next, OCTN enables reconstruction of dense volumetric structure from sparsely acquired B-scans, while reducing acquisition time by 4x and preserving clinically relevant structures. Across the newly generated Duke TissueOCT dataset and public OCT datasets, OCTN achieves robust, high-fidelity reconstruction with PSNR > 30 dB and training time < 10 s, while preserving surface consistency within 10

μ

m Chamfer distance relative to baseline reconstruction. The TissueOCT dataset and code are publicly available at raprakashvi.github.io/octn

Abstract

Explore similar work

OCTN: Neural OCT Representations for Robot-Guided Precision Intervention

Towards Comprehensive Real-Time Scene Understanding in Ophthalmic Surgery through Multimodal Image Fusion

Transferable Tool-Tissue Contact Detection from Stereo Depth in Robot-Assisted Surgery