MISTac: A Vision-Based Tactile Sensor for Minimally Invasive Surgery

Robin Koch; Annabella Mascot; Rayan Younis; Martin Wagner; Stefanie Speidel; Mark Cutkosky; Ingo Sieber; Roberto Calandra

cs.ROAug 14, 2026

MISTac: A Vision-Based Tactile Sensor for Minimally Invasive Surgery

Authors: Robin Koch, Annabella Mascot, Rayan Younis, Martin Wagner, Stefanie Speidel, Mark Cutkosky, Ingo Sieber, Roberto Calandra

Abstract

Minimally invasive and robot-assisted surgery offer many advantages over traditional open surgery, but deprive surgeons of tactile feedback and the ability to palpate tissue with their fingers. To address this lack of tactile feedback, we introduce the MISTac, a high resolution vision-based tactile sensor specifically designed for palpation in MIS. The sensor has a replaceable sensor tip with a diameter of 8 mm which allows it to fit through the trocars used in minimally invasive surgery. Its modular 3D-printed case design allows the use of bulky off-the-shelf illumination and imaging hardware that can easily be exchanged and upgraded. The sensor has an optical resolution of 176.68 $μm$ , a tactile resolution of 250 $μm$ , and can resolve forces as little as 24.3 mN. An in vivo study with the sensor shows its usability in minimally invasive surgery. We trained a machine learning model with the tactile data collected in the trial on a tissue classification task achieving an aggregate accuracy of ~84% in a leave-one-out cross validation. Tactile sensors have the potential to one day aid surgeons during minimally invasive surgery with tasks such as tissue classification or intra-operative tumor localization; MISTac is a small step towards this vision. We open-source MISTac at https://github.com/lasr-lab/mistac

Figures & tables

Sensor/Category	Tip Diameter	Optical Resolution	Tactile Resolution	Force Resolution	Flexibility	Modularity	Housing	Dynamic illumination
DIGIT Pinki [ 7 ]	$15\text{\,}\mathrm{mm}$	$222.72\text{\,}\mathrm{\SIUnitSymbolMicro m}$	$250\text{\,}\mathrm{\SIUnitSymbolMicro m}$	$5\text{\,}\mathrm{mN}$	flexible	fully modular, removable tips	NA	not implemented
MiniTac [ 8 ]	$8\text{\,}\mathrm{mm}$	$10\text{\,}\mathrm{\SIUnitSymbolMicro m}$ *	NA	$0.6\text{\,}\mathrm{mN}$ **	rigid	not modular, tips not replaceable	metal shell with carbon handle	not possible
TacScope [ 9 ]	$7\text{\,}\mathrm{mm}$	$55.68\text{\,}\mathrm{\SIUnitSymbolMicro m}$	$250\text{\,}\mathrm{\SIUnitSymbolMicro m}$	NA	rigid	modular	3D-printed Shell	not possible
EndoTac [ 10 ]	$8\text{\,}\mathrm{mm}$	NA	NA	$23.97\text{\,}\mathrm{mN}$ **	flexible	not modular	NA	not implemented
MISTac (Ours)	$8\text{\,}\mathrm{mm}$	$176.68\text{\,}\mathrm{\SIUnitSymbolMicro m}$	$250\text{\,}\mathrm{\SIUnitSymbolMicro m}$	$24.3\text{\,}\mathrm{mN}$	flexible	fully modular, removable tips	3D-printed, fully modular	possible

Table I : Comparison of various VBTSs for tissue palpation with the MISTac. *Nominal camera resolution, not experimentally verified. **Pixel-level force-resolution.

Figure 1: Sensor Overview. The MISTac consists of a gel sensor tip at the distal end and an illumination system and an imaging system located at the proximal end. The sensor case comprises four modular subassemblies: the collimator, fiber holder, microscope holder, and camera holder. The fiber holder is connected to the microscope holder through an adjustment mechanism. The modular 3D-printed case enables easy repair, customization, and upgrades of the imaging system.

Panel of Figure 2 — Figure 2: Distal End. The built-in illumination fibers of the fiberscope are sorted into 6 illumination zones (a). This allows to miniaturize the sensor tip which attaches to the fiber bundle and is fastened with a 3D-printed nut (b). A cross section of the $8\text{\,}\mathrm{mm}$ sensor tip is shown in (c).

Panel of Figure 3 — Figure 3: Tactile measurements of the MISTac under dynamic illumination. Setting the LEDs to different colors (a)-(e) illuminates the test object from slightly different angles. This highlights different structures of the test object shown in (f).

Panel of Figure 4 — Figure 4: The distal end of the MISTac is mounted on a test bench made from an optical breadboard and 3D-printed parts, where it images a USAF 1951 resolution target (a). The resulting image resolves element 4 of group 1, corresponding to an optical resolution of $176.68\text{\,}\mathrm{\SIUnitSymbolMicro m}$ . A scratch caused by fiber damage during assembly is visible and extends from the edge toward the green rectangle (b). Tactile resolution is evaluated using a milled aluminum block with line pairs at four different spacings (c). The tactile output shows that the MISTac has a tactile resolution of $250\text{\,}\mathrm{\SIUnitSymbolMicro m}$ (d).

Panel of Figure 5 — Figure 5: Our data-collection setup consists of an x-arm 7 robot arm with the MISTac’s distal end attached. While recording tactile readings, the robot presses the sensor tip against a $1.2\text{\,}\mathrm{mm}$ metal probe mounted on a BotaSys SensOne fts which provides the ground truth force values (a). The predicted vs. actual normal force [N] is shown in (b). Our modified ResNet18 model achieves an RMSE of $24.3\text{\,}\mathrm{mN}$ ± $0.9\text{\,}\mathrm{mN}$ .

Figure 6: The MISTac used in an minimally invasive in vivo trial. The bottom image shows endoscope footage of the sensor tip touching a cauterized liver cut. The raw sensor output in the top left clearly shows the cut surface. Healthy liver tissue, shown in the top right, produces only subtle features in the raw output. Subtracting a neutral image and normalizing the output highlights the surface features in the corresponding heatmaps.

Figure 7: Row-normalized confusion matrix after leave-one-out cross validation. Our model classifies tactile readings of colon, healthy liver tissue, a cauterized liver cut and spleen. The results indicate that the MISTac’s tactile readings can be used to perform tissue classification tasks.

Explore similar work

MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery
Jul 21, 2026Md Rakibul Islam Prince, Jaeeun Kim, Yuhao Zhou +9Vision-Based Tactile Sensing Tactile Sensing
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA · Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA · School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA +3
TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception
Jun 24, 2026Lan Wei, Gurmeher Khurana, Sirine Bhouri +6Vision-Based Tactile Sensing Few-Shot Learning
Imperial College London, UK. · Queen Mary University of London, UK · King’s College London, UK
Hall Effect-Based Tactile Force Detection Sensor for Robot-Assisted Minimally Invasive Surgery
Oct 7, 2026Charles DeLorey, Michael Brockdorff, Adam Metcalf +3Tactile Sensing Robotic Tactile Sensing
STORM Lab, School of Electronic and Electrical Engineering, University of Leeds, Leeds, UK

Jul 21, 2026, cs.ROJ/K move · Enter open · S save

MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery

Md Rakibul Islam Prince, Jaeeun Kim, Yuhao Zhou, Mason Vrshek+8

Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA · Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA · School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA+3

Vision-Based Tactile Sensing Tactile Sensing Surgical Robotics

Open paper AskPDF arXiv

Robot-assisted minimally invasive surgery (RMIS) offers major benefits over open and conventional laparoscopic procedures, yet it still lacks tactile feedback for palpation while operating under strict requirements to preserve reliable vision for navigation and safety. In practice, visual feedback is indispensable, and tactile solutions that cannot coexist with vision are difficult to translate into RMIS tools. To address both needs, we introduce MVP-Tac, a compact, vision-based tactile sensor that provides co-located vision and tactile sensing. MVP-Tac uses reflective photoelastic imaging: a thin photoelastic elastomer produces stress-dependent interferograms under contact that are captured by an embedded camera through a miniaturized reflective polariscope. A semi-transparent membrane and controllable illumination enable switching between visual mode and tactile mode, enabling tactile perception without sacrificing vision. We validate MVP-Tac through force calibration in the 0 to 2 N range and demonstrate its potential for tumor palpation via video-based hardness classification on tissue phantoms, achieving 97% accuracy for exposed-tumor classification and 92% accuracy for subdermal-tumor classification. Finally, we conduct a simulated colonoscopy to validate both visual and tactile modalities in a constrained lumen, including vision-guided 3D photomapping of the luminal wall and in situ hardness classification of localized nodules. Overall, MVP-Tac provides a practical path toward restoring clinically useful palpation in RMIS while maintaining essential visual feedback. The design, fabrication, and firmware of MVP-Tac are open-sourced at https://mvp-tac.github.io/

Abstract

Explore similar work

MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery

TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception

Hall Effect-Based Tactile Force Detection Sensor for Robot-Assisted Minimally Invasive Surgery