Robotic Endoscopy

Follow this topic Ask about this topic

Latest papers

17 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Oct 22, 2026

Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy

Flexible robotic endoscopes provide access through tortuous anatomy, where friction, deformation, and wall interaction separate commanded insertion from useful distal progress. Operators interpret this mismatch and retreat to recover a navigable configuration. Recognizing the approach to that decision requires relating motor intent to distal response and the local visual scene. We introduce Multimodal Progress-Residual Fusion (MPRF), a causal model that anticipates operator retreat from commands, electromagnetic (EM) tracking, and endoscopic video. A nominal-progress model predicts the response expected in normal or successful navigation; a temporal predictor combines its EM residual with all three observed modalities. In a 150-episode colon-phantom collection, MPRF achieves 0.580 +/- 0.007 AUPRC across three seeds, compared with 0.511 for the strongest reduced-modality control. Complete multimodal histories provide the principal ranking advantage, while the residual adds an explicit intent-response representation with a small, uncertain increment over raw fusion. Analysis of frozen alarms relates warning timing to the behavioral label and explains the increase in event recall under a longer matching window. These results establish a basis for behavioral assistance during operator-controlled navigation.

Chi Kit Ng, Tianchun Wu, Jiatan Sun +3

Robotic Endoscopy Surgical Robotics Next-Action Prediction Multimodal Sensor Fusion

Oct 22, 2026

EBV-CAN: Vision--Language Grounding for Contact-Aided Endoscope Navigation

Flexible endoscopes exploit structural compliance to traverse narrow lumens, pipes, and cavities, where navigation involves distributed environmental contact. This compliance makes tip motion depend on the evolving support configuration. We present external bird's-eye-view contact-aided navigation (EBV-CAN), a vision-language policy that combines one initial global static bird's-eye image with evolving endoscopic observations and motor state. Three supervised semantic queries locate the tip, next waypoint, and instructed target on the cached global image; their residual representations predict bending and insertion commands. We introduce the Vision-Language Contact-Aided (VLCA) dataset, linking 517 demonstrations and 75k action samples to 958k spatial entries and 93k remaining-route records. On 50 held-out episodes, tip-coordinate and waypoint localization reach 97.5% and 94.8%. Swapping the target instruction under identical observations changes target attention and motor output across 17 dual-target episodes. Across 1,920 physical trials of eight policies trained under a shared protocol, EBV-CAN achieves 183/240 successes (76.25%) under automatic blue-area scoring. Gated recurrent unit behavior cloning (GRU-BC) achieves the lowest matched-task offline mean absolute error (MAE), 0.056, and 1/240 physical successes, establishing distinct rankings for action imitation and task completion. Successful multi-guide executions and representative failures connect spatial progress, steering transitions, and visibility to physical navigation.

Chi Kit Ng, Shuchang Liu, Jinsong Lin +4

The Chinese University of Hong Kong. · National University of Singapore.

Robotic Endoscopy Endoscopy Vision-Language Navigation Visual Grounding for Robotics Robot Imitation Learning

Oct 17, 2026

EndoVLA-OOV: Language-Conditioned Out-of-View Target Acquisition in Robotic Endoscopy

The scan-then-revisit workflow of upper gastrointestinal endoscopy requires bringing a previously observed suspicious region back into view for closer inspection. Acquiring such targets requires translating anatomical and appearance descriptions into camera motions from partial observations of a confined, deformable environment. We present EndoVLA-OOV, a language-conditioned vision-language-action policy for out-of-view target acquisition in robotic endoscopy. The policy adapts a pretrained backbone to continuous two-degree-of-freedom distal bending commands and combines initial, intermediate, and current images to guide search. To support policy learning, we provide an open-source VLA dataset containing 1,050 episodes across six tasks and approximately 262k observation-action pairs. Physical evaluation spans wet-lens and low-light conditions, two unseen stomach phantoms, and cross-embodiment transfer to an unseen ex-vivo environment. In the cross-embodiment evaluation, the full-data policy achieves 93.3% target acquisition success (28/30). In a controlled training-data comparison, expanding the corpus from 312 to 1,050 episodes increases acquisition success from 40.0% to 90.0% under the same model, training, and test configuration. Instruction and observation interventions further support the use of language and temporal context in the evaluated acquisition tasks. These results support language-guided endoscopic revisitation across the evaluated environments and robot embodiments and demonstrate the benefit of expanded task-specific demonstrations.

Chi Kit Ng, Yidong Zhang, Tianchun Wu +9

Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China · Department of Surgery, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong SAR, China +1

Robotic Endoscopy Vision-Based Robot Control VLMs for Robotics Surgical Robotics

Oct 9, 2026

EndoRSI: Physical-Evidence-Driven Recursive Improvement of Endoscopic Robot Policies

Artificial intelligence is beginning to automate scientific discovery, yet physical robot-policy research still relies on scientists and engineers to interpret experimental failures and decide how controllers should be revised. We present EndoRSI (Endoscopic Recursive Self-Improvement), a human-governed, self-improving robotic research system in which the embedded EndoForge coding agent iteratively develops and refines endoscopic robot policies using evidence from physical experiments. Physical execution remains isolated within a safety-constrained runtime under human authorisation. In motorised endoscopy, validation evidence first caused the agent to reject an initial action representation, while subsequent physical experiments falsified an offline-supported insertion strategy by revealing hazardous closed-loop behaviour absent from the source data. Failed physical rollouts were then incorporated into policy revision; insertion success subsequently increased from 2/21 in the first generation to 17/21 in the fourth. The frozen final policy retained competence across visual and hardware shifts and was the best performing autonomous policy across six independent ex-vivo porcine specimens. These results show that physical robot interaction can serve as experimental evidence for agent-driven robot policy research.

Chi Kit Ng, Yidong Zhang, Tianchun Wu +4

Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China

Robotic Endoscopy Robot Policy Adaptation Autonomous Research Agents Surgical Robotics Robotic Policy Evaluation

Oct 1, 2026

ColoACT: Multi-Cue Action Chunking for Smooth Autonomous Colon Navigation on a Self-Propelled Endoscopic Robot

Autonomous colonoscopic navigation can reduce operator burden and the risk of loop formation or tissue trauma, but remains challenging due to deformable anatomy, weak-texture and specular endoscopic visuals, and contact-rich viscoelastic interactions. Existing methods either rely on geometry-driven pipelines, which are efficient and interpretable yet brittle due to manually engineered features and switching logic, or adopt learning-based policies, whose inferred depth/geometry can become temporally inconsistent or overly smooth under weak texture and specular highlights while simulation-trained variants (e.g., deep reinforcement learning) may further suffer from a sim-to-real gap. We propose ColoACT, an autonomous navigation system that integrates an RGB-D-E based Action Chunking Transformer policy (ColoACT policy) for a compact self-propelled Bevel-Gear-Based Endoscopic Robot (BGER). The ColoACT policy augments RGB with estimated relative depth and a gradient-based pseudo-elevation map to enhance fold-ridge saliency and other high-frequency geometric cues, and enables smooth continuous control of the BGER by predicting overlapping action chunks and fusing them via temporal ensembling. In different ex-vivo porcine colons (approximately 60 cm), our system achieves success rates of 85.4% and 72.5% in straight and curved segments, respectively, and achieves 70% success in 90-degree turns and 60% in double-bend sequences, with feasibility further demonstrated in challenging triple-bend segments. The project page is available at: https://Adamhu1.github.io/ColoACT/.

Jian Hu, Shujing He, Leixin Chang +4

Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China · School of Mechanical Engineering and ZJU-UIUC Institute, Zhejiang University, Zhejiang, China · School of Mechanical Engineering, Tianjin University, Tianjin, 300072, China

Endoscopy Robot Policy Learning Robot Navigation Action Chunking Robotic Perception Robotic Endoscopy

Sep 29, 2026

EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior

Dynamic endoscopic reconstruction is fundamental to robotic surgery and computer-assisted interventions. While 3D Gaussian Splatting (3DGS) realises real-time rendering, its application to deformable intraoperative environments remains constrained by spurious geometry and varying illuminations. To address these limitations, we introduce EndoPrior-GS, a novel pipeline that explicitly couples frame-extracted vision heuristics and estimated depth maps. EndoPrior-GS derives a joint texture prior from a tool-filtered valid tissue mask, a non-specular photometric filter, and anatomical structural salience, yielding a probability map that guides primitive initialisation and subsequent density control. The prior is further extended to the temporal domain through a texture-aware term that dynamically weighs pairwise primitive contributions during training. We conduct extensive experiments on benchmark datasets EndoNeRF and SCARED, and the obtained results show that our method EndoPrior-GS reduces Flow Error by 27.7% and 25.8% over the representative approaches while preserving competitive rendering quality and real-time rendering speed. Our project website is available at https://jiaqi-huang-77.github.io/EndoPrior-GS/.

Jiaqi Huang, Shidong Wang, Tong Xin +1

School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK · School of Computing, Newcastle University, Newcastle upon Tyne NE4 5TG, UK

3D Gaussian Splatting Medical Imaging Dynamic Scene Reconstruction Deformable Shape Reconstruction Endoscopic 3D Reconstruction Robotic Endoscopy

Sep 19, 2026

ARCGym: Benchmarking Deep Reinforcement Learning in Autonomous Robotic Colonoscopy

Simulations for learning-based autonomous colonoscopic navigation focus mainly on fully actuated capsule robots, failing to capture the contact-rich navigation of long and flexible clinical colonoscopes. We present the Autonomous Robotic Colonoscopy Gym (ARCGym), an open-source reinforcement learning environment and benchmark for image-based navigation in clinically derived deformable colon anatomies. ARCGym supports multiple types of colonoscope robots, spanning capsule robots and flexible endoscopes, with this work focusing on flexible endoscopes including magnetic-driven tip actuation and clinically used proximally translational actuation. This work includes five CT-reconstructed colons representing typical clinical scenarios, a set of clinically meaningful navigation subtasks, and unified success metrics. We introduce a reward combining depth-based lumen alignment with a lumen-visibility score to improve learning under occlusions. Experiments across tasks, robots, and anatomies show that autonomous navigation remains challenging for both magnetic-driven and proximal-insertion flexible robots, with proximal-insertion actuation remaining an open problem.

Guanglin Ji, Martina Finocchiaro, Kenny Erleben +1

Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

Robotic Endoscopy RL Benchmarks Endoscopy Robotics Simulation Robotic RL Surgical Robotics

Sep 3, 2026

A Multi-Vine Soft Robot Enabling Accessible Working Channel and Steering

Soft eversion robots, also known as vine robots, have attracted growing interest for navigation and inspection tasks, including minimally invasive medical applications [1]. A vine robot consists of a thin, flexible, inextensible tube folded inward that everts and grows forward when pressurized. This tip-growth enables navigation with minimal friction, making vine robots well suited for complex environments such as the human colon [2]. While their inherent softness allows passive conforma- tion to curved pathways in confined spaces, navigation performance strongly depends on environmental inter- actions, including contact angle and the length of un- constrained deployed material [3], [4]. Sharp directional changes, such as those in the sigmoid colon, often limit passive growth and necessitate active steering. Existing solutions include distributed artificial muscles [5] or dedicated tip-based steering mechanisms [6]. In addition, many applications require payload delivery, such as sensors and tools [7], [8]. Within the ERC Synergy project EndoTheranostics, this motivates the development of vine robots capable of delivering micro- surgical tools during growth. Prior work has integrated working channels within the vine body [8], [9], but these approaches constrain tool size, introduce friction, and limit access to the environment to the robot tip. In this work, we propose a multi-vine architecture in which two vine robots are coupled to an externally integrated working channel via soft mounting tips [10]. Independent vine actuation enables active tip steering while advancing the working channel without embed- ding it within the vine bodies Figure 1. Experiments demonstrate sharp steering of nearly 90 degrees during growth, highlighting the potential of this architecture for versatile medical and non-medical applications.

Reza Kashef, Cem Suulker, Mohammad Sheikh Sofla +1

School of Engineering and Materials Science, Queen Mary University of London, United Kingdom.

Robotic Endoscopy Soft Robotics Surgical Robotics

Aug 22, 2026

EndoNav: Semantic-to-Geometric Grounding for Language-Guided Robotic Endoscopic Examination

Minimally invasive procedures performed within confined anatomical spaces depend on continuous endoscopic visualization. Current robotic endoscope systems can stabilize or reposition an endoscope, but they do not possess relevant context to provide effective visualization assistance. We present EndoNav, an anatomy-grounded natural-language framework that translates high-level surgeon commands into autonomous endoscopic visualization behaviors within patient-specific sinonasal anatomy. Spoken surgeon commands are transcribed and interpreted by an endoscopic viewpoint agent conditioned on a patient-specific anatomical scene representation. Rather than generating robot motion directly, the viewpoint agent generates structured visualization objectives that are converted into target viewpoints and inspection trajectories, which are then executed through geometry-constrained endoscope motion planning and joint-space control. We evaluate EndoNav using a structured three-pass sinus examination across three CT-derived anatomical models. For one cadaveric specimen, autonomous visualization is compared with sinus examinations performed by two resident surgeons. EndoNav achieved mean visualization IoUs of 87.04% and 84.37% relative to the two surgeon examinations, compared with an inter-surgeon IoU of 87.44%, while recovering 92.91% and 93.20% of surgeon-observed anatomical surfaces, respectively. These results demonstrate the feasibility of grounding high-level anatomical commands into patient-specific geometric objectives and translating them into anatomically constrained robotic visualization behaviors.

Jecia Z. Y. Mao, Hisashi Ishida, Kathryn Jung +3

Robotic Endoscopy Surgical Robotics Language-Conditioned Robot Control

Aug 2, 2026

EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation

Autonomous endoscopic navigation can reduce clinicians' operational burden, yet robust control remains challenging due to tissue deformation, transient occlusions, and rapidly changing viewpoints. Existing learning-based policies typically predict actions from current observations without explicitly modeling future dynamics, limiting their robustness and reliability in safety-critical settings. World Action Models (WAMs) offer a promising alternative by coupling predictive visual dynamics with action generation, but extending them to robotic endoscopy remains challenging due to limited training data, restricted viewpoint diversity, deformable anatomy, and high inference latency. We present EndoWAM, which is, to our knowledge, the first WAM for generalizable robotic endoscopic navigation. EndoWAM introduces future grounding, which predicts task-relevant target regions in future observations from intermediate denoising features of a video world model. Specifically, EndoWAM couples a lightweight diffusion transformer for future target-region prediction with a discrete action expert through a shared predictive representation. This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes while enabling real-time control in a single denoising pass. We further introduce EndoMotion, a robotic endoscopic motion dataset spanning three anatomically distinct procedures: ureteroscopy, esophagoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). EndoWAM consistently outperforms all baselines and alternative grounding strategies, while demonstrating strong zero-shot generalization to unseen viewpoints, environments, and targets. These results establish EndoWAM as a predictive, target-grounded framework for accurate, generalizable, and long-horizon navigation in visually constrained endoscopic environments.

Jinsong Lin, Zikang Pan, Wanhao Liu +10

CUHK · SLAI · WFU +3

World Models for Robotics Robotic Endoscopy World Action Models Surgical Robotics

Jun 22, 2026

BiliVLA: Scene-Aware Vision-Language-Action Model with Reinforcement Learning for Autonomous Biliary Endoscopic Navigation

Endoscopic retrograde cholangiopancreatography (ERCP) demands precise endoscopic navigation and stable biliary cannulation within a narrow monocular field characterized by specular reflections, partial occlusions, and frequent tissue contact. Although recent robotic systems and vision-based assistance techniques improve operator ergonomics and provide perceptual cues, their performance degrades under pronounced anatomical variability and safety-critical visual artifacts, which hinders reliable autonomy in cannulation-grade procedures. Here, we present BiliVLA, a scene-aware Vision-Language-Action (VLA) framework that formulates biliary endoscopic navigation as an instruction-conditioned visuomotor learning problem. Given an endoscopic observation and a stage-specific language instruction, BiliVLA jointly predicts the target category, a grounded bounding box, and a discrete three-degree-of-freedom (3-DoF) motor command for a continuum endoscope. The proposed framework incorporates scene-aware supervision to improve semantic target consistency and safety-aware recovery supervision to induce conservative retreat behaviors under luminal wall contact. A key component of BiliVLA is a two-stage training paradigm that combines grounding-enhanced supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO), thereby improving action reliability and decision consistency during closed-loop navigation. Across three ERCP subtasks, BiliVLA achieves the best overall performance in physical phantom experiments, with a total mIoU of 0.9625, an overall action precision of 91.96%, and an overall success rate (SR) of 84.85%. These results indicate that integrating semantic grounding, scene-aware learning, and reward-guided optimization strengthens perception--action alignment and enables more robust autonomous biliary endoscopic navigation.

Jinsong Lin, Chi Kit Ng, Zhiyong Xiong +8

The Chinese University of Hong Kong, Hong Kong SAR, China. · The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. · University of Cambridge, Cambridge, United Kingdom. +2

Endoscopy Robotic Endoscopy Visuomotor Policy Learning Vision-Language-Action Models Surgical Robotics

Apr 4, 2026

OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research

Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled dynamics of operator control, instrument motion, and visual feedback. This gap restricts reproducible closed-loop research in robotic colonoscopy, medical imaging, and emerging vision-language-action (VLA) learning paradigms. To address this challenge, we present OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. We experimentally validated motion consistency and quantified cross-modal latency across sensing streams. Using this platform, we collected a multimodal dataset comprising 1,894 teleoperated episodes ~19 hours across 10 structured task variations of routine navigation, failure events, and recovery behaviors. By unifying open hardware and an aligned multimodal dataset, OpenRC provides a reproducible foundation for research in multimodal robotic colonoscopy and surgical autonomy. Both the hardware design and dataset are available at https://github.com/artslab2019/openrc-robotic-colonoscopy. The dataset has been contributed to the Open-H-Embodiment Initiative by NVIDIA.

Siddhartha Kapuria, Mohammad Rafiee Javazm, Naruhiko Ikoma +4

Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, 78712, TX, USA · Department of Surgical Oncology, Division of Surgery, The University of Texas MD Anderson Cancer Center, Houston, 77030, TX, USA · Department of Surgery & Perioperative Care, Dell Medical School, The University of Texas at Austin, Austin, 78712, TX, USA +1

Robotic Endoscopy Surgical Robotics Robotic Data Collection Robot Teleoperation

Mar 26, 2026

Seeing Through Smoke: Surgical Desmoking for Improved Visual Perception

Minimally invasive and robot-assisted surgery relies heavily on endoscopic imaging, yet surgical smoke produced by electrocautery and vessel-sealing instruments can severely degrade visual perception and hinder vision-based functionalities. We present a transformer-based surgical desmoking model with a physics-inspired desmoking head that jointly predicts smoke-free image and corresponding smoke map. To address the scarcity of paired smoky-to-smoke-free training data, we develop a synthetic data generation pipeline that blends artificial smoke patterns with real endoscopic images, yielding over 80,000 paired samples for supervised training. We further curate, to our knowledge, the largest paired surgical smoke dataset to date, comprising 5,817 image pairs captured with the da Vinci robotic surgical system, enabling benchmarking on high-resolution endoscopic images. Extensive experiments on both a public benchmark and our dataset demonstrate state-of-the-art performance in image reconstruction compared to existing dehazing and desmoking approaches. We also assess the impact of desmoking on downstream stereo depth estimation and instrument segmentation, highlighting both the potential benefits and current limitations of digital smoke removal methods.

Jingpei Lu, Fengyi Jiang, Xiaorui Zhang +2

Intuitive Surgical, Sunnyvale, CA 94086, USA

Medical Imaging Physics-Guided Image Restoration Robotic Endoscopy Surgical Robotics

Mar 9, 2026

EndoSERV: A Vision-based Endoluminal Robot Navigation System

Robot-assisted endoluminal procedures are increasingly used for early cancer intervention. However, the intricate, narrow and tortuous pathways within the luminal anatomy pose substantial difficulties for robot navigation. Vision-based navigation offers a promising solution, but existing localization approaches are error-prone due to tissue deformation, in vivo artifacts and a lack of distinctive landmarks for consistent localization. This paper presents a novel EndoSERV localization method to address these challenges. It includes two main parts, i.e., SEgment-to-structure and Real-to-Virtual mapping, and hence the name. For long-range and complex luminal structures, we divide them into smaller sub-segments and estimate the odometry independently. To cater for label insufficiency, an efficient transfer technique maps real image features to the virtual domain to use virtual pose ground truth. The training phases of EndoSERV include an offline pretraining to extract texture-agnostic features, and an online phase that adapts to real-world conditions. Extensive experiments based on both public and clinical datasets have been performed to demonstrate the effectiveness of the method even without any real pose labels.

Junyang Wu, Fangfang Xie, Minghui Zhang +4

Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China, 200240. · Shanghai Chest Hospital, Shanghai, China.

Robot Navigation Robot Localization Robotic Endoscopy Visual Odometry

Mar 9, 2026

Long-Short Term Agents for Pure-Vision Bronchoscopy Robotic Autonomy

Accurate intraoperative navigation is essential for robot-assisted endoluminal intervention, but remains difficult because of limited endoscopic field of view and dynamic artifacts. Existing navigation platforms often rely on external localization technologies, such as electromagnetic tracking or shape sensing, which increase hardware complexity and remain vulnerable to intraoperative anatomical mismatch. We present a vision-only autonomy framework that performs long-horizon bronchoscopic navigation using preoperative CT-derived virtual targets and live endoscopic video, without external tracking during navigation. The framework uses hierarchical long-short agents: a short-term reactive agent for continuous low-latency motion control, and a long-term strategic agent for decision support at anatomically ambiguous points. When their recommendations conflict, a world-model critic predicts future visual states for candidate actions and selects the action whose predicted state best matches the target view. We evaluated the system in a high-fidelity airway phantom, three ex vivo porcine lungs, and a live porcine model. The system reached all planned segmental targets in the phantom, maintained 80% success to the eighth generation ex vivo, and achieved in vivo navigation performance comparable to the expert bronchoscopist. These results support the preclinical feasibility of sensor-free autonomous bronchoscopic navigation.

Junyang Wu, Mingyi Luo, Fangfang Xie +7

Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, 200240, China. · Shanghai Chest Hospital, Shanghai, 10587, China.

Robot Navigation Robotic Endoscopy Vision-Based Robot Control Endoscopy Robot Task Planning

Feb 17, 2026

OSCAR: An Ovipositor-Inspired Self-Propelling Capsule Robot for Colonoscopy

Self-propelling robotic capsules eliminate shaft looping of conventional colonoscopy, reducing patient discomfort. However, reliably moving within the slippery, viscoelastic environment of the colon remains a significant challenge. We present OSCAR, an ovipositor-inspired self-propelling capsule robot that translates the transport strategy of parasitic wasps into a propulsion mechanism for colonoscopy. OSCAR mechanically encodes the ovipositor-inspired motion pattern through a spring-loaded cam system that drives twelve circumferential sliders in a coordinated, phase-shifted sequence. By tuning the motion profile to maximize the retract phase relative to the advance phase, the capsule creates a controlled friction anisotropy at the interface that generates net forward thrust. We developed an analytical model incorporating a Kelvin-Voigt formulation to capture the viscoelastic stick--slip interactions between the sliders and the tissue, linking the asymmetry between advance and retract phase durations to mean thrust, and slider-reversal synchronization to thrust stability. Comprehensive force characterization experiments in ex-vivo porcine colon revealed a mean steady-state traction force of 0.85 N, closely matching the model. Furthermore, experiments confirmed that thrust generation is speed-independent and scales linearly with the phase asymmetry, in agreement with theoretical predictions, underscoring the capsule's predictable performance and scalability. In locomotion validation experiments, OSCAR demonstrated robust performance, achieving an average speed of 3.08 mm/s, a velocity sufficient to match the cecal intubation times of conventional colonoscopy. By coupling phase-encoded friction anisotropy with a predictive model, OSCAR delivers controllable thrust generation at low normal loads, enabling safer and more robust self-propelling locomotion for robotic capsule colonoscopy.

Mostafa A. Atalla, Anand S. Sekar, Remi van Starkenburg +4

M. A. Atalla, A. S. Sekar, R. van Starkenburg, D. J. Jager, A. Sakes, and P. Breedveld are with the Department of BioMechanical Engineering, Faculty of Mechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands. · M. A. Atalla and M. Wiertlewski are with the Department of Cognitive Robotics, Faculty of Mechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands.

Robotic Endoscopy Robot Locomotion Endoscopy Surgical Robotics

Feb 16, 2026

Real-time Monocular 2D and 3D Perception of Endoluminal Scenes for Controlling Flexible Robotic Endoscopic Instruments

Endoluminal surgery offers a minimally invasive option for early-stage gastrointestinal and urinary tract cancers but is limited by surgical tools and a steep learning curve. Robotic systems, particularly continuum robots, provide flexible instruments that enable precise tissue resection, potentially improving outcomes. This paper presents a visual perception platform for a continuum robotic system in endoluminal surgery. Our goal is to utilize monocular endoscopic image-based perception algorithms to identify position and orientation of flexible instruments and measure their distances from tissues. We introduce 2D and 3D learning-based perception algorithms and develop a physically-realistic simulator that models flexible instruments dynamics. This simulator generates realistic endoluminal scenes, enabling control of flexible robots and substantial data collection. Using a continuum robot prototype, we conducted module and system-level evaluations. Results show that our algorithms improve control of flexible instruments, reducing manipulation time by over 70% for trajectory-following tasks and enhancing understanding of surgical scenarios, leading to robust endoluminal surgeries.

Ruofeng Wei, Kai Chen, Yui Lun Ng +7

Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China. · Department of Mechanical Engineering, The University of Hong Kong, China. · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, and also with the Department of Mechanical Engineering, The University of Hong Kong.

Robotic Endoscopy Robotic Perception Surgical Robotics Continuum Robot Control Object Pose Estimation Monocular 3D Object Detection