OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research
Organizations: Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, 78712, TX, USA · Department of Surgical Oncology, Division of Surgery, The University of Texas MD Anderson Cancer Center, Houston, 77030, TX, USA · Department of Surgery & Perioperative Care, Dell Medical School, The University of Texas at Austin, Austin, 78712, TX, USA · School of Medicine and Health, Technical University of Munich, Munich, 80333, Germany
Abstract
Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled dynamics of operator control, instrument motion, and visual feedback. This gap restricts reproducible closed-loop research in robotic colonoscopy, medical imaging, and emerging vision-language-action (VLA) learning paradigms. To address this challenge, we present OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. We experimentally validated motion consistency and quantified cross-modal latency across sensing streams. Using this platform, we collected a multimodal dataset comprising 1,894 teleoperated episodes ~19 hours across 10 structured task variations of routine navigation, failure events, and recovery behaviors. By unifying open hardware and an aligned multimodal dataset, OpenRC provides a reproducible foundation for research in multimodal robotic colonoscopy and surgical autonomy. Both the hardware design and dataset are available at https://github.com/artslab2019/openrc-robotic-colonoscopy. The dataset has been contributed to the Open-H-Embodiment Initiative by NVIDIA.
Figures & tables
| Dataset | Video | Operator Actions | Actuation State | 6DoF Pose | Cross-Modal Alignment | Open Hardware |
| Clinical image/video datasets [ 4 , 13 , 6 , 5 ] | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| SimCol / SimCol3D [ 23 , 22 ] | ✓ | ✗ | ✗ | ✓ (synthetic) | ✓ | ✗ |
| EndoSLAM [ 21 ] | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ |
| EndoMapper [ 2 ] | ✓ | ✗ | ✗ | ✓ (simulated subset) | ✓ | ✗ |
| C3VDv2 [ 11 ] | ✓ | ✗ | ✗ | ✓ | ✓ | Phantom STL/CAD |
| Copenhagen Colonoscopy Coordinate DB [ 9 ] | ✗ | ✗ | ✗ | ✓ | Single-modality | ✗ |