ARCGym: Benchmarking Deep Reinforcement Learning in Autonomous Robotic Colonoscopy

Guanglin Ji; Martina Finocchiaro; Kenny Erleben; Hang Yin

cs.ROSep 19, 2026

ARCGym: Benchmarking Deep Reinforcement Learning in Autonomous Robotic Colonoscopy

Authors: Guanglin Ji, Martina Finocchiaro, Kenny Erleben, Hang Yin

Organizations: Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

Abstract

Simulations for learning-based autonomous colonoscopic navigation focus mainly on fully actuated capsule robots, failing to capture the contact-rich navigation of long and flexible clinical colonoscopes. We present the Autonomous Robotic Colonoscopy Gym (ARCGym), an open-source reinforcement learning environment and benchmark for image-based navigation in clinically derived deformable colon anatomies. ARCGym supports multiple types of colonoscope robots, spanning capsule robots and flexible endoscopes, with this work focusing on flexible endoscopes including magnetic-driven tip actuation and clinically used proximally translational actuation. This work includes five CT-reconstructed colons representing typical clinical scenarios, a set of clinically meaningful navigation subtasks, and unified success metrics. We introduce a reward combining depth-based lumen alignment with a lumen-visibility score to improve learning under occlusions. Experiments across tasks, robots, and anatomies show that autonomous navigation remains challenging for both magnetic-driven and proximal-insertion flexible robots, with proximal-insertion actuation remaining an open problem.

Figures & tables

Fig. 1: Reinforcement learning framework for autonomous colonoscopy in ARCGym. An RL agent is trained to interact with the parallel deformable colon models based on the RGB observation and the reward derived from the depth image.

Fig. 2: Architecture of the Autonomous Robotic Colonoscopy Gym (ArcGym). (a–d) Clinical CT data are reconstructed into colon meshes, rendered soft-tissue models, and replicated into parallel simulation environments for scalable reinforcement learning training. The framework integrates deformable anatomical models and multiple robot actuation modes within a reinforcement learning environment built on Isaac Lab/Isaac Sim. Depth images are processed to compute lumen-based rewards for RL policy training.

Literature	Robot Type			Deformable	Parallel Env.	RL Benchmark
	Capsule	Long Flexible	dVRK	Deformable	Parallel Env.	RL Benchmark
SurRoL [ 7 ]	-	-	✓	-	-	✓
Surgical Gym [ 8 ]	-	-	✓	-	✓	✓
Sim4EndoR [ 18 ]	-	✓	-	✓	-	✓
LapGym [ 19 ]	-	-	✓	✓	✓	✓
CathSim [ 20 ]	-	✓	-	-	-	✓

TABLE I: Capability Comparison Between ARCGym and Existing Platforms

Fig. 3: Vision-based actor–critic learning pipeline in ARCGym. The agent observes stacked monocular RGB frames $(I_{t-n},\ldots,I_{t})$ , which are encoded by a CNN feature extractor and fed into a standard actor–critic architecture.

Fig. 4: Definition of navigation subtasks. Task 1: Rectum navigation; Task 2: Sigmoid navigation; Task 3: Relatively Straight Part Navigation (Descending); Task 4: Turning Behavior at the Splenic Flexure. Task 2 and Task 4 with sharp turns demonstrate the most challenging navigation behavior.

Fig. 5: Normalized reward and ROI alignment rate over training timesteps for four colonoscopy navigation tasks. (a)–(b) Magnetic-driven flexible endoscope. (c)–(d) Proximal-insertion flexible endoscope. Results indicate that Task 2 and Task 4 are the most challenging overall. Under the proximal-insertion endoscope, (c) and (d) demonstrate a learnable behavior, although the goal reaching success rate is reported all zeros in Table III .

Algorithm	Success Rate (%): Distal Magnetic-driven Endoscope				Success Rate (%): Proximal-insertion Endoscope
	Task 1	Task 2	Task 3	Task 4	All Tasks
PPO	20.0	0.0	40.0	0.0	0.0
Constrained PPO [ 3 ]	100.0	30.0	90.0	80.0	0.0
A2C	20.0	0.0	20.0	0.0	0.0
SAC	0.0	0.0	0.0	0.0	0.0

TABLE III: Evaluation of reinforcement learning algorithms on the designed tasks using the magnetic-driven flexible endoscope and proximal-insertion flexible endoscope. On-policy algorithms perform better, especially for constrained PPO, all tasks besides Task 2 can achieve high success rate. In contrast, the off-policy algorithm SAC fails to finish all tasks.

Fig. 6: Average reward ablation results on Colon 1 and Colon 2 across Task 1–Task 4. Mean center-alignment score $s_{c}$ is reported for PPO with different reward terms and constrained PPO with full reward.

Fig. 7: Simulation results. (a)-(b) Deformation caused by interaction; (c)-(d) Proximal-insertion and $\alpha$ loop formation; (e) Rectum; (f) Sigmoid; (g) Descending Colon; (h) Splenic Flexure; (i)-(l) Example of Colon Deformation.

$E$ (MPa)	Poisson Ratio $\nu$	Mean $s_{c}$ Score	ROI
0.5	0.45	0.855	0.700
0.5	0.49	0.846	0.600
5.0	0.45	0.832	0.600
5.0	0.49	0.826	0.550
10.0	0.45	0.837	0.600
10.0	0.49	0.830	0.550

TABLE IV: Material sensitivity on Colon 1. Results are averaged over Tasks 1–4.

Task	SR(%)			Mean Step Reward
Task	Sparse	Default	Dense	Sparse	Default	Dense
T1	100.0	100.0	100.0	0.593	0.696	0.695
T2	16.7	30.0	80.0	-0.377	0.534	0.195
T3	100.0	90.0	60.0	0.672	0.593	0.677
T4	100.0	80.0	100.0	0.589	0.622	0.590

TABLE V: Per-task anchor-point sensitivity results. Task 2 corresponds to sigmoid-colon navigation.

Colon	Magnetic-driven				Proximal-insertion
	T1	T2	T3	T4	All Tasks
Colon 3	✓	10%	50%	20%	-
Colon 4	✓	20%	70%	20%	-
Colon 5	-	-	50%	-	-

TABLE VI: Cross-colon test using constrained PPO with two flexible colonoscopes. “T” is short for “Task”. ✓ represents all trials succeed, and “-” represents all trials fail.

Robotic Endoscopy

See all 17 papers

ColoACT: Multi-Cue Action Chunking for Smooth Autonomous Colon Navigation on a Self-Propelled Endoscopic Robot
Oct 1, 2026Jian Hu, Shujing He, Leixin Chang +4Endoscopy Robot Policy Learning
Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China · School of Mechanical Engineering and ZJU-UIUC Institute, Zhejiang University, Zhejiang, China · School of Mechanical Engineering, Tianjin University, Tianjin, 300072, China
Anatomical Landmark-Guided Deep Reinforcement Learning for Autonomous Gastric Navigation
May 8, 2026Haoxuan Wu, Sishen Yuan, Haitao Gao +3RL Control Endoscopy
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research
Apr 4, 2026Siddhartha Kapuria, Mohammad Rafiee Javazm, Naruhiko Ikoma +4Robotic Endoscopy Surgical Robotics
Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, 78712, TX, USA · Department of Surgical Oncology, Division of Surgery, The University of Texas MD Anderson Cancer Center, Houston, 77030, TX, USA · Department of Surgery & Perioperative Care, Dell Medical School, The University of Texas at Austin, Austin, 78712, TX, USA +1

Oct 1, 2026J/K move · Enter open · S save

ColoACT: Multi-Cue Action Chunking for Smooth Autonomous Colon Navigation on a Self-Propelled Endoscopic Robot

Jian Hu, Shujing He, Leixin Chang, Zongze Li+3

Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China · School of Mechanical Engineering and ZJU-UIUC Institute, Zhejiang University, Zhejiang, China · School of Mechanical Engineering, Tianjin University, Tianjin, 300072, China

Endoscopy Robot Policy Learning Robot Navigation Action Chunking Robotic Perception Robotic Endoscopy

Open paper AskPDF arXiv

Autonomous colonoscopic navigation can reduce operator burden and the risk of loop formation or tissue trauma, but remains challenging due to deformable anatomy, weak-texture and specular endoscopic visuals, and contact-rich viscoelastic interactions. Existing methods either rely on geometry-driven pipelines, which are efficient and interpretable yet brittle due to manually engineered features and switching logic, or adopt learning-based policies, whose inferred depth/geometry can become temporally inconsistent or overly smooth under weak texture and specular highlights while simulation-trained variants (e.g., deep reinforcement learning) may further suffer from a sim-to-real gap. We propose ColoACT, an autonomous navigation system that integrates an RGB-D-E based Action Chunking Transformer policy (ColoACT policy) for a compact self-propelled Bevel-Gear-Based Endoscopic Robot (BGER). The ColoACT policy augments RGB with estimated relative depth and a gradient-based pseudo-elevation map to enhance fold-ridge saliency and other high-frequency geometric cues, and enables smooth continuous control of the BGER by predicting overlapping action chunks and fusing them via temporal ensembling. In different ex-vivo porcine colons (approximately 60 cm), our system achieves success rates of 85.4% and 72.5% in straight and curved segments, respectively, and achieves 70% success in 90-degree turns and 60% in double-bend sequences, with feasibility further demonstrated in challenging triple-bend segments. The project page is available at: https://Adamhu1.github.io/ColoACT/.

Abstract

Robotic Endoscopy

Explore similar work

ColoACT: Multi-Cue Action Chunking for Smooth Autonomous Colon Navigation on a Self-Propelled Endoscopic Robot

Anatomical Landmark-Guided Deep Reinforcement Learning for Autonomous Gastric Navigation

OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research