Toward AI Autonomous Navigation for Mechanical Thrombectomy using Hierarchical Modular Multi-agent Reinforcement Learning (HM-MARL)

Harry Robertshaw; Nikola Fischer; Lennart Karstensen; Benjamin Jackson; Xingyu Chen; S. M. Hadi Sadati; Christos Bergeles; Alejandro Granados

cs.ROFeb 20, 2026

Toward AI Autonomous Navigation for Mechanical Thrombectomy using Hierarchical Modular Multi-agent Reinforcement Learning (HM-MARL)

Authors: Harry Robertshaw, Nikola Fischer, Lennart Karstensen, Benjamin Jackson, Xingyu Chen, S. M. Hadi Sadati, Christos Bergeles, Alejandro Granados,

Organizations: School of Biomedical Engineering & Imaging Sciences, Kings College London, London, UK · School of Engineering & Materials Science, Queen Mary University of London, London, UK

Abstract

Mechanical thrombectomy (MT) is typically the optimal treatment for acute ischemic stroke involving large vessel occlusions, but access is limited due to geographic and logistical barriers. Reinforcement learning (RL) shows promise in autonomous endovascular navigation, but generalization across 'long' navigation tasks remains challenging. We propose a Hierarchical Modular Multi-Agent Reinforcement Learning (HM-MARL) framework for autonomous two-device navigation in vitro, enabling efficient and generalizable navigation. HM-MARL was developed to autonomously navigate a guide catheter and guidewire from the femoral artery to the internal carotid artery (ICA). A modular multi-agent approach was used to decompose the complex navigation task into specialized subtasks, each trained using Soft Actor-Critic RL. The framework was validated in both in silico and in vitro testbeds to assess generalization and real-world feasibility. In silico, a single-vasculature model achieved 92-100% success rates on individual anatomies, while a multi-vasculature model achieved 56-80% across multiple patient anatomies. In vitro, both HM-MARL models successfully navigated 100% of trials from the femoral artery to the right common carotid artery and 80% to the right ICA but failed on the left-side vessel superhuman challenge due to the anatomy and catheter type used in navigation. This study presents the first demonstration of in vitro autonomous navigation in MT vasculature. While HM-MARL enables generalization across anatomies, the simulation-to-real transition introduces challenges. Future work will refine RL strategies using world models and validate performance on unseen in vitro data, advancing autonomous MT towards clinical translation.

Figures & tables

Figure 1: Full patient MT vasculature with sub-tasks labeled for the first phase of MT (navigating a guide wire and catheter from the common iliac artery the left or right ICA). Labels based on radiological left and right, where left sided anatomy is harder to navigate due to more acute angle of left common carotid artery (CCA) compared to the right. $A_{1}$ begins at the insertion point, hence no start area is labeled.

Figure 2: in vitro testbed showing the 3D phantom and tracking camera. Inset is a side view of the 3D phantom.

Figure 3: Concentric tube robot (CTR) manipulator used for in vitro experiments [ 29 ] .

Figure 4: HM-MARL. TSM is used to select $a_{t+1}$ based on $s_{t}$ .

Task	SA-RL-1			HM-MARL-1			HM-MARL-10
Task	SR (%)	PT (s)	PR (%)	SR (%)	PT (s)	PR (%)	SR (%)	PT (s)	PR (%)
$A_{1,2}$ (R)	$98\pm 14$	$33.2\pm 18.3$	$47\pm 14$	$100\pm 0$	$40.1\pm 33.8$	$-$	$100\pm 0$	$77.8\pm 51.4$	$-$
$A_{2,3}$ (R)	$78\pm 42$	$50.1\pm 43.6$	$22\pm 13$	$100\pm 0$	$22.3\pm 9.9$	$-$	$100\pm 0$	$58.1\pm 38.0$	$-$
$A_{1,2,3}$ (R)	$0\pm 0$	$-$	$18\pm 21$	$100\pm 0$	$44.3\pm 31.1$	$-$	$90\pm 30$	$53.8\pm 28.3$	$60\pm 26$
$A_{1,2}$ (L)	$18\pm 39$	$106.0\pm 41.9$	$66\pm 21$	$100\pm 0$	$45.9\pm 29.6$	$-$	$100\pm 0$	$44.0\pm 31.9$	$-$
$A_{2,3}$ (L)	$70\pm 46$	$37.5\pm 28.4$	$28\pm 27$	$94\pm 24$	$42.8\pm 31.0$	$70\pm 16$	$76\pm 43$	$35.2\pm 23.1$	$47\pm 14$

Table 1: in silico evaluation on a single patient vasculature, where each task was independent of each other. The reported values are mean

\pm

standard deviation (standard deviation values may exceed logical bounds due to the statistical calculation). ‘

-

’ denotes cell is not applicable (PT is not recorded when SR is 0%, and PR is not recorded when SR is 100%).

Task	SR (%)	PT (s)	PR (%)
$A_{1,2}$ (R)	$98\pm 14$	$68.0\pm 48.3$	$85\pm 0$
$A_{2,3}$ (R)	$92\pm 27$	$33.1\pm 35.5$	$74\pm 12$
$A_{1,2,3}$ (R)	$66\pm 48$	$61.1\pm 41.2$	$66\pm 23$
$A_{1,2}$ (L)	$62\pm 49$	$29.1\pm 14.8$	$69\pm 11$
$A_{2,3}$ (L)	$66\pm 48$	$43.5\pm 32.3$	$34\pm 21$
$A_{1,2,3}$ (L)	$46\pm 50$	$74.6\pm 55.7$	$57\pm 16$

Table 2: in silico evaluation of the HM-MARL-10 model across ten different patient vasculatures, where each task was independent of each other.

Task	SA-RL-1 Steps	HM-MARL-1 Steps	HM-MARL-10 Steps
$A_{1}$	$-$	$0.75\times 10^{6}$	$0.75\times 10^{6}$
$A_{2R}$	$-$	$0.75\times 10^{6}$	$8.25\times 10^{6}$
$A_{2L}$	$-$	$5.00\times 10^{6}$	$4.25\times 10^{6}$
$A_{3R}$	$-$	$1.50\times 10^{6}$	$6.75\times 10^{6}$
$A_{3L}$	$-$	$0.75\times 10^{6}$	$1.25\times 10^{6}$
$A_{1,2}$	$4.5\times 10^{6}$	$-$	$-$

Table 3: Exploration steps required to train in silico agents.

1\times 10^{6}

exploration steps

\approx 80

\mathrm{min}

. ‘

-

’ denotes cell is not applicable as training for a particular task was not required, due to the training procedure.

Task	HM-MARL-1			HM-MARL-10
Task	SR (%)	PT (s)	PR(%)	SR (%)	PT (s)	PR (%)
$A_{1,2}$ (R)	$100\pm 0$	$249\pm 72$	$-$	$100\pm 0$	$261\pm 62$	$-$
$A_{2,3}$ (R)	$70\pm 46$	$421\pm 67$	$81\pm 4$	$80\pm 40$	$386\pm 68$	$87\pm 7$
$A_{1,2}$ (L)	$0\pm 0$	$-$	$73\pm 4$	$0\pm 0$	$-$	$75\pm 3$
$A_{2,3}$ (L)	$0\pm 0$	$-$	$32\pm 11$	$0\pm 0$	$-$	$36\pm 7$

Table 4: in vitro evaluation of HM-MARL-1 and HM-MARL-10, where each task was independent of each other. ‘

-

’ denotes cell is not applicable (PT is not recorded when SR is 0%, and PR is not recorded when SR is 100%).

Figure 5: in vitro navigation paths for $A_{1,2}$ and $A_{2,3}$ . Successful trajectories are for right-sided navigation tasks, while unsuccessful trajectories are for left-sided navigation tasks.

Figure 6: Selected frames from recording showing autonomous in vitro navigation for a) $A_{1,2}$ (R), b) and $A_{2,3}$ (R).

Robotic Endovascular Intervention

Robotic endovascular intervention uses robotic systems to navigate vascular instruments and perform procedures with greater precision, control, or autonomy.

See all 24 papers

Toward Safe Autonomous Robotic Endovascular Interventions using World Models
Apr 22, 2026Harry Robertshaw, Nikola Fischer, Han-Ru Wu +6World Models for Robotics Robotic Endovascular Intervention
Surgical & Interventional Engineering, School of Biomedical Engineering & Imaging Sciences, King’s College London, UK · Department of Radiology, National Taiwan University Hospital, Taiwan · Department of Neuroradiology, King’s College Hospital, UK
Vascular Geometry Characterization for AI-Based Endovascular Navigation
Jul 10, 2026Han-Ru Wu, Harry Robertshaw, Lisa Dwyer-Joyce +2Reinforcement Learning Medical Image Analysis
Department of Medical Imaging Science National Taiwan University Hospital Taiwan · Surgical & Interventional Engineering School of Biomedical Engineering & Imaging Sciences Kings College London UK · Chelsea & Westminster Hospital UK
A Position Statement on Endovascular Models and Effectiveness Metrics for Mechanical Thrombectomy Navigation, on behalf of the Stakeholder Taskforce for AI-assisted Robotic Thrombectomy (START)
Mar 30, 2026Harry Robertshaw, Anna Barnes, Phil Blakelock +20Robotic Endovascular Intervention Surgical Robotics
School of Biomedical Engineering & Imaging Sciences, Kings College London, London, UK · King’s Technology Evaluation Centre (KiTEC), Kings College London, London, UK · Patient & Public Involvement, Kings College London, London, UK +18

Apr 22, 2026J/K move · Enter open · S save

Toward Safe Autonomous Robotic Endovascular Interventions using World Models

Harry Robertshaw, Nikola Fischer, Han-Ru Wu, Andrea Walker Perez+5

Surgical & Interventional Engineering, School of Biomedical Engineering & Imaging Sciences, King’s College London, UK · Department of Radiology, National Taiwan University Hospital, Taiwan · Department of Neuroradiology, King’s College Hospital, UK

World Models for Robotics Robotic Endovascular Intervention Surgical Robotics Model-Based RL

Open paper AskPDF arXiv

Autonomous mechanical thrombectomy (MT) presents substantial challenges due to highly variable vascular geometries and the requirements for accurate, real-time control. While reinforcement learning (RL) has emerged as a promising paradigm for the automation of endovascular navigation, existing approaches often show limited robustness when faced with diverse patient anatomies or extended navigation horizons. In this work, we investigate a world-model-based framework for autonomous endovascular navigation built on TD-MPC2, a model-based RL method that integrates planning and learned dynamics. We evaluate a TD-MPC2 agent trained on multiple navigation tasks across hold out patient-specific vasculatures and benchmark its performance against the state-of-the-art Soft Actor-Critic (SAC) algorithm agent. Both approaches are further validated in vitro using patient-specific vascular phantoms under fluoroscopic guidance. In simulation, TD-MPC2 demonstrates a significantly higher mean success rate than SAC (58% vs. 36%, p < 0.001), and mean tip contact forces of 0.15 N, well below the proposed 1.5 N vessel rupture threshold. Mean success rates for TD-MPC2 (68%) were comparable to SAC (60%) in vitro, but TD-MPC2 achieved superior path ratios (p = 0.017) at the cost of longer procedure times (p < 0.001). Together, these results provide the first demonstration of autonomous MT navigation validated across both hold out in silico data and fluoroscopy-guided in vitro experiments, highlighting the promise of world models for safe and generalizable AI-assisted endovascular interventions.

Abstract

Robotic Endovascular Intervention

Explore similar work

Toward Safe Autonomous Robotic Endovascular Interventions using World Models

Vascular Geometry Characterization for AI-Based Endovascular Navigation

A Position Statement on Endovascular Models and Effectiveness Metrics for Mechanical Thrombectomy Navigation, on behalf of the Stakeholder Taskforce for AI-assisted Robotic Thrombectomy (START)