Live Thread Reconstruction for Autonomous Manipulation

Emma Huang; Omar Hernandez; Zih-Yun Chiu; Michael Yip

cs.ROOct 22, 2026

Live Thread Reconstruction for Autonomous Manipulation

Authors: Emma Huang, Omar Hernandez, Zih-Yun Chiu, Michael Yip

Organizations: Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA. · Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218 USA.

Abstract

Autonomous robotic surgery has drawn increased attention in recent years due to its potential to relieve surgeons of time-consuming, mentally straining, and tedious tasks, particularly during prolonged operations. One suitable task for automation is suturing, which involves tracking the thread as it deforms during needle retrieval, de-tangling, knot tightening, and other tasks. Unlike rigid-object manipulation, deformable thread manipulation has no fixed pose to track and effectively infinite degrees of freedom, making its configuration difficult to predict and requiring the thread's state to be re-estimated rather than assumed. At the same time, thread manipulations involve sequential motions that depend on past and current thread contact points, which must be accurately and continuously tracked. This paper presents an image-to-manipulation pipeline that concurrently generates live thread reconstruction while conducting safe and optimized sequential manipulations to achieve long-horizon tasks.

Figures & tables

Fig. 1 : The thread reconstruction remains consistent through common surgical thread manipulations such as knot tightening while handling occlusions from the tool and self intersections causing ambiguity.

Fig. 2 : The thread reconstruction is showcased on three tasks commonly seen in surgery including needle pick up, thread de-tangling, and knot tightening. For each task, the key object such as the needle, intersection, and knot shape the formation of manipulation motions.

	Static		Live
Setup	Easy+Medium	Hard	Easy+Medium	Hard
Needle Pick Up	18/20	7/10	13/15	14/15
Tightening	-	-	15/15	13/15

TABLE I : Manipulation Task Success Rates

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,671 papers

Reeling It In: Flexible Needle Pick Up via Thread Manipulation for Autonomous Suturing
Jul 28, 2026Emma Huang, Zih-Yun Chiu, Neelay Joglekar +2Robotic Grasping Robotic Manipulation
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA. · Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218 USA. · CMU Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA. +1
Video-to-Model: Automatic Modeling of Deformable Linear Objects
Oct 7, 2026Akshun Sharma, Kimia Forghani, Yancy Diaz-MercadoDeformable Object Simulation
Department of Mechanical Engineering, University of Maryland College Park, Maryland, USA
Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery
Jul 9, 2026Tianyi Song, Sierra Bonilla, Xinwei Ju +6Simultaneous Localization and Mapping 3D Gaussian Splatting
Department of Computer Science and UCL Hawkes Institute, University College London, London WC1E 6BT, UK · Intuitive Surgical, Inc., Sunnyvale, CA 94086, USA

Jul 28, 2026J/K move · Enter open · S save

Reeling It In: Flexible Needle Pick Up via Thread Manipulation for Autonomous Suturing

Emma Huang, Zih-Yun Chiu, Neelay Joglekar, Shanglei Liu+1

Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA. · Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218 USA. · CMU Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA.+1

Robotic Grasping Robotic Manipulation Surgical Robotics

Open paper AskPDF arXiv

Suture-needle pickup is necessary for autonomous suturing, as a needle can be unexpectedly dropped or strategically released to adjust the grasping configuration. Current methods for autonomous needle pickup typically guide a robot to move straight toward the needle and grasp it, limited to conditions where the needle is observable and directly approachable. In addition, grasping the needle lying on tissue can lead to the robot pinching nearby tissue or the needle jumping around due to its slippery surface, posing potential safety issues. This work proposes an autonomous framework that uses a suture thread as an assistive tool for indirect needle pickup, avoiding unnecessary tool-tissue contact and enabling pickup even when the needle is occluded or inaccessible. The framework spans the entire workflow, including thread and tissue reconstruction, safe grasp-point selection, stable thread lifting, and bimanual thread-following until securing needle grasping. The robot policies account for visual uncertainty to maximize robustness in real-world environments. We evaluate the proposed framework on a da Vinci Research Kit under various real-world conditions. The results demonstrate robust performance even with a challenging thread configuration or a non-approachable needle, closing the gap in applying autonomous robot policies to unstructured suturing environments.

Abstract

Robotic Manipulation

Explore similar work

Reeling It In: Flexible Needle Pick Up via Thread Manipulation for Autonomous Suturing

Video-to-Model: Automatic Modeling of Deformable Linear Objects

Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery