Controlling continuum robots' shape to operate in confined, enclosed environments is challenging, since it cannot rely on external observation. Existing embedded shape sensing methods, although they can provide shape reconstruction, are insufficient to enable real-time shape control due to limitations in reconstruction accuracy, computational speed, and compatibility with shape-state representation. In this work, we present an embedded shape-sensing framework that uses readily available IMU and actuator feedback to reconstruct the continuum-robot shape with a neural network, providing accurate shape feedback at 400Hz for physical closed-loop control. We systematically investigate four key reconstruction design choices: shape-state representation, network architecture, training objective, and sensing inputs, through more than 5,000 trained models on one- and three-segment continuum robots. The selected framework achieves global shape errors of 0.9%L and 1.7%L (of the robot length) for the one- and three-segment systems, respectively. In closed-loop control, one-segment tip tracking achieves a 6.1%L error and three-segment shape control achieves a 2.3%L global shape error, whereas IMU--PCC reconstruction using the same uncompensated measurements fails. Enclosed-space experiments further demonstrate operation without external observation. These results provide a practical framework for translating readily available embedded sensing into high-rate feedback for continuum robot closed-loop control.
Figures & tables
Fig. 1: Overview of the proposed embedded shape-sensing and control framework for continuum robots working in enclosed space. Readily available embedded sensing reconstructs the shape state x^k for closed-loop control without external observation. External motion capture provides ground-truth supervision only during training. Video: https://youtu.be/sBmrXCk2B-4 .
Ref.
Seg.
Sensor
Reconstructed Tip Error (% L )
Rate (Hz)
Closed-Loop Control
[ 23 ]
1
IMU
0.91
60
×
2
IMU
1.21
60
×
[ 10 ]
2
Strain
1.8
–
✓ (tip)
[ 9 ]
3
Strain
2.6
–
×
[ 24 ]
2
FBG + IMU
3.6
–
×
[ 5 ]
2
IMU
4.1
20
×
TABLE I: Comparison of embedded shape-sensing methods and their validation in closed-loop continuum-robot control.
Fig. 2: Three-segment continuum robot and embedded sensing system. (a) Physical prototype consisting of base, middle, and tip segments mounted on a three-layer motor bank. (b) Interlocking vertebrae and orthogonally arranged NiTi hinge. (c) Integrated IMU and motion-capture marker mount. (d) Communication architecture for seven IMUs and nine motors. (e) Tendon-routing configuration within each segment. (f) Three-layer motor bank with guiding pulleys and three motors per layer.
Fig. 3: Ablation study of embedded full-body shape reconstruction using eglobal for the one-segment (a)–(d) and three-segment (e)–(h) robots. The four columns evaluate (a),(e) network architecture, (b),(f) shape-state representation, (c),(g) training objective, and (d),(h) sensing inputs. Values above the box plots denote the mean global shape error in millimeters. S1–S4 denote the increasing sensing configurations: S1: encoder; S2: encoder + IMU; S3: encoder + IMU + torque; and S4: encoder + IMU + torque + input ( uk−1 ).
Fig. 4: Embedded full-body shape reconstruction on the one- and three-segment continuum robots. (a), (c) Reconstruction error distributions for the MLP, LSTM, and CC estimators on the one- and three-segment systems, respectively. (b), (d) Representative reconstructed configurations compared with the motion-capture reference.
Fig. 5: One-segment tip tracking of a “tmech” reference trajectory using external and embedded shape feedback. (a) Tip trajectories using MoCap, MLP, and LSTM feedback. (b) Corresponding tip tracking errors over time.
Fig. 6: Three-segment full-body shape control using external and embedded shape feedback. (a) Representative target shapes and steady-state robot configurations obtained using MoCap, MLP, and LSTM feedback. (b) Front and top views of the controlled robot for Shape 5. The target shape is shown as a semi-transparent overlay on the measured robot configuration. (c) Tip, global, and local shape-control RMSE over 10 randomly selected target shapes.
Fig. 7: Enclosed-space demonstrations using the proposed embedded shape sensing and control framework without external observation. (a)–(c) Obstacle slalom, 0–60 s, (d)–(f) confined-aperture traversal, 60–120 s, and (g)–(i) tip targeting, 120–180 s.
Continuum robots possess high flexibility and redundancy, making them well suited for safe interaction in complex environments, yet their continuous deformation and nonlinear dynamics pose fundamental challenges to perception, modeling, and control. Existing vision-based control approaches often rely on end-to-end learning, achieving shape regulation without explicit awareness of robot geometry or its interaction with the environment. Here, we introduce a shape-interpretable visual self-modeling framework for continuum robots that enables geometry-aware control. Robot shapes are encoded from multi-view planar images using a Bezier-curve representation, transforming visual observations into a compact and physically meaningful shape space that uniquely characterizes the robot's three-dimensional configuration. Based on this representation, neural ordinary differential equations are employed to self-model both shape and end-effector dynamics directly from data, enabling hybrid shape-position control without analytical models or dense body markers. The explicit geometric structure of the learned shape space allows the robot to reason about its body and surroundings, supporting environment-aware behaviors such as obstacle avoidance and self-motion while maintaining end-effector objectives. Experiments on a cable-driven continuum robot demonstrate accurate shape-position regulation and tracking, with shape errors within 1.56% of image resolution and end-effector errors within 2% of robot length, as well as robust performance in constrained environments. By elevating visual shape representations from two-dimensional observations to an interpretable three-dimensional self-model, this work establishes a principled alternative to vision-based end-to-end control and advances autonomous, geometry-aware manipulation for continuum robots.
Peng Yu, Xin Wang, Ning Tan
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China.
Continuum robots are well suited for gentle manipulation because of their inherent compliance and ability to adapt to complex environments. However, their continuously deformable structure makes accurate configuration estimation challenging, particularly when external vision systems are unavailable or obstructed. In this work, we present an embedded pose sensing framework that combines inertial measurement units (IMUs) and active magnetic fields to estimate the robot configuration without relying on external cameras. The angular measurements from the IMU and magnetic-field references are fused to improve local orientation estimation and reduce accumulated orientation error during operation. This pose sensing scheme achieves an update rate of 16.7~Hz, allowing real-time feedback. The proposed system is experimentally validated through closed-loop control, where the estimated robot configuration is used to maintain the end-effector at a desired position while interacting with an object. These results demonstrate the potential of distributed magnetic--inertial sensing for real-time pose estimation and closed-loop control of continuum robots.
Continuum robots enable smooth shape morphing and safe interaction in confined environments. However, most existing systems are task-specific and depend on external sensing infrastructure, limiting their adaptability and real-world deployment. This paper presents a self-contained modular continuum robotic platform that combines mechanical reconfigurability with onboard pose estimation. The robot is constructed from interchangeable continuum joints with analytically precomputed stiffness, allowing rapid assembly and direct programming of the robot shape. Proprioceptive sensing is achieved using magnetic sensors and a modular learning-based framework, where a single model is trained per joint and reused across configurations. The system is experimentally validated in real world, demonstrating self-sensing capabilities and adaptation without external tracking.